Multi-sensor fusion simulation fish underwater environment real-time monitoring method and simulation fish

Abstract

The application discloses a multi-sensor fusion simulation fish underwater environment real-time monitoring method, comprising the following steps: S1, deploying the simulation fish to the target underwater environment, initializing and collecting video stream, audio stream, water quality data and hydrological data; S2, respectively preprocessing the video stream, audio stream, water quality data and hydrological data; S3, fusing the preprocessed video stream, audio stream, water quality data and hydrological data to generate a fusion feature matrix; S4, compressing the fusion feature matrix to obtain final compressed data, and transmitting the final compressed data through underwater acoustic communication. Thus, the data quality and comprehensive utilization efficiency can be improved through targeted preprocessing, space-time correlation and multi-feature fusion, the data compression and verification design is adapted to underwater acoustic communication, the transmission effect is guaranteed, the split structure of the simulation fish is convenient to disassemble, maintain and move flexibly, and the integrated design improves the convenience and adaptability of monitoring.

Description

Technical Field

[0001] This invention relates to the field of simulated machinery, specifically to a method for real-time monitoring of the underwater environment of simulated fish using multi-sensor fusion and the simulated fish thereof. Background Technology

[0002] Underwater environmental monitoring is of great significance for water resource protection, ecological research, and other fields. Currently, collecting underwater environmental data through sensors is a common monitoring method. These sensors include equipment for image acquisition, devices for sound capture, and sensors for detecting water quality and hydrological parameters.

[0003] In terms of data processing, current technologies often preprocess data collected from multiple sensors individually before integration. This approach may result in insufficient temporal and spatial correlation between different types of data, affecting the comprehensive utilization of the data. Furthermore, issues such as noise and data redundancy in the underwater environment can also reduce the accuracy and effectiveness of monitoring data.

[0004] Regarding monitoring equipment, some underwater monitoring devices are structurally designed to be difficult to assemble and disassemble, hindering equipment maintenance and sensor replacement. Furthermore, some devices have limited mobility, making them ill-suited for complex underwater environments and potentially limiting monitoring range and efficiency. Summary of the Invention

[0005] The present invention aims to at least partially solve the technical problems in the above-mentioned technologies.

[0006] Therefore, this invention discloses a multi-sensor fusion simulation method for real-time monitoring of the underwater environment of fish, comprising the following steps:

[0007] S1: Deploy the simulated fish to the target underwater environment, initialize it, and collect video streams, audio streams, water quality data, and hydrological data;

[0008] S2: Preprocess the video stream, the audio stream, the water quality data, and the hydrological data respectively;

[0009] S3: The preprocessed video stream, audio stream, water quality data, and hydrological data are fused to generate a fusion feature matrix;

[0010] S4: The fused feature matrix is ​​compressed to obtain the final compressed data, and the final compressed data is transmitted through underwater acoustic communication.

[0011] In addition, the multi-sensor fusion simulation method for real-time monitoring of underwater fish environment disclosed in this invention may also have the following additional technical features:

[0012] Further, step S2 includes the following sub-steps:

[0013] S2.1: Perform spatiotemporal joint denoising on the video stream to obtain a denoised frame sequence, and perform inter-frame redundancy suppression on the denoised frame sequence to obtain a preprocessed video stream, specifically:

[0014]

[0015] in, For the video stream, The denoised frame sequence, For spatial domain Gaussian filtering, For time weighting coefficients, The preprocessed video stream, For norm functions, The difference threshold;

[0016] S2.2: Perform frequency domain transformation on the audio stream to obtain an audio frequency domain signal, perform adaptive noise band suppression on the audio frequency domain signal to obtain a denoised audio frequency domain signal, and perform time domain transformation on the denoised audio frequency domain signal to obtain a preprocessed audio signal, specifically:

[0017]

[0018] in, For the audio stream, The audio frequency domain signal, For Fourier transform operators, The denoised audio frequency domain signal. For frequency band weighting functions, To preset the noise frequency band threshold, The preprocessed audio signal;

[0019] S2.3: The water quality data is subjected to noise removal to obtain de-noising water quality data. Baseline drift correction is then performed on the de-noising water quality data to obtain preprocessed water quality data. Specifically:

[0020]

[0021] in, For the water quality data, The water quality data after the jump was removed. For the jump threshold, The water quality data is the result of preprocessing. To adjust the sliding window size;

[0022] S2.4: Perform high-frequency interference filtering on the hydrological data to obtain filtered hydrological data, and perform physical range constraint correction on the hydrological data to obtain preprocessed hydrological data, specifically as follows:

[0023]

[0024] in, For the aforementioned hydrological data, The filtered hydrological data, These are the filter coefficients. The preprocessed hydrological data, These are the upper and lower limits of the physical range of the parameter.

[0025] Further, step S3 includes the following sub-steps:

[0026] S3.1: The time reference of the preprocessed video stream, audio stream, water quality data and hydrological data is unified to obtain a time-synchronized signal set, and the time-synchronized signal set is spatially correlated to obtain a spatially correlated signal set;

[0027] S3.2: Extract the features from the spatially correlated signal set to obtain video low-frequency features, audio feature vectors, and environmental feature vectors;

[0028] S3.3: Obtain a fused feature matrix from the low-frequency features, the audio feature vector, and the environmental feature vector.

[0029] Further, step S3.1 includes the following sub-steps:

[0030] S3.1.1: Using the sampling frequency of the preprocessed video stream as a reference, linear interpolation is performed on the preprocessed audio stream, water quality data, and hydrological data to obtain time-synchronized audio stream, water quality data, and hydrological data. The preprocessed video stream, time-synchronized audio stream, water quality data, and hydrological data are then combined into a time-synchronized signal set, specifically:

[0031] ;

[0032] in, The audio stream is time-synchronized. The water quality data is synchronized with the time. The hydrological data is synchronized in time. For indicator functions, The set of signals for time synchronization;

[0033] S3.1.2: By simulating the movement trajectory of fish, the time-synchronized water quality data and hydrological data are mapped to the video space of the preprocessed video stream to obtain spatially correlated water quality data and hydrological data, specifically:

[0034]

[0035] in, The water quality data is spatially correlated. The hydrological data is spatially correlated. For the Dirac function, for Simulates the fish's position at all times. This refers to the signal set after spatial correlation.

[0036] Further, step S3.2 includes the following sub-steps:

[0037] S3.2.1: Extract the low-frequency video features of the video stream from the spatially correlated signal set using a two-dimensional Fourier transform, specifically:

[0038]

[0039] in, The low-frequency features of the video, For frequency domain coordinates, Low-frequency threshold;

[0040] S3.2.2: Calculate the short-time energy and zero-crossing rate of the time-synchronized audio stream to extract the audio feature vector, specifically:

[0041]

[0042] in, The audio feature vector, For window length, The sampling interval;

[0043] S3.2.3: Extract the spatially correlated water quality data and hydrological data to extract environmental feature vectors, specifically:

[0044]

[0045] in, This is the environmental feature vector.

[0046] Further, step S3.3 includes the following sub-steps:

[0047] S3.3.1: Calculate the weights of the low-frequency features of the video, the audio feature vector, and the environmental feature vector based on the signal-to-noise ratio to obtain a dynamic weight set;

[0048] S3.3.2: Generate a spatiotemporal fusion matrix based on the video low-frequency features, the audio feature vector, the environmental feature vector, and the dynamic weight set, specifically as follows:

[0049]

[0050] in, The spatiotemporal fusion matrix is... This refers to the dynamic weight set.

[0051] Furthermore, in step 4, when compressing the spatiotemporal fusion matrix, a parity check bit is added to the last bit of the encoding of each element.

[0052] To this end, the present invention discloses a simulated fish, which is a split structure. The whole is composed of three detachable parts: a head section, a body section, and a tail section. The head section, the body section, and the tail section are all symmetrical split structures, each of which independently includes two matching half-shells. The two half-shells of each part are interlocked along the left and right symmetrical planes to form a cavity structure.

[0053] Two matching connectors are provided at the connection between the fish body segment and the fish tail segment. They are fastened together to form a complete ring structure. After fastening, the connection between the fish body segment and the fish tail segment is closed and the fish body segment and the fish tail segment are fixed.

[0054] A pressure plate is provided in the cavity at the connection between the fish body segment and the fish tail segment;

[0055] The tail section is provided with a tail fin at its very end. Two other matching connectors are provided at the connection between the tail fin and the tail section. They are fastened together to form a complete ring structure. After fastening, the inner wall of the connector is pressed tightly against the outer wall of the tail section and the outer wall of the tail fin, respectively.

[0056] Side wings are provided on the left and right sides of the outer wall of the fish body section, and two swinging components are provided in the cavity of the fish body section.

[0057] In addition, the simulated fish disclosed in this invention may also have the following additional technical features:

[0058] Furthermore, the cavity of the fish head segment is equipped with an image sensor, an acoustic sensor, a water quality sensor, a hydrological sensor, a data processing module, an underwater acoustic communication module, and a power supply.

[0059] Furthermore, the swinging component is equipped with two cylinders and two servo motors. The cylinders are equipped with pistons, and the two servo motors drive the two pistons to move through a connecting rod structure.

[0060] The multi-sensor fusion simulation fish underwater environment real-time monitoring method and its simulated fish disclosed in this invention have at least the following beneficial effects:

[0061] In terms of monitoring methods, targeted preprocessing of video streams, audio streams, water quality data, and hydrological data, such as spatiotemporal domain joint denoising, inter-frame redundancy suppression, frequency domain noise suppression, and jump noise removal, effectively improves the quality of various types of data and reduces the impact of noise and redundant information.

[0062] By unifying the time reference and associating spatial coordinates with the preprocessed data, the temporal and spatial consistency of multi-sensor data is enhanced, laying the foundation for subsequent data fusion. The fusion of processed video low-frequency features, audio feature vectors, and environmental feature vectors to generate a fusion feature matrix improves the comprehensive utilization efficiency of multi-source data, enabling a more comprehensive reflection of underwater environmental information.

[0063] Compressing the fused feature matrix and adding parity bits reduces the amount of data transmitted while ensuring the accuracy of data transmission, thus adapting to the characteristics of underwater acoustic communication.

[0064] The modular design of the simulated fish structure allows for easy assembly and disassembly of the head, body, and tail sections, facilitating equipment maintenance and internal component replacement. The symmetrical semi-shell structure and connecting parts ensure structural stability and sealing. The oscillating components enhance the simulated fish's underwater maneuverability, enabling it to better adapt to different underwater environments and expand the monitoring range.

[0065] The fish head section integrates multiple sensors and processing and communication modules, realizing the integration of data acquisition, processing and transmission, and improving the real-time performance and convenience of monitoring.

[0066] Additional features and advantages of this invention will be set forth in the description which follows, or may be learned by practicing the invention. Attached Figure Description

[0067] The technical solution and beneficial effects of the present invention will become apparent and readily understood from the following description in conjunction with the accompanying drawings, wherein:

[0068] Figure 1 This is a flowchart illustrating the workflow of the multi-sensor fusion simulation method for real-time monitoring of the underwater environment of fish according to the present invention.

[0069] Figure 2 This is a schematic diagram of the structure of the simulated fish of the present invention;

[0070] Figure 3 This is another structural schematic diagram of the simulated fish of the present invention;

[0071] Figure 4 This is another structural schematic diagram of the simulated fish of the present invention;

[0072] Figure 5 This is another structural schematic diagram of the simulated fish of the present invention.

[0073] As shown in the figure:

[0074] 10-Fish head segment;

[0075] 20 - Fish body segments;

[0076] 30 - Fish tail segment;

[0077] 40 - Connector;

[0078] 50-Pressure plate;

[0079] 60-Caudal fin;

[0080] 70-Flank;

[0081] 80-Swing component;

[0082] 801-Cylinder, 802-Steering motor, 803-Piston, 804-First connecting rod, 805-Second connecting rod, 806-Bearing, 807-Pressure cap. Detailed Implementation

[0083] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0084] The following description, with reference to the accompanying drawings, outlines the multi-sensor fusion simulation fish method for real-time underwater environment monitoring and the simulated fish thereof.

[0085] like Figure 1 As shown, the multi-sensor fusion simulation method for real-time monitoring of the underwater environment of fish includes the following steps:

[0086] S1: Deploy the simulated fish to the target underwater environment, initialize it, and collect video streams, audio streams, water quality data, and hydrological data;

[0087] S2: Preprocess the video stream, audio stream, water quality data, and hydrological data respectively;

[0088] S3: Fuse the preprocessed video stream, audio stream, water quality data, and hydrological data to generate a fusion feature matrix;

[0089] S4: Compress the fused feature matrix to obtain the final compressed data, and transmit the final compressed data through underwater acoustic communication.

[0090] Step S2 includes the following sub-steps:

[0091] S2.1: Perform joint spatiotemporal denoising on the video stream to obtain a denoised frame sequence, and then perform inter-frame redundancy suppression on the denoised frame sequence to obtain a preprocessed video stream, specifically:

[0092]

[0093] in, For video stream, This is the denoised frame sequence. For spatial domain Gaussian filtering, For time weighting coefficients, For the preprocessed video stream, For norm functions, The difference threshold;

[0094] S2.2: Perform frequency domain transformation on the audio stream to obtain the audio frequency domain signal, perform adaptive noise band suppression on the audio frequency domain signal to obtain the denoised audio frequency domain signal, and perform time domain transformation on the denoised audio frequency domain signal to obtain the preprocessed audio signal, specifically:

[0095]

[0096] in, For audio streams, It is an audio frequency domain signal. For Fourier transform operators, The audio frequency domain signal after denoising. For frequency band weighting functions, To preset the noise frequency band threshold, This is the preprocessed audio signal;

[0097] S2.3: The water quality data is subjected to noise removal to obtain de-noising water quality data. Baseline drift correction is then applied to the de-noising water quality data to obtain pre-processed water quality data. Specifically:

[0098]

[0099] in, For water quality data, To remove the water quality data after the jump, For the jump threshold, For pretreated water quality data, To adjust the sliding window size;

[0100] S2.4: High-frequency interference filtering is applied to the hydrological data to obtain filtered hydrological data, and physical range constraint correction is applied to the hydrological data to obtain preprocessed hydrological data. Specifically:

[0101]

[0102] in, For hydrological data, The filtered hydrological data, These are the filter coefficients. For preprocessed hydrological data, These are the upper and lower limits of the physical range of the parameter.

[0103] Step S3 includes the following sub-steps:

[0104] S3.1: Unify the time base of the preprocessed video stream, audio stream, water quality data and hydrological data to obtain a time-synchronized signal set, and perform spatial coordinate association on the time-synchronized signal set to obtain a spatially associated signal set;

[0105] S3.2: Extract features from the spatially correlated signal set to obtain video low-frequency features, audio feature vectors, and environmental feature vectors;

[0106] S3.3: Combine low-frequency features, audio feature vectors, and environmental feature vectors to obtain a fused feature matrix.

[0107] Step S3.1 includes the following sub-steps:

[0108] S3.1.1: Using the sampling frequency of the preprocessed video stream as a reference, linear interpolation is performed on the preprocessed audio stream, water quality data, and hydrological data to obtain time-synchronized audio stream, water quality data, and hydrological data. The preprocessed video stream, time-synchronized audio stream, water quality data, and hydrological data are then combined into a time-synchronized signal set, specifically as follows:

[0109] ;

[0110] in, For time-synchronized audio streams, For time-synchronized water quality data, Hydrological data for time synchronization For indicator functions, A set of signals for time synchronization;

[0111] S3.1.2: By simulating fish movement trajectories, the time-synchronized water quality and hydrological data are mapped to the video space of the preprocessed video stream to obtain spatially correlated water quality and hydrological data. Specifically:

[0112]

[0113] in, For spatially correlated water quality data, For spatially correlated hydrological data, For the Dirac function, for Simulates the fish's position at all times. This is the signal set after spatial correlation.

[0114] Step S3.2 includes the following sub-steps:

[0115] S3.2.1: Extract the low-frequency features of the video stream from the spatially correlated signal set using two-dimensional Fourier transform, specifically:

[0116]

[0117] in, Low-frequency features of the video For frequency domain coordinates, Low-frequency threshold;

[0118] S3.2.2: Calculate the short-time energy and zero-crossing rate of the time-synchronized audio stream to extract the audio feature vector, specifically:

[0119]

[0120] in, For audio feature vectors, For window length, The sampling interval;

[0121] S3.2.3: Extract spatially correlated water quality and hydrological data to extract environmental feature vectors, specifically:

[0122]

[0123] in, This is the environmental feature vector.

[0124] Step S3.3 includes the following sub-steps:

[0125] S3.3.1: Calculate the weights of video low-frequency features, audio feature vectors, and environmental feature vectors based on the signal-to-noise ratio to obtain a dynamic weight set;

[0126] S3.3.2: Generate a spatiotemporal fusion matrix based on video low-frequency features, audio feature vectors, environmental feature vectors, and dynamic weight sets, specifically as follows:

[0127]

[0128] in, For spatiotemporal fusion matrix, It is a dynamic weight set.

[0129] In step 4, when compressing the spatiotemporal fusion matrix, a parity check bit is added to the last bit of the encoding of each element.

[0130] like Figure 2 , Figure 3 , Figure 4 and Figure 5 As shown, the simulated fish has a split structure, consisting of three detachable parts: a head section 10, a body section 20, and a tail section 30. The head section 10, body section 20, and tail section 30 are all symmetrical split structures, each consisting of two matching half-shells. The two half-shells of each fish are interlocked along the left and right symmetrical planes to form a cavity structure.

[0131] Two matching connectors 40 are provided at the connection between the fish body segment 20 and the fish tail segment 30. They are fastened together to form a complete ring structure. After fastening, the connection between the fish body segment 20 and the fish tail segment 30 is closed and the fish body segment 20 and the fish tail segment 30 are fixed.

[0132] A pressure plate 50 is provided in the cavity at the connection between the fish body segment 20 and the fish tail segment 30;

[0133] The tail segment 30 is provided with a tail fin 60 at the very end. Two other matching connectors 40 are provided at the connection between the tail fin 60 and the tail segment 30. They are fastened together to form a complete ring structure. After fastening, the inner wall is pressed tightly against the outer wall of the tail segment 30 and the outer wall of the tail fin 60, respectively.

[0134] Side wings 70 are provided on the left and right sides of the outer wall of the fish body section 20, and two swinging parts 80 are provided in the cavity of the fish body section 20.

[0135] The cavity of the fish head segment 10 is equipped with an image sensor, an acoustic sensor, a water quality sensor, a hydrological sensor, a data processing module, an underwater acoustic communication module, and a power supply.

[0136] The swing component 80 is equipped with two cylinders 801 and two servo motors 802. The cylinders 801 are equipped with pistons 803. The two servo motors 802 drive the two pistons 803 to move through a connecting rod structure.

[0137] The linkage structure consists of a first linkage 804 and a second linkage 805. Bearings 806 are respectively installed at both ends of the second linkage. The two ends of the second linkage 805 are respectively connected to the piston 803 and one end of the first linkage 804. The other end of the first linkage 804 is connected to the output shaft of the servo motor 802.

[0138] A pressure cap 807 is provided on the bearing 806 at the end where the second connecting rod 805 and piston 803 are connected.

[0139] Specifically:

[0140] Before deploying the simulated fish to the target waters, the assembly of the modular structure must be completed: the head section 10, body section 20, and tail section 30 are snapped together using a snap-fit ​​structure, and the seams are sealed with nitrile rubber sealing rings to ensure no leakage within a water depth of 50 meters; the connecting parts between the body section 20 and the tail section 30 are made of 316 stainless steel, and are fastened by pressure plates after being snapped together, which can withstand the torque generated by the water flow impact and facilitates later disassembly and maintenance.

[0141] During the initialization phase, the built-in system needs to be activated by sending a command from the host computer: the image sensor (using a 1080P underwater camera with a frame rate of 30fps) is calibrated for white balance to eliminate interference from water scattering light; the acoustic sensor (Hydrophone underwater microphone with a frequency response of 20Hz-20kHz) is warmed up for 3 minutes to reduce circuit noise; the water quality sensor (integrated pH, dissolved oxygen, and turbidity detection modules) is zero-point calibrated and errors are corrected using a standard solution; and the hydrological sensors (pressure depth gauge and water temperature sensor) are synchronized with the base station to ensure that the data timestamp accuracy is at the millisecond level.

[0142] During the data acquisition phase, each sensor operates at a preset frequency: the video stream has a resolution of 1920×1080 per frame, the audio stream has a sampling rate of 44.1kHz, and water quality and hydrological data are acquired 10 times per second. The preprocessing stage utilizes a data processing module for parallel computation.

[0143] In video preprocessing, the Gaussian filter kernel size for spatiotemporal joint denoising is set to 5×5, the time weight coefficient β is set to 0.4 (balancing real-time performance and denoising effect), the inter-frame difference threshold τ is dynamically adjusted according to the turbidity of the water (τ=50 for clear water and τ=80 for turbid water), and the moving target area (such as the swimming trajectory of fish) is preserved through the mask matrix, reducing the amount of redundant data by more than 60%.

[0144] In audio preprocessing, a noise frequency threshold of f0=200Hz (the concentrated area of ​​low-frequency noise underwater) is preset. The frequency bands below f0 are set to zero through Fourier transform, and then the time domain signal is restored through inverse transform, which effectively suppresses water flow noise and equipment self-noise.

[0145] In water quality data preprocessing, the jump threshold T Q Based on sensor accuracy settings (e.g., dissolved oxygen sensor T) Q =0.5mg / L), with a sliding window of N=10, the influence of sensor temperature drift is eliminated through baseline drift correction, improving data stability by 40%.

[0146] In hydrological data preprocessing, the filtering coefficient α = 0.2, and the physical range constraint refers to historical data of the target water area (e.g., the water temperature range H of a freshwater lake). min =4℃, H max =35℃), and outliers that exceed the reasonable range are removed.

[0147] During the time synchronization phase, linear interpolation is performed on audio, water quality, and hydrological data based on the video sampling frequency (30Hz). For example, the original sampling points of the audio stream are expanded to three audio feature points corresponding to each frame of video through an interpolation algorithm, ensuring that the time dimension alignment error is ≤1ms. Spatial correlation is achieved by using the simulated fish's built-in nine-axis gyroscope and sonar positioning data to obtain the motion trajectory (x) in real time. L(t) ,y_ L(t) When mapping water quality and hydrological data to video space, the Dirac function δ is used to limit the data to only be effective at the coordinates of the sensor sampling points, thus avoiding spatial miscorrelation.

[0148] In the feature extraction stage, the low-frequency features of the video are preserved after two-dimensional Fourier transform, retaining the low-frequency components with a frequency domain radius of U0=30 (containing more than 90% of the image energy); the window length of the audio feature vector is N=256, and the sampling interval is Δt=10ms. The combination of short-time energy and zero-crossing rate can effectively distinguish between biological sound and mechanical noise; the environmental feature vector is calculated using the spatial region mean, reflecting the average water quality and hydrological values ​​within a 10×10m range. During fusion, the dynamic weight set is adaptively adjusted according to the signal-to-noise ratio: when the video signal-to-noise ratio is <20dB, ω I Decreased to 0.2, ω QH Increased to 0.5, prioritizing the retention of environmental data.

[0149] Data compression employs a lossy compression algorithm based on wavelet transform, with a compression ratio of 5:1. Parity bits are added to each encoded byte to detect and correct single-bit errors in underwater acoustic communication (transmission rate 9600bps).

[0150] The oscillating component achieves flexible movement through the coordinated action of dual cylinders and a servo motor: the servo motor (torque 5 kg・cm) drives a piston to reciprocate within the cylinder via a first connecting rod, with a piston stroke of 50 mm. The swaying angle of the side fins (±30°) is controlled by changing the pressure difference between the left and right cylinders, thus achieving steering. The tail fin is driven by another set of servos, with an adjustable swaying frequency of 1-5 Hz. Combined with the side fin movements, this allows the simulated fish to reach a maximum swimming speed of 0.5 m / s, enabling it to adapt to water environments with current speeds ≤0.3 m / s. In complex terrain (such as rocky areas), the tail fin swaying frequency is reduced to 1 Hz to improve stability and prevent structural collision damage.

[0151] In summary, the multi-sensor fusion simulation fish underwater environment real-time monitoring method and its simulated fish disclosed in this invention have at least the following beneficial effects:

[0152] In terms of monitoring methods, targeted preprocessing of video streams, audio streams, water quality data, and hydrological data, such as spatiotemporal domain joint denoising, inter-frame redundancy suppression, frequency domain noise suppression, and jump noise removal, effectively improves the quality of various types of data and reduces the impact of noise and redundant information.

[0153] By unifying the time reference and associating spatial coordinates with the preprocessed data, the temporal and spatial consistency of multi-sensor data is enhanced, laying the foundation for subsequent data fusion. The fusion of processed video low-frequency features, audio feature vectors, and environmental feature vectors to generate a fusion feature matrix improves the comprehensive utilization efficiency of multi-source data, enabling a more comprehensive reflection of underwater environmental information.

[0154] Compressing the fused feature matrix and adding parity bits reduces the amount of data transmitted while ensuring the accuracy of data transmission, thus adapting to the characteristics of underwater acoustic communication.

[0155] The modular design of the simulated fish structure allows for easy assembly and disassembly of the head, body, and tail sections, facilitating equipment maintenance and internal component replacement. The symmetrical semi-shell structure and connecting parts ensure structural stability and sealing. The oscillating components enhance the simulated fish's underwater maneuverability, enabling it to better adapt to different underwater environments and expand the monitoring range.

[0156] The fish head section integrates multiple sensors and processing and communication modules, realizing the integration of data acquisition, processing and transmission, and improving the real-time performance and convenience of monitoring.

[0157] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A multi-sensor fusion simulation method for real-time monitoring of underwater fish environment, characterized in that, Includes the following steps: S1: Deploy the simulated fish to the target underwater environment, initialize it, and collect video streams, audio streams, water quality data, and hydrological data; S2: Preprocess the video stream, the audio stream, the water quality data, and the hydrological data respectively, specifically as follows: S2.1: Perform spatiotemporal joint denoising on the video stream to obtain a denoised frame sequence, and perform inter-frame redundancy suppression on the denoised frame sequence to obtain a preprocessed video stream, specifically: in, For the video stream, The denoised frame sequence, For spatial domain Gaussian filtering, For time weighting coefficients, The preprocessed video stream, For norm functions, The difference threshold; S2.2: Perform frequency domain transformation on the audio stream to obtain an audio frequency domain signal, perform adaptive noise band suppression on the audio frequency domain signal to obtain a denoised audio frequency domain signal, and perform time domain transformation on the denoised audio frequency domain signal to obtain a preprocessed audio signal, specifically: in, For the audio stream, The audio frequency domain signal, For Fourier transform operators, The denoised audio frequency domain signal. For frequency band weighting function, To preset the noise frequency band threshold, The preprocessed audio signal; S2.3: The water quality data is subjected to noise removal to obtain de-noising water quality data. Baseline drift correction is then performed on the de-noising water quality data to obtain preprocessed water quality data. Specifically: in, For the water quality data, The water quality data after the jump was removed. For the jump threshold, The water quality data is the result of preprocessing. To adjust the sliding window size; S2.4: Perform high-frequency interference filtering on the hydrological data to obtain filtered hydrological data, and perform physical range constraint correction on the hydrological data to obtain preprocessed hydrological data, specifically as follows: in, For the aforementioned hydrological data, The filtered hydrological data, These are the filter coefficients. The preprocessed hydrological data, These are the upper and lower limits of the physical range of the parameter; S3: The preprocessed video stream, audio stream, water quality data, and hydrological data are fused to generate a fusion feature matrix; S4: The fused feature matrix is ​​compressed to obtain the final compressed data, and the final compressed data is transmitted through underwater acoustic communication.

2. The multi-sensor fusion simulation method for real-time monitoring of underwater fish environment as described in claim 1, characterized in that, Step S3 includes the following sub-steps: S3.1: The time reference of the preprocessed video stream, audio stream, water quality data and hydrological data is unified to obtain a time-synchronized signal set, and the time-synchronized signal set is spatially correlated to obtain a spatially correlated signal set; S3.2: Extract the features from the spatially correlated signal set to obtain video low-frequency features, audio feature vectors, and environmental feature vectors; S3.3: Obtain a fused feature matrix from the low-frequency features, the audio feature vector, and the environmental feature vector.

3. The multi-sensor fusion simulation method for real-time monitoring of underwater fish environment as described in claim 2, characterized in that, Step S3.1 includes the following sub-steps: S3.1.1: Using the sampling frequency of the preprocessed video stream as a reference, linear interpolation is performed on the preprocessed audio stream, water quality data, and hydrological data to obtain time-synchronized audio stream, water quality data, and hydrological data. The preprocessed video stream, time-synchronized audio stream, water quality data, and hydrological data are then combined into a time-synchronized signal set, specifically: ； in, The audio stream is time-synchronized. The water quality data is synchronized with the time. The hydrological data is synchronized in time. For indicator functions, The set of signals for time synchronization; S3.1.2: By simulating the movement trajectory of fish, the time-synchronized water quality data and hydrological data are mapped to the video space of the preprocessed video stream to obtain spatially correlated water quality data and hydrological data, specifically: in, The water quality data is spatially correlated. The hydrological data is spatially correlated. For the Dirac function, for Simulates the fish's position at all times. This refers to the signal set after spatial correlation.

4. The multi-sensor fusion simulation method for real-time monitoring of underwater fish environment as described in claim 3, characterized in that, Step S3.2 includes the following sub-steps: S3.2.1: Extract the low-frequency video features of the video stream from the spatially correlated signal set using a two-dimensional Fourier transform, specifically: in, The low-frequency features of the video, For frequency domain coordinates, Low-frequency threshold; S3.2.2: Calculate the short-time energy and zero-crossing rate of the time-synchronized audio stream to extract the audio feature vector, specifically: in, The audio feature vector, For window length, The sampling interval; S3.2.3: Extract the spatially correlated water quality data and hydrological data to extract environmental feature vectors, specifically: in, This is the environmental feature vector.

5. The multi-sensor fusion simulation method for real-time monitoring of underwater fish environment as described in claim 4, characterized in that, Step S3.3 includes the following sub-steps: S3.3.1: Calculate the weights of the low-frequency features of the video, the audio feature vector, and the environmental feature vector based on the signal-to-noise ratio to obtain a dynamic weight set; S3.3.2: Generate a spatiotemporal fusion matrix based on the video low-frequency features, the audio feature vector, the environmental feature vector, and the dynamic weight set, specifically as follows: in, The spatiotemporal fusion matrix is... This refers to the dynamic weight set.

6. The multi-sensor fusion simulation method for real-time monitoring of underwater fish environment as described in claim 5, characterized in that, In step 4, when compressing the spatiotemporal fusion matrix, a parity check bit is added to the last bit of the encoding of each element.

7. A simulated fish based on the multi-sensor fusion simulation method for real-time underwater environment monitoring as described in claim 1, characterized in that, It has a split structure, consisting of three detachable parts: the head section, the body section, and the tail section. The head section, the body section, and the tail section are all symmetrical split structures, each consisting of two matching half-shells. The two half-shells of each part are interlocked along the left and right symmetrical planes to form a cavity structure. Two matching connectors are provided at the connection between the fish body segment and the fish tail segment. They are fastened together to form a complete ring structure. After fastening, the connection between the fish body segment and the fish tail segment is closed and the fish body segment and the fish tail segment are fixed. A pressure plate is provided in the cavity at the connection between the fish body segment and the fish tail segment; The tail section is provided with a tail fin at its very end. Two other matching connectors are provided at the connection between the tail fin and the tail section. They are fastened together to form a complete ring structure. After fastening, the inner wall of the connector is pressed tightly against the outer wall of the tail section and the outer wall of the tail fin, respectively. Side wings are provided on the left and right sides of the outer wall of the fish body section, and two swinging components are provided in the cavity of the fish body section.

8. The simulated fish as described in claim 7, characterized in that, The cavity of the fish head section is equipped with an image sensor, an acoustic sensor, a water quality sensor, a hydrological sensor, a data processing module, an underwater acoustic communication module, and a power supply.

9. The simulated fish as described in claim 8, characterized in that, The swing component is equipped with two cylinders and two servo motors. The cylinders are equipped with pistons, and the two servo motors drive the two pistons to move through a connecting rod structure.

Images