Unmanned monitoring ship based on sonar vision meteorological water quality fusion and monitoring method thereof

Abstract

The invention provides an unmanned monitoring ship based on sonar vision meteorological water quality fusion and a monitoring method thereof, a multi-beam sonar and an underwater camera are used to collect an acoustic image and a visual image of an underwater target, and an embedded main control board is used to realize synchronous registration and fusion of sonar echoes and images. Target detection and recognition are completed based on a deep learning model; meanwhile, a composite water quality probe, an anemorumbometer, a temperature and humidity sensor and an air pressure sensor are used for collecting water body parameters and meteorological parameters in real time to assist in analyzing the water quality change trend; the communication module transmits fusion data to an upper computer in real time through a 5G network, and remote monitoring and intelligent decision making are achieved. According to the invention, through heterogenous information fusion of sonar and underwater vision, the penetrating power of acoustic detection and the high resolution of visual perception are considered, and high-precision detection of fish data in a turbid environment is realized; according to the cross-modal fusion network based on Transform, the target recognition rate and the positioning precision in a complex environment are improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent unmanned surface monitoring and underwater detection technology, and particularly relates to an unmanned monitoring vessel and its monitoring method based on sonar vision meteorological water quality fusion. Background Technology

[0002] With the rapid development of aquaculture, fish monitoring platforms play a crucial role in water quality monitoring, fish behavior analysis, and environmental data collection. Aquaculture, reservoir inspection, and ecological surveys place higher demands on long-term, continuous, and refined monitoring. Existing underwater monitoring technologies have the following shortcomings: underwater detection platforms relying solely on acoustics offer strong penetration but suffer from insufficient imaging detail and target confusion in shallow water, clutter, and multipath environments; systems relying solely on vision offer high resolution but are significantly limited by turbidity and lighting conditions; and inspection vessels integrating meteorological and water quality sensors on single-ship platforms primarily focus on surface or body parameter acquisition, lacking robust modeling of underwater targets and terrain. Therefore, this invention creatively combines sonar, vision, meteorological, and water quality sensors to form a highly robust fusion detection vessel. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides an unmanned monitoring vessel and its monitoring method based on sonar vision meteorological water quality fusion. It overcomes the problems of insufficient accuracy of single sensors, poor environmental adaptability and limited monitoring range in existing technologies, and achieves high-precision identification, positioning and terrain modeling of underwater targets. It is suitable for long-term intelligent monitoring of complex water areas such as lakes, reservoirs, ports and aquaculture areas.

[0004] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0005] An unmanned monitoring vessel based on sonar-visual meteorological and water quality fusion includes a catamaran hull with a propulsion module installed at the stern. A remotely controlled lifting boom is also mounted on the hull, with its main body fixed to the hull and a mounting plate fixed to the end of its telescopic section. A multibeam sonar and an underwater camera are symmetrically fixed to the mounting plate. An anemometer and wind vane are mounted on the top of the lifting boom, and temperature, humidity, and air pressure sensors are arranged inside. An automatically telescopic sampling arm, also controlled by a remote controller, is installed inside one of the catamaran hulls. A composite water quality probe is installed at the end of the automatically telescopic sampling arm to collect data on pH, dissolved oxygen, ammonia nitrogen, and water temperature. A control box is also installed near the stern of the hull, containing a lithium battery pack, a communication module, a BeiDou satellite navigation module, and a control system. A solar panel is installed above the control box.

[0006] The monitoring method using the above-mentioned unmanned monitoring vessel based on sonar-visual meteorological and water quality fusion includes the following process:

[0007] Step 1: Use the remote controller to control the propulsion module to guide the monitoring vessel into the work area, generate cruise waypoints on the map, and set the cruise route and cruise time of the monitoring vessel.

[0008] Step 2: Once the monitoring vessel enters the monitoring area, the boom is lowered using the remote controller. When the underwater camera and multibeam sonar reach the designated depth, the control system activates them and begins recording. The recorded information is then transmitted back to the control system for analysis and processing. The specific processing steps are as follows:

[0009] Step 2.1: Perform visual distortion correction and edge enhancement processing on the data acquired by the underwater camera and multibeam sonar;

[0010] Step 2.2: Convert the sonar polar coordinates to Cartesian coordinates and perform attitude compensation;

[0011] Step 2.3: Transform the sonar points from the sonar Cartesian coordinate system to the underwater camera coordinate system and project them onto the image plane;

[0012] Step 2.4: The data acquired by the underwater camera after processing in Step 2.1 and the data acquired by the multibeam sonar after processing in Step 2.3 are processed together as follows: a candidate box set is obtained using a lightweight segmentation network;

[0013] Step 2.5: Construct a weighted sum of Dice loss and Focal loss to improve the segmentation quality of the lightweight segmentation network in Step 2.4;

[0014] Step 2.6: Predict the candidate point set after mask cropping, and constrain the sonar points with visual foreground to reduce false alarms;

[0015] Step 2.7: Construct an adjacency graph in the range neighborhood to provide topological adjacency relationships for subsequent triple regularization;

[0016] Step 2.8: Use triple regularization to suppress false alarms. Triple regularization includes spatial continuity regularization, intensity consistency regularization, and boundary alignment regularization. First, perform spatial continuity regularization.

[0017] Step 2.9: Intensity consistency regularization processing;

[0018] Step 2.10: Boundary alignment regularization processing;

[0019] Step 2.11: Density-based clustering condenses candidate points into instance-level targets, providing stable input units for cross-modal coding and detection heads;

[0020] Step 2.12: Perform cross-modal Transformer fusion discrimination. First, construct sonar token features, encode sonar attributes, vectorize acoustic geometry and intensity information, and align them with visual features in the same dimensional space to facilitate Transformer fusion.

[0021] Step 2.13: Visual ROI pooling processing;

[0022] Step 2.14: Employ a bidirectional cross-attention + self-attention mechanism to explicitly align visual boundary textures with sonar geometric intensity, thereby enhancing cross-modal consistency modeling capabilities;

[0023] Step 2.15: The detection head outputs the category, confidence score, and bounding box to generate a decisionable target-level judgment result, namely, category, confidence score, and location;

[0024] Step 2.16: Optimize the results obtained in Step 2.15 to reduce errors;

[0025] Step 3: After obtaining the sonar and underwater video data with fish confidence scores and borders from Step 2, calculate the pixel ratio of each fish to estimate the fish length. Estimate the weight of each fish by using the linear relationship between fish length and weight. Transmit the obtained data to the cloud platform via 5G for users to observe in real time and intuitively see the body length and weight data of underwater fish. At the same time, view real-time data on wind speed and direction, temperature and humidity, air pressure, pH, ammonia nitrogen, and water temperature.

[0026] Furthermore, in step 2.5, the weighted sum of Dice loss and Focal loss is constructed using the following formula to improve the segmentation quality of the lightweight segmentation network in step 2.4:

[0027]

[0028] in, For predicting masks; As a truth mask; , For loss weights; Indicates the total loss; Indicates Focal loss; This indicates Dice's loss.

[0029] Furthermore, in step 2.8, the spatial continuity regularization process is as follows:

[0030]

[0031] in, For spatial continuity regularity; Indicates the first One camera system point; Indicates the first One camera system point; Represents a small constant; Represents a point set; It is an adjacent edge set; This indicates that the result of the L2 norm is squared.

[0032] Furthermore, in step 2.9, the intensity consistency regularization process is as follows:

[0033]

[0034] in, For strength consistency regularization; For the first A cluster; Echo intensity; Indicates variance.

[0035] Furthermore, in step 2.10, the boundary alignment regularization process is as follows:

[0036]

[0037] in, For candidate projection boundaries; The set of image edges; For projection mapping; Indicates the boundary; This represents the coordinates of a two-dimensional pixel on the candidate boundary after projection onto the image plane; This represents the coordinates of a two-dimensional pixel in the image edge set.

[0038] Furthermore, the optimization process in step 2.16 includes:

[0039] Target association and temporal smoothing are achieved by constructing an association cost matrix and matching it using the Hungarian algorithm. This ensures stable association of the same entity in multi-target scenarios and reduces track breakage.

[0040]

[0041] in, For matching costs; For intersection, union, and comparison; This is the observation vector for the current frame; To predict the vector to the measurement domain; For covariance; , , As weight; Indicates the first Confidence score of each detection target; Indicates matrix transpose;

[0042] Uniform Kalman and EMA smoothing suppress jitter and missing measurements, providing continuous and reliable pose and motion trends:

[0043]

[0044] in, State; For unit array; For covariance; The observation matrix; To measure the noise covariance; Kalman gain; Confidence level; Indicates the interval between adjacent frames; Indicated by For the current frame, based on The predicted prior state; Represents the prior error covariance, characterizing Uncertainty; Represents the observation matrix Transpose of; Indicates time The posterior state.

[0045] The present invention has the following beneficial effects:

[0046] By fusing heterogeneous information from sonar and underwater vision, and taking into account both the penetration capability of acoustic detection and the high resolution of visual perception, high-precision detection of fish data in turbid environments is achieved.

[0047] This invention proposes a Transformer-based cross-modal fusion network, which effectively improves target recognition rate and positioning accuracy in complex environments;

[0048] The lightweight catamaran design and dual power supply system of solar and lithium batteries significantly improve endurance and environmental adaptability.

[0049] The system supports unattended continuous operation, automatically completing data collection, processing and uploading, reducing manual intervention and operating costs;

[0050] This invention can be widely applied to fields such as aquaculture monitoring, underwater structure inspection, and water quality and ecological surveys, and has significant economic and social benefits. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the structure of an unmanned monitoring vessel;

[0052] Figure 2 This is a schematic diagram of the ship's hull structure.

[0053] Figure 3 This is a schematic diagram of the lifting boom structure;

[0054] In the diagram: 1. Hull; 2. Lifting mast; 3. Composite water quality probe; 4. Anemometer; 5. Solar panel; 6. Control box; 7. Propulsion module; 8. Crossbeam; 9. Catamaran structure; 10. Underwater camera; 11. Multibeam sonar. Detailed Implementation

[0055] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0056] The present invention describes an unmanned monitoring vessel based on sonar vision meteorological water quality fusion, comprising a hull 1, a lifting mast 2, a propulsion module 7, a multibeam sonar 11, an underwater camera 10, a composite water quality probe 3, an anemometer 4, a communication module, and a power module.

[0057] like Figure 1 , 2 As shown in Figure 3, the hull 1 preferably adopts a lightweight catamaran floating structure 9 to ensure navigation stability and wave resistance. The two sub-hulls are connected by crossbeams 8 to form a complete hull 1. A propulsion module 7 is installed at the stern of the hull 1. The propulsion module 7 adopts a twin-propeller structure to achieve differential drive and precise steering control, and can be used with a small-angle rudder to improve low-speed attitude stability. The main body of the boom 2 is fixed to the hull 1, and the end of the telescopic part is fixed with a mounting plate. The raising and lowering of the boom 2 is controlled by a remote controller.

[0058] like Figure 1 , 3 As shown, the multibeam sonar 11 is fixed to the mounting plate and is used to generate high-resolution acoustic point cloud data; the underwater camera 10 is symmetrically arranged to the left of the multibeam sonar 11, working through an optical porthole to acquire underwater RGB image data; the underwater camera 10 and the multibeam sonar 11 are respectively connected to the time synchronization trigger line, and the sampling frequency and exposure are uniformly issued by the control system. In actual operation, the telescopic movement of the lifting rod 2 is controlled by the remote controller, thereby driving the multibeam sonar 11 and the underwater camera 10 on the mounting plate to extend underwater to a designated depth or retract to their original position.

[0059] like Figure 1 As shown, an automatic telescopic sampling arm is installed on the inner side of one of the sub-ships. It is controlled by a remote controller and the telescopic distance and telescopic time can be freely adjusted. The telescopic distance is 0-50 cm. A composite water quality probe 3 is installed at the end of the automatic telescopic sampling arm to collect indicators such as pH, dissolved oxygen, ammonia nitrogen, and water temperature. The sampling interval is adjustable from 0.5 to 2 seconds.

[0060] like Figure 1As shown, a wind speed and direction instrument 4 is installed on the top of the lifting mast 2, and a temperature and humidity sensor and a pressure sensor are arranged inside it; the sampled data and the ship's hull track are timestamped together to facilitate subsequent coupling analysis.

[0061] The power module includes solar panel 5, lithium battery pack, and power management unit. Solar panel 5 and lithium battery pack work together to provide power, ensuring that the monitoring vessel can operate continuously for 8 to 12 hours. The power management unit monitors in real time and supports shore power fast charging.

[0062] like Figure 1 As shown, a control box 6 is also installed on the hull 1 near the stern. The control box 6 contains a lithium battery pack, a communication module, a Beidou satellite navigation module, and a control system. A solar panel 5 is installed on top of the control box 6.

[0063] The communication module includes a 5G communication terminal. The control system is based on a Raspberry Pi 4B (a Linux-based microcontroller computer) main control platform, integrating data acquisition and embedded GPU computing units to achieve real-time data fusion and autonomous control. The BeiDou satellite navigation module, multi-beam sonar 11, underwater camera 10, composite water quality probe 3, anemometer 4, temperature and humidity sensor, and barometric pressure sensor are all connected to the control system to transmit various monitoring data.

[0064] Data processing and multi-source fusion workflow: Sonar echoes are transformed into point clouds using polar coordinates and Cartesian transformation; camera images undergo distortion correction and color normalization; water quality and meteorological data are cached as time series. On the visual side, a lightweight segmentation network is used to obtain masks and target boxes; the sonar point cloud is projected onto the image plane, and candidate point sets are selected based on the mask, introducing spatial continuity and intensity consistency losses to suppress false alarms; subsequently, texture boundary features and geometric intensity features are jointly encoded in a cross-modal Transformer fusion network to output target category and confidence level, and temporal smoothing is performed using a uniform speed model.

[0065] Environmental assessment and cloud migration: Indicators such as wind speed and direction, temperature and humidity, air pressure, pH, ammonia nitrogen, and water temperature are uploaded to the host computer via a 5G link, and the display of indicator curves is supported.

[0066] This invention supports multi-tasking operation mode:

[0067] Autonomous cruise mode: The control system generates speed and heading commands based on preset track and channel constraints;

[0068] Dynamic obstacle avoidance mode: When the multibeam sonar 11 detects an obstacle ahead, the control system generates a safe course and adjusts the propulsion speed in real time;

[0069] Target tracking mode: After detecting a specific target, hull 1 automatically maintains distance and orientation to achieve continuous tracking and data collection;

[0070] Site monitoring mode: hovers at a designated coordinate point and periodically collects underwater images and acoustic information.

[0071] The monitoring method using an unmanned monitoring vessel based on sonar-visual meteorological and water quality fusion as described in this invention includes the following steps:

[0072] Step 1: Use a remote controller to control the twin propeller structure, guide the monitoring vessel into the work area, generate cruise waypoints on the map, and set the cruise route and cruise time of the monitoring vessel.

[0073] Step 2: Once the monitoring vessel enters the monitoring area, the remote controller lowers the boom 2. When the underwater camera 10 and multibeam sonar 11 reach a suitable depth, the Raspberry Pi 4B main control platform activates the underwater camera 10 and multibeam sonar 11 and begins recording. The recorded information is transmitted to the Raspberry Pi 4B main control platform. The Raspberry Pi 4B main control platform processes the collected data, as follows:

[0074] Step 2.1: Perform visual distortion correction and edge enhancement processing on the data acquired by the underwater camera 10 and the multibeam sonar 11:

[0075]

[0076] in, Edge intensity map; , These are the gradients in the x and y directions, respectively; , These represent the first-order gradients in the x and y directions, respectively. The data collected by the underwater camera 10 after the above processing is used for lightweight segmentation in the subsequent step 2.4, and the data collected by the multibeam sonar 11 after the above processing is used for boundary alignment in the subsequent step 2.10.

[0077] Step 2.2: Convert sonar polar coordinates to Cartesian coordinates and perform attitude compensation:

[0078]

[0079] in, For range; It is the bend angle; For deep compensation; This is a sonar point.

[0080] Step 2.3: Transform the sonar points from the sonar Cartesian coordinate system to the underwater camera coordinate system and project them onto the image plane:

[0081]

[0082] in, For camera attachment points; These are homogeneous pixel coordinates; They are homogeneous and equivalent; Indicates a preprocessed image; Represents the camera intrinsic parameter matrix; This represents the rigid body transformation matrix from the sonar Cartesian coordinate system to the camera coordinate system. Step 2.3 outputs the position of the sonar point on the image plane, which is used for mask clipping in the subsequent step 2.6.

[0083] Step 2.4: The data acquired by the underwater camera 10 after processing in Step 2.1 and the data acquired by the multibeam sonar 11 after processing in Step 2.3 are processed together as follows: A candidate box set is obtained using a lightweight segmentation network. :

[0084]

[0085] in, , Center of the frame; , Width and height; This represents the set of candidate boxes. The output of step 2.4 is used for mask clipping and graph structure construction in subsequent steps 2.6–2.7, and provides the visual token range for steps 2.12 and 2.13.

[0086] Step 2.5: Construct a weighted sum of Dice loss and Focal loss to improve the segmentation quality of the lightweight segmentation network in Step 2.4.

[0087]

[0088] in, For predicting masks; As a truth mask; , For loss weights; Indicates the total loss; Indicates Focal loss; This indicates Dice's loss.

[0089] Step 2.6: Predict the candidate point set after mask cropping, and constrain the sonar points with visual foreground to reduce false alarms:

[0090]

[0091] in, For candidate point set; Foreground selection for masking.

[0092] Step 2.7: Construct an adjacency graph in the range neighborhood to provide topological adjacency relationships for the triple regularization in subsequent steps 2.8 to 2.10:

[0093]

[0094] in, It is an adjacent edge set; , These are the beam angle and the range threshold, respectively. , They represent the first , The bend angle between the points; , They represent the first , The range between points; Represents a point set.

[0095] Step 2.8: Triple regularization suppresses false alarms. Triple regularization is used for the fusion score in subsequent step 2.12; the spatial continuity processing is as follows:

[0096]

[0097] in, For spatial continuity regularity; Indicates the first One camera system point; Indicates the first One camera system point; Represents a small constant; This indicates that the result of the L2 norm is squared.

[0098] Step 2.9: Strength Consistency Processing:

[0099]

[0100] in, For strength consistency regularization; For the first A cluster; Echo intensity; Indicates variance.

[0101] Step 2.10: Boundary Alignment Processing:

[0102]

[0103] in, For candidate projection boundaries; The set of image edges; For projection mapping; Indicates the boundary; This represents the coordinates of a two-dimensional pixel on the candidate boundary after projection onto the image plane; This represents the coordinates of a two-dimensional pixel in the image edge set.

[0104] Step 2.11: Density-based clustering condenses candidate points into instance-level targets (clusters), providing stable input units for cross-modal coding and detection heads.

[0105]

[0106] in, The radius is ; The minimum number of points; Determined by the core point; This represents a candidate point to be determined. Indicates and Compared to neighboring points; Indicates the number of points.

[0107] Step 2.12: Perform cross-modal Transformer fusion discrimination. First, construct sonar token features, encode sonar attributes, vectorize acoustic geometry and intensity information, and align them with visual features to the same dimensional space to facilitate Transformer fusion.

[0108]

[0109] in, For the first A sonar attribute; For bundle index; Features of sonar tokens; Represents the real number field; These represent the three-dimensional coordinates of the sonar point in the selected coordinate system; Indicates azimuth; This indicates the echo intensity at that sonar point; Indicates the range; This represents a multilayer perceptron, used to map the raw attributes of sonar points into fixed-dimensional sonar token features.

[0110] Step 2.13: Visual ROI pooling processing:

[0111]

[0112] in, Visual token features; Main backbone network; For pooling; express Image frames after preprocessing at a given time.

[0113] Step 2.14: Employing a bidirectional cross-attention + self-attention mechanism, explicitly align visual boundary textures with sonar geometric intensity to improve cross-modal consistency modeling capabilities.

[0114]

[0115] in, , , These are respectively query, key, and value; For dimensions; For normalized weights.

[0116] Step 2.15: The detection head outputs the category, confidence score, and bounding box to generate a decisionable target-level determination result, namely, category, confidence score, and location.

[0117]

[0118] in, Features of fusion; Distribution by category; Confidence level; For borders; , As weight; , For bias; Represents the sigmoid function; This represents a multilayer perceptron.

[0119] Step 2.16: Target association and temporal smoothing. Construct the association cost matrix and use the Hungarian algorithm for matching to stably associate the same entity in multi-target scenarios and reduce track breaks.

[0120]

[0121] in, For matching costs; For intersection, union, and comparison; This is the observation vector for the current frame; To predict the vector to the measurement domain; For covariance; , , As weight; Let represent the confidence score of the i-th detected target.

[0122] Step 2.17: Uniform Kalman and EMA smoothing suppresses jitter and missing measurements, providing continuous and reliable pose and motion trends:

[0123]

[0124] in, State; For unit array; For covariance; The observation matrix; To measure the noise covariance; Kalman gain; Confidence level; Indicates the interval between adjacent frames; Indicated by For the current frame, based on The predicted prior state; Represents the prior error covariance, characterizing Uncertainty; Represents the observation matrix Transpose of; Indicates time The posterior state.

[0125] Step 3: Using the sonar and underwater video data with fish confidence scores and borders obtained in Step 2, calculate the pixel ratio of each fish to estimate its length, and estimate its weight using the linear relationship between length and weight. This data is transmitted to a cloud platform via the Raspberry Pi 4B's 5G link. Users can observe the calculated sonar and image data in real time on the cloud platform, easily and intuitively viewing the length and weight of underwater fish. Simultaneously, users can view indicators such as wind speed and direction, temperature, air pressure, pH, ammonia nitrogen, and water temperature on the cloud platform.

[0126] The embodiments described above are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. An unmanned monitoring ship based on the fusion of sonar vision meteorological water quality, characterized in that, The application relates to a ship body (1) comprising a catamaran structure (9), wherein a propelling module (7) is arranged at the tail of the ship body (1), a lifting rod (2) controlled by a remote controller is arranged on the ship body (1), the main body of the lifting rod (2) is fixed on the ship body (1), a mounting plate is arranged at the end of the telescopic part, a multi-beam sonar (11) and an underwater camera (10) are symmetrically arranged on the mounting plate, a wind speed and direction instrument (4) is arranged on the top of the lifting rod (2), and a temperature and humidity sensor and an air pressure sensor are arranged in the lifting rod (2); an automatic telescopic sampling arm controlled by a remote controller is arranged in the inner side of one of the two sub-ship bodies of the catamaran structure (9), a composite water quality probe (3) is arranged at the end of the automatic telescopic sampling arm, and the composite water quality probe (3) is used for collecting pH, dissolved oxygen, ammonia nitrogen and water temperature index data; a control box (6) is arranged on the ship body (1) close to the tail, lithium batteries, a communication module, a Beidou satellite navigation module and a control system are arranged in the control box (6), and a solar panel (5) is arranged above the control box (6).

2. The monitoring method using the unmanned monitoring ship based on the fusion of sonar vision meteorological water quality according to claim 1, characterized in that, The application also discloses a monitoring method of the ship body (1). Step 1: the remote controller is used to control the propelling module (7) to guide the monitoring ship into a working area, cruise navigation points are generated on a map, and the cruise path and cruise time of the monitoring ship are set; Step 2: when the monitoring ship enters a monitoring area, the remote controller is used to control the lifting rod (2) to descend, the underwater camera (10) and the multi-beam sonar (11) are waited to descend to a specified depth under water, the control system is used to control the underwater camera (10) and the multi-beam sonar (11) to be turned on and record, and the recording information is transmitted back to the control system for analysis and processing; the specific processing process is as follows: Step 2.1: the data collected by the underwater camera (10) and the multi-beam sonar (11) are subjected to visual distortion removal and edge enhancement processing; Step 2.2: the sonar polar coordinates are converted into Cartesian coordinates and subjected to attitude compensation; Step 2.3: the sonar points are transformed from the sonar Cartesian coordinate system into the underwater camera coordinate system and projected onto the image plane; Step 2.4: the data collected by the underwater camera (10) after the processing in step 2.1 and the data collected by the multi-beam sonar (11) after the processing in step 2.3 are jointly subjected to the following processing: a lightweight segmentation network is used to obtain a candidate box set; Step 2.5: a weighted sum of Dice loss and focal loss is constructed to improve the segmentation quality of the lightweight segmentation network in step 2.4; Step 2.6: the candidate point set after mask cropping is predicted, the sonar points are constrained by visual foreground, and false alarms are reduced; Step 2.7: a neighborhood graph is constructed in a range neighborhood to provide topological neighborhood relations for subsequent triple regularization; Step 2.8: triple regularization is adopted to suppress false alarms, and the triple regularization comprises spatial continuity regularization, intensity consistency regularization and boundary alignment regularization; firstly, the spatial continuity regularization is processed; Step 2.9: the intensity consistency regularization is processed; Step 2.10: the boundary alignment regularization is processed; Step 2.11: based on density, the candidate points are condensed into instance-level targets to provide stable input units for cross-modal encoding and detection heads. Step 2.12: Cross-modal Transformer fusion discrimination is performed, first constructing sonar token features, encoding sonar attributes, and vectorizing acoustic geometry and intensity information to align with visual features in the same dimension space for Transformer fusion; Step 2.13: Visual Roi pooling processing; Step 2.14: A bidirectional cross-attention + self-attention mechanism is used to explicitly align visual boundary texture and sonar geometry and intensity, improving cross-modal consistency modeling capability; Step 2.15: The detection head outputs the category, confidence, and bounding box, generating a decision-making target-level judgment result, i.e., category, confidence, and position; Step 2.16: The results obtained in step 2.15 are optimized to reduce errors; Step 3: After processing in step 2, the sonar and underwater video data with fish confidence and bounding box are obtained, the pixel proportion of each fish is calculated to estimate the length of the fish, the weight of each fish is estimated through the linear relationship between the length and the weight of the fish, and the obtained data is transmitted to the cloud platform through the 5G transmission mode, so that the user can observe the length and weight data of the underwater fish in real time, and simultaneously view the wind speed and direction instrument (4), temperature and humidity sensor, air pressure sensor, and composite water quality probe (3) collected wind speed and direction, temperature and humidity, air pressure, pH, ammonia nitrogen, and water temperature index data.

3. The monitoring method according to claim 2, characterized in that, In step 2.5, the following formula is used to construct the weighted sum of Dice loss + Focal loss to improve the segmentation quality of the lightweight segmentation network in step 2.4: ； wherein, is a prediction mask; is a true value mask; , is a loss weight; represents a total loss; represents a Focal loss; represents a Dice loss.

4. The monitoring method of claim 2, wherein, In step 2.8, the spatial continuity regularization is processed as follows: ； wherein, is spatially continuous regular; represents the th camera system point; represents the th camera system point; represents a small constant; represents a set of points; is a set of adjacent edges; represents taking the square of the result of a two-norm.

5. The monitoring method of claim 2, wherein, In step 2.9, the intensity consistency regularization is processed as follows: ； wherein, is a consistent regularity of intensity; is a first cluster; is an echo intensity; denotes a variance.

6. The monitoring method of claim 2, wherein, In step 2.10, the boundary alignment regularization is processed as follows: ； wherein, is a candidate projection boundary; is a set of image edges; is a projection mapping; denotes a boundary; denotes a two-dimensional pixel coordinate on the candidate boundary after projection onto the image plane; denotes a two-dimensional pixel coordinate in the set of image edges; denotes taking the square of the result of the two-norm.

7. The monitoring method of claim 2, wherein, The optimization processing of step 2.16 includes: Target association and time series smoothing, constructing an association cost matrix and using the Hungarian algorithm for matching to stabilize the association of the same entity in a multi-target scene and reduce broken tracks: ； wherein, is a matching cost; is a Jaccard index; is an observation vector of the current frame; is a vector predicted to the measurement domain; is a covariance; , , is a weight; denotes a confidence score of the th detected object; denotes a matrix transpose; Uniform speed Kalman and EMA smoothing to suppress jitter and missing data and provide continuous and reliable pose and motion trends: ； in, State; For unit array; For covariance; The observation matrix; To measure the noise covariance; Kalman gain; Confidence level; Indicates the interval between adjacent frames; Indicates For the current frame, based on The predicted prior state; Represents the prior error covariance, characterizing Uncertainty; Represents the observation matrix transpose; Indicates time The posterior state.

Images