Unmanned monitoring ship based on fusion of sonar vision and meteorological water quality and monitoring method thereof

Abstract

The application provides an unmanned monitoring ship based on sonar visual meteorological water quality fusion and a monitoring method thereof, acoustic images and visual images of underwater targets are collected by using a multi-beam sonar and an underwater camera, synchronous registration and fusion of sonar echoes and images are realized by using an embedded main control board, target detection and identification are completed based on a deep learning model; meanwhile, water body parameters and meteorological parameters are collected in real time by using a composite water quality probe, a wind speed and direction instrument, a temperature and humidity sensor and an air pressure sensor, and water quality change trends are analyzed; communication modules transmit fusion data to an upper computer in real time through a 5G network, remote monitoring and intelligent decision-making are realized. Through the fusion of heterogeneous information of sonar and underwater vision, the penetration ability of acoustic detection and the high resolution of visual perception are taken into account, high-precision detection of fish data in turbid environment is realized; the cross-modal fusion network based on the Transformer improves the target recognition rate and positioning accuracy in complex environments.

Description

Technical Field

[0001] This invention belongs to the field of intelligent unmanned surface monitoring and underwater detection technology, and particularly relates to an unmanned monitoring vessel and its monitoring method based on sonar vision meteorological water quality fusion. Background Technology

[0002] With the rapid development of aquaculture, fish monitoring platforms play a crucial role in water quality monitoring, fish behavior analysis, and environmental data collection. Aquaculture, reservoir inspection, and ecological surveys place higher demands on long-term, continuous, and refined monitoring. Existing underwater monitoring technologies have the following shortcomings: underwater detection platforms relying solely on acoustics offer strong penetration but suffer from insufficient imaging detail and target confusion in shallow water, clutter, and multipath environments; systems relying solely on vision offer high resolution but are significantly limited by turbidity and lighting conditions; and inspection vessels integrating meteorological and water quality sensors on single-ship platforms primarily focus on surface or body parameter acquisition, lacking robust modeling of underwater targets and terrain. Therefore, this invention creatively combines sonar, vision, meteorological, and water quality sensors to form a highly robust fusion detection vessel. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides an unmanned monitoring vessel and its monitoring method based on sonar vision meteorological water quality fusion. It overcomes the problems of insufficient accuracy of single sensors, poor environmental adaptability and limited monitoring range in existing technologies, and achieves high-precision identification, positioning and terrain modeling of underwater targets. It is suitable for long-term intelligent monitoring of complex water areas such as lakes, reservoirs, ports and aquaculture areas.

[0004] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0005] An unmanned monitoring vessel based on sonar-visual meteorological and water quality fusion includes a catamaran hull with a propulsion module installed at the stern. A remotely controlled lifting boom is also mounted on the hull, with its main body fixed to the hull and a mounting plate fixed to the end of its telescopic section. A multibeam sonar and an underwater camera are symmetrically fixed to the mounting plate. An anemometer and wind vane are mounted on the top of the lifting boom, and temperature, humidity, and air pressure sensors are arranged inside. An automatically telescopic sampling arm, also controlled by a remote controller, is installed inside one of the catamaran hulls. A composite water quality probe is installed at the end of the automatically telescopic sampling arm to collect data on pH, dissolved oxygen, ammonia nitrogen, and water temperature. A control box is also installed near the stern of the hull, containing a lithium battery pack, a communication module, a BeiDou satellite navigation module, and a control system. A solar panel is installed above the control box.

[0006] The monitoring method using the above-mentioned unmanned monitoring vessel based on sonar-visual meteorological and water quality fusion includes the following process:

[0007] Step 1: Use the remote controller to control the propulsion module to guide the monitoring vessel into the work area, generate cruise waypoints on the map, and set the cruise route and cruise time of the monitoring vessel.

[0008] Step 2: Once the monitoring vessel enters the monitoring area, the boom is lowered using the remote controller. When the underwater camera and multibeam sonar reach the designated depth, the control system activates them and begins recording. The recorded information is then transmitted back to the control system for analysis and processing. The specific processing steps are as follows:

[0009] Step 2.1: Perform visual distortion correction and edge enhancement processing on the data acquired by the underwater camera and multibeam sonar;

[0010] Step 2.2: Convert the sonar polar coordinates to Cartesian coordinates and perform attitude compensation;

[0011] Step 2.3: Transform the sonar points from the sonar Cartesian coordinate system to the underwater camera coordinate system and project them onto the image plane;

[0012] Step 2.4: The data acquired by the underwater camera after processing in Step 2.1 and the data acquired by the multibeam sonar after processing in Step 2.3 are processed together as follows: a set of candidate boxes is obtained using a lightweight segmentation network;

[0013] Step 2.5: Construct a weighted sum of Dice loss and Focal loss to improve the segmentation quality of the lightweight segmentation network in Step 2.4;

[0014] Step 2.6: Predict the candidate point set after mask cropping, and constrain the sonar points with visual foreground to reduce false alarms;

[0015] Step 2.7: Construct an adjacency graph in the range neighborhood to provide topological adjacency relationships for subsequent triple regularization;

[0016] Step 2.8: Use triple regularization to suppress false alarms. Triple regularization includes spatial continuity regularization, intensity consistency regularization, and boundary alignment regularization. First, perform spatial continuity regularization.

[0017] Step 2.9: Intensity consistency regularization processing;

[0018] Step 2.10: Boundary alignment regularization processing;

[0019] Step 2.11: Density-based clustering condenses candidate points into instance-level targets, providing stable input units for cross-modal coding and detection heads;

[0020] Step 2.12: Perform cross-modal Transformer fusion discrimination. First, construct sonar token features, encode sonar attributes, vectorize acoustic geometry and intensity information, and align them with visual features in the same dimensional space to facilitate Transformer fusion.

[0021] Step 2.13: Visual ROI pooling processing;

[0022] Step 2.14: Employ a bidirectional cross-attention + self-attention mechanism to explicitly align visual boundary textures with sonar geometric intensity, thereby enhancing cross-modal consistency modeling capabilities;

[0023] Step 2.15: The detection head outputs the category, confidence score, and bounding box to generate a decisionable target-level judgment result, namely, category, confidence score, and location;

[0024] Step 2.16: Optimize the results obtained in Step 2.15 to reduce errors;

[0025] Step 3: After obtaining the sonar and underwater video data with fish confidence scores and borders from Step 2, calculate the pixel ratio of each fish to estimate the fish length. Estimate the weight of each fish by using the linear relationship between fish length and weight. Transmit the obtained data to the cloud platform via 5G for users to observe in real time and intuitively see the body length and weight data of underwater fish. At the same time, view real-time data on wind speed and direction, temperature and humidity, air pressure, pH, ammonia nitrogen, and water temperature.

[0026] Furthermore, in step 2.5, the weighted sum of Dice loss and Focal loss is constructed using the following formula to improve the segmentation quality of the lightweight segmentation network in step 2.4:

[0027]

[0028] in, For predicting masks; As a truth mask; , For loss weights; Indicates the total loss; Indicates Focal loss; This indicates Dice's loss.

[0029] Furthermore, in step 2.8, the spatial continuity regularization process is as follows:

[0030]

[0031] in, For spatial continuity regularity; Indicates the first One camera system point; Indicates the first One camera system point; Represents a small constant; Represents a point set; It is an adjacent edge set; This indicates that the result of the L2 norm is squared.

[0032] Furthermore, in step 2.9, the intensity consistency regularization process is as follows:

[0033]

[0034] in, For strength consistency regularization; For the first A cluster; Echo intensity; Indicates variance.

[0035] Furthermore, in step 2.10, the boundary alignment regularization process is as follows:

[0036]

[0037] in, For candidate projection boundaries; The set of image edges; For projection mapping; Indicates the boundary; This represents the coordinates of a two-dimensional pixel on the candidate boundary after projection onto the image plane; This represents the coordinates of a two-dimensional pixel in the image edge set.

[0038] Furthermore, the optimization process in step 2.16 includes:

[0039] Target association and temporal smoothing are achieved by constructing an association cost matrix and matching it using the Hungarian algorithm. This ensures stable association of the same entity in multi-target scenarios and reduces track breakage.

[0040]

[0041] in, For matching costs; For intersection, union, and comparison; This is the observation vector for the current frame; To predict the vector to the measurement domain; For covariance; , , As weight; Indicates the first Confidence score of each detection target; Indicates matrix transpose;

[0042] Uniform Kalman and EMA smoothing suppress jitter and missing measurements, providing continuous and reliable pose and motion trends:

[0043]

[0044] in, State; For unit array; For covariance; The observation matrix; To measure the noise covariance; Kalman gain; Confidence level; Indicates the interval between adjacent frames; Indicates For the current frame, based on The predicted prior state; Represents the prior error covariance, characterizing Uncertainty; Represents the observation matrix Transpose of; Indicates time The posterior state.

[0045] The present invention has the following beneficial effects:

[0046] By fusing heterogeneous information from sonar and underwater vision, and taking into account both the penetration capability of acoustic detection and the high resolution of visual perception, high-precision detection of fish data in turbid environments is achieved.

[0047] This invention proposes a Transformer-based cross-modal fusion network, which effectively improves target recognition rate and positioning accuracy in complex environments;

[0048] The lightweight catamaran design and dual power supply system of solar and lithium batteries significantly improve endurance and environmental adaptability.

[0049] The system supports unattended continuous operation, automatically completing data collection, processing and uploading, reducing manual intervention and operating costs;

[0050] This invention can be widely applied to fields such as aquaculture monitoring, underwater structure inspection, and water quality and ecological surveys, and has significant economic and social benefits. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the structure of an unmanned monitoring vessel;

[0052] Figure 2 This is a schematic diagram of the ship's hull structure.

[0053] Figure 3 This is a schematic diagram of the lifting boom structure;

[0054] In the diagram: 1. Hull; 2. Lifting mast; 3. Composite water quality probe; 4. Anemometer; 5. Solar panel; 6. Control box; 7. Propulsion module; 8. Crossbeam; 9. Catamaran structure; 10. Underwater camera; 11. Multibeam sonar. Detailed Implementation

[0055] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0056] The present invention describes an unmanned monitoring vessel based on sonar vision meteorological water quality fusion, comprising a hull 1, a lifting mast 2, a propulsion module 7, a multibeam sonar 11, an underwater camera 10, a composite water quality probe 3, an anemometer 4, a communication module, and a power module.

[0057] like Figure 1 , 2 As shown in Figure 3, the hull 1 preferably adopts a lightweight catamaran floating structure 9 to ensure navigation stability and wave resistance. The two sub-hulls are connected by crossbeams 8 to form a complete hull 1. A propulsion module 7 is installed at the stern of the hull 1. The propulsion module 7 adopts a twin-propeller structure to achieve differential drive and precise steering control, and can be used with a small-angle rudder to improve low-speed attitude stability. The main body of the boom 2 is fixed to the hull 1, and the end of the telescopic part is fixed with a mounting plate. The raising and lowering of the boom 2 is controlled by a remote controller.

[0058] like Figure 1 , 3 As shown, the multibeam sonar 11 is fixed to the mounting plate and is used to generate high-resolution acoustic point cloud data; the underwater camera 10 is symmetrically arranged to the left of the multibeam sonar 11, working through an optical porthole to acquire underwater RGB image data; the underwater camera 10 and the multibeam sonar 11 are respectively connected to the time synchronization trigger line, and the sampling frequency and exposure are uniformly issued by the control system. In actual operation, the telescopic movement of the lifting rod 2 is controlled by the remote controller, thereby driving the multibeam sonar 11 and the underwater camera 10 on the mounting plate to extend underwater to a designated depth or retract to their original position.

[0059] like Figure 1 As shown, an automatic telescopic sampling arm is installed on the inner side of one of the sub-ships. It is controlled by a remote controller and the telescopic distance and telescopic time can be freely adjusted. The telescopic distance is 0-50 cm. A composite water quality probe 3 is installed at the end of the automatic telescopic sampling arm to collect indicators such as pH, dissolved oxygen, ammonia nitrogen, and water temperature. The sampling interval is adjustable from 0.5 to 2 seconds.

[0060] like Figure 1As shown, a wind speed and direction instrument 4 is installed on the top of the lifting mast 2, and a temperature and humidity sensor and a pressure sensor are arranged inside it; the sampled data and the ship's hull track are timestamped together to facilitate subsequent coupling analysis.

[0061] The power module includes solar panel 5, lithium battery pack, and power management unit. Solar panel 5 and lithium battery pack work together to provide power, ensuring that the monitoring vessel can operate continuously for 8 to 12 hours. The power management unit monitors in real time and supports shore power fast charging.

[0062] like Figure 1 As shown, a control box 6 is also installed on the hull 1 near the stern. The control box 6 contains a lithium battery pack, a communication module, a Beidou satellite navigation module, and a control system. A solar panel 5 is installed on top of the control box 6.

[0063] The communication module includes a 5G communication terminal. The control system is based on a Raspberry Pi 4B (a Linux-based microcontroller computer) main control platform, integrating data acquisition and embedded GPU computing units to achieve real-time data fusion and autonomous control. The BeiDou satellite navigation module, multi-beam sonar 11, underwater camera 10, composite water quality probe 3, anemometer 4, temperature and humidity sensor, and barometric pressure sensor are all connected to the control system to transmit various monitoring data.

[0064] Data processing and multi-source fusion workflow: Sonar echoes are transformed into point clouds using polar coordinates and Cartesian transformation; camera images undergo distortion correction and color normalization; water quality and meteorological data are cached as time series. On the visual side, a lightweight segmentation network is used to obtain masks and target boxes; the sonar point cloud is projected onto the image plane, and candidate point sets are selected based on the mask, introducing spatial continuity and intensity consistency losses to suppress false alarms; subsequently, texture boundary features and geometric intensity features are jointly encoded in a cross-modal Transformer fusion network to output target category and confidence level, and temporal smoothing is performed using a uniform speed model.

[0065] Environmental assessment and cloud migration: Indicators such as wind speed and direction, temperature and humidity, air pressure, pH, ammonia nitrogen, and water temperature are uploaded to the host computer via a 5G link, and the display of indicator curves is supported.

[0066] This invention supports multi-tasking operation mode:

[0067] Autonomous cruise mode: The control system generates speed and heading commands based on preset track and channel constraints;

[0068] Dynamic obstacle avoidance mode: When the multibeam sonar 11 detects an obstacle ahead, the control system generates a safe course and adjusts the propulsion speed in real time;

[0069] Target tracking mode: After detecting a specific target, hull 1 automatically maintains distance and orientation to achieve continuous tracking and data collection;

[0070] Site monitoring mode: hovers at a designated coordinate point and periodically collects underwater images and acoustic information.

[0071] The monitoring method for unmanned monitoring vessels based on sonar-visual meteorological and water quality fusion as described in this invention includes the following processes:

[0072] Step 1: Use a remote controller to control the twin propeller structure, guide the monitoring vessel into the work area, generate cruise waypoints on the map, and set the cruise route and cruise time of the monitoring vessel.

[0073] Step 2: Once the monitoring vessel enters the monitoring area, the remote controller lowers the boom 2. When the underwater camera 10 and multibeam sonar 11 reach a suitable depth, the Raspberry Pi 4B main control platform activates the underwater camera 10 and multibeam sonar 11 and begins recording. The recorded information is transmitted to the Raspberry Pi 4B main control platform. The Raspberry Pi 4B main control platform processes the collected data, as follows:

[0074] Step 2.1: Perform visual distortion correction and edge enhancement processing on the data acquired by the underwater camera 10 and the multibeam sonar 11:

[0075]

[0076] in, Edge intensity map; , These are the gradients in the x and y directions, respectively; , These represent the first-order gradients in the x and y directions, respectively. The data collected by the underwater camera 10 after the above processing is used for lightweight segmentation in the subsequent step 2.4, and the data collected by the multibeam sonar 11 after the above processing is used for boundary alignment in the subsequent step 2.10.

[0077] Step 2.2: Convert sonar polar coordinates to Cartesian coordinates and perform attitude compensation:

[0078]

[0079] in, For range; It is the bend angle; For deep compensation; This is a sonar point.

[0080] Step 2.3: Transform the sonar points from the sonar Cartesian coordinate system to the underwater camera coordinate system and project them onto the image plane:

[0081]

[0082] in, For camera attachment points; These are homogeneous pixel coordinates; They are homogeneous and equivalent; Indicates a preprocessed image; Represents the camera intrinsic parameter matrix; This represents the rigid body transformation matrix from the sonar Cartesian coordinate system to the camera coordinate system. Step 2.3 outputs the position of the sonar point on the image plane, which is used for mask clipping in the subsequent step 2.6.

[0083] Step 2.4: The data acquired by the underwater camera 10 after processing in Step 2.1 and the data acquired by the multibeam sonar 11 after processing in Step 2.3 are processed together as follows: A candidate box set is obtained using a lightweight segmentation network. :

[0084]

[0085] in, , Center of the frame; , Width and height; This represents the set of candidate boxes. The output of step 2.4 is used for mask clipping and graph structure construction in subsequent steps 2.6–2.7, and provides the visual token range for steps 2.12 and 2.13.

[0086] Step 2.5: Construct a weighted sum of Dice loss and Focal loss to improve the segmentation quality of the lightweight segmentation network in Step 2.4.

[0087]

[0088] in, For predicting masks; As a truth mask; , For loss weights; Indicates the total loss; Indicates Focal loss; This indicates Dice's loss.

[0089] Step 2.6: Predict the candidate point set after mask cropping, and constrain the sonar points with visual foreground to reduce false alarms:

[0090]

[0091] in, For candidate point set; Foreground selection for masking.

[0092] Step 2.7: Construct an adjacency graph in the range neighborhood to provide topological adjacency relationships for the triple regularization in subsequent steps 2.8 to 2.10:

[0093]

[0094] in, It is an adjacent edge set; , These are the beam angle and the range threshold, respectively. , They represent the first , The bend angle between the points; , They represent the first , The range between points; Represents a point set.

[0095] Step 2.8: Triple regularization suppresses false alarms. Triple regularization is used for the fusion score in subsequent step 2.12; the spatial continuity processing is as follows:

[0096]

[0097] in, For spatial continuity regularity; Indicates the first One camera system point; Indicates the first One camera system point; Represents a small constant; This indicates that the result of the L2 norm is squared.

[0098] Step 2.9: Strength Consistency Processing:

[0099]

[0100] in, For strength consistency regularization; For the first A cluster; Echo intensity; Indicates variance.

[0101] Step 2.10: Boundary Alignment Processing:

[0102]

[0103] in, For candidate projection boundaries; The set of image edges; For projection mapping; Indicates the boundary; This represents the coordinates of a two-dimensional pixel on the candidate boundary after projection onto the image plane; This represents the coordinates of a two-dimensional pixel in the image edge set.

[0104] Step 2.11: Density-based clustering condenses candidate points into instance-level targets (clusters), providing stable input units for cross-modal coding and detection heads.

[0105]

[0106] in, The radius is ; The minimum number of points; Determined by the core point; This represents a candidate point to be determined. Indicates and Compared to neighboring points; Indicates the number of points.

[0107] Step 2.12: Perform cross-modal Transformer fusion discrimination. First, construct sonar token features, encode sonar attributes, vectorize acoustic geometry and intensity information, and align them with visual features to the same dimensional space to facilitate Transformer fusion.

[0108]

[0109] in, For the first A sonar attribute; For bundle index; Features of sonar tokens; Represents the real number field; These represent the three-dimensional coordinates of the sonar point in the selected coordinate system; Indicates azimuth; This indicates the echo intensity at that sonar point; Indicates the range; This represents a multilayer perceptron, used to map the raw attributes of sonar points into fixed-dimensional sonar token features.

[0110] Step 2.13: Visual ROI pooling processing:

[0111]

[0112] in, Visual token features; Main backbone network; For pooling; express Image frames after preprocessing at a given time.

[0113] Step 2.14: Employing a bidirectional cross-attention + self-attention mechanism, explicitly align visual boundary textures with sonar geometric intensity to improve cross-modal consistency modeling capabilities.

[0114]

[0115] in, , , These are respectively query, key, and value; For dimensions; For normalized weights.

[0116] Step 2.15: The detection head outputs the category, confidence score, and bounding box to generate a decisionable target-level determination result, namely, category, confidence score, and location.

[0117]

[0118] in, Features of fusion; Distribution by category; Confidence level; For borders; , As weight; , For bias; Represents the sigmoid function; This represents a multilayer perceptron.

[0119] Step 2.16: Target association and temporal smoothing. Construct the association cost matrix and use the Hungarian algorithm for matching to stably associate the same entity in multi-target scenarios and reduce track breaks.

[0120]

[0121] in, For matching costs; For intersection, union, and comparison; This is the observation vector for the current frame; To predict the vector to the measurement domain; For covariance; , , As weight; Let represent the confidence score of the i-th detected target.

[0122] Step 2.17: Uniform Kalman and EMA smoothing suppresses jitter and missing measurements, providing continuous and reliable pose and motion trends:

[0123]

[0124] in, State; For unit array; For covariance; The observation matrix; To measure the noise covariance; Kalman gain; Confidence level; Indicates the interval between adjacent frames; Indicates For the current frame, based on The predicted prior state; Represents the prior error covariance, characterizing Uncertainty; Represents the observation matrix Transpose of; Indicates time The posterior state.

[0125] Step 3: Using the sonar and underwater video data with fish confidence scores and borders obtained in Step 2, calculate the pixel ratio of each fish to estimate its length, and estimate its weight using the linear relationship between length and weight. This data is transmitted to a cloud platform via the Raspberry Pi 4B's 5G link. Users can observe the calculated sonar and image data in real time on the cloud platform, easily and intuitively viewing the length and weight of underwater fish. Simultaneously, users can view indicators such as wind speed and direction, temperature, air pressure, pH, ammonia nitrogen, and water temperature on the cloud platform.

[0126] The embodiments described above are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A monitoring method for an unmanned monitoring vessel based on sonar-visual-meteorological-water-quality fusion, characterized in that, The unmanned monitoring vessel based on sonar vision meteorological water quality fusion includes a hull (1) with a catamaran floating structure (9), a propulsion module (7) installed at the stern of the hull (1), and a lifting rod (2) controlled by a remote controller installed on the hull (1). The main body of the lifting rod (2) is fixed on the hull (1), and the end of the telescopic part is fixed with an installation plate. A multibeam sonar (11) and an underwater camera (10) are symmetrically fixed on the installation plate. A wind speed and wind direction instrument (4) is installed on the top of the lifting rod (2), and a temperature and humidity sensor and a pressure sensor are arranged inside. An automatic telescopic sampling arm controlled by a remote controller is set on the inner side of one of the sub-hulls of the catamaran floating structure (9). A composite water quality probe (3) is installed at the end of the automatic telescopic sampling arm to collect data on pH, dissolved oxygen, ammonia nitrogen, and water temperature. A control box (6) is also installed on the hull (1) near the stern. The control box (6) contains a lithium battery pack, a communication module, a Beidou satellite navigation module, and a control system. A solar panel (5) is installed above the control box (6). The monitoring methods include the following processes: Step 1: Use the remote controller to control the propulsion module (7) to guide the monitoring vessel into the work area, generate cruise waypoints on the map, and set the cruise route and cruise time of the monitoring vessel. Step 2: Once the monitoring vessel enters the monitoring area, the remote controller is used to control the lifting boom (2) to descend. When the underwater camera (10) and multibeam sonar (11) descend to the designated underwater depth, the control system controls the underwater camera (10) and multibeam sonar (11) to turn on and record. The recorded information is transmitted back to the control system for analysis and processing. The specific processing procedure is as follows: Step 2.1: Perform visual distortion correction and edge enhancement processing on the data collected by the underwater camera (10) and multibeam sonar (11); Step 2.2: Convert the sonar polar coordinates to Cartesian coordinates and perform attitude compensation; Step 2.3: Transform the sonar points from the sonar Cartesian coordinate system to the underwater camera coordinate system and project them onto the image plane; Step 2.4: The data collected by the underwater camera (10) after processing in Step 2.1 and the data collected by the multibeam sonar (11) after processing in Step 2.3 are processed together as follows: a set of candidate boxes is obtained using a lightweight segmentation network; Step 2.5: Construct a weighted sum of Dice loss and Focal loss to improve the segmentation quality of the lightweight segmentation network in Step 2.4; Step 2.6: Predict the candidate point set after mask cropping, and constrain the sonar points with visual foreground to reduce false alarms; Step 2.7: Construct an adjacency graph in the range neighborhood to provide topological adjacency relationships for subsequent triple regularization; Step 2.8: Use triple regularization to suppress false alarms. Triple regularization includes spatial continuity regularization, intensity consistency regularization, and boundary alignment regularization. First, perform spatial continuity regularization. Step 2.9: Intensity consistency regularization processing; Step 2.10: Boundary alignment regularization processing; Step 2.11: Density-based clustering condenses candidate points into instance-level targets, providing stable input units for cross-modal coding and detection heads; Step 2.12: Perform cross-modal Transformer fusion discrimination. First, construct sonar token features, encode sonar attributes, vectorize acoustic geometry and intensity information, and align them with visual features in the same dimensional space to facilitate Transformer fusion. Step 2.13: Visual ROI pooling processing; Step 2.14: Employ a bidirectional cross-attention + self-attention mechanism to explicitly align visual boundary textures with sonar geometric intensity, thereby enhancing cross-modal consistency modeling capabilities; Step 2.15: The detection head outputs the category, confidence score, and bounding box to generate a decisionable target-level judgment result, namely, category, confidence score, and location; Step 2.16: Optimize the results obtained in Step 2.15 to reduce errors; Step 3: After obtaining the sonar and underwater video data with fish confidence scores and borders after processing in Step 2, calculate the pixel ratio of each fish to estimate the fish length, estimate the weight of each fish through the linear relationship between fish length and weight, and transmit the obtained data to the cloud platform through 5G transmission so that users can observe the underwater fish length and weight data in real time. At the same time, they can view the wind speed and direction, temperature and humidity, air pressure, pH, ammonia nitrogen and water temperature index data collected by the wind speed and direction instrument (4), temperature and humidity sensor, air pressure sensor and composite water quality probe (3) in real time.

2. The monitoring method according to claim 1, characterized in that, In step 2.5, the weighted sum of Dice loss and Focal loss is constructed using the following formula to improve the segmentation quality of the lightweight segmentation network in step 2.4: ； in, For predicting masks; As a truth mask; , For loss weights; Indicates the total loss; Indicates Focal loss; This indicates Dice's loss.

3. The monitoring method according to claim 1, characterized in that, In step 2.8, the spatial continuity regularization is performed as follows: ； in, For spatial continuity regularity; Indicates the first One camera system point; Indicates the first One camera system point; Represents a small constant; Represents a point set; It is an adjacent edge set; This indicates that the result of the L2 norm is squared.

4. The monitoring method according to claim 1, characterized in that, In step 2.9, the intensity consistency regularization process is as follows: ； in, For strength consistency regularization; For the first A cluster; Echo intensity; Indicates variance.

5. The monitoring method according to claim 1, characterized in that, In step 2.10, the boundary alignment regularization is performed as follows: ； in, For candidate projection boundaries; The set of image edges; For projection mapping; Indicates the boundary; This represents the coordinates of a two-dimensional pixel on the candidate boundary after projection onto the image plane; This represents the coordinates of a two-dimensional pixel in the image edge set; This indicates that the result of the L2 norm is squared.

6. The monitoring method according to claim 1, characterized in that, The optimization process in step 2.16 includes: Target association and temporal smoothing are achieved by constructing an association cost matrix and matching it using the Hungarian algorithm. This ensures stable association of the same entity in multi-target scenarios and reduces track breakage. ； in, For matching costs; For intersection, union, and comparison; This is the observation vector for the current frame; To predict the vector to the measurement domain; For covariance; , , As weight; Indicates the first Confidence score of each detection target; Indicates matrix transpose; Uniform Kalman and EMA smoothing suppress jitter and missing measurements, providing continuous and reliable pose and motion trends: ； in, State; For unit array; For covariance; The observation matrix; To measure the noise covariance; Kalman gain; Confidence level; Indicates the interval between adjacent frames; Indicated by For the current frame, based on The predicted prior state; Represents the prior error covariance, characterizing Uncertainty; Represents the observation matrix transpose; Indicates time The posterior state.

Images