Net cage fouling identification and tracking integrated device and method based on electromagnetic and optical fusion perception

Abstract

The invention discloses a net cage contamination identification and tracking integrated device and method based on electromagnetic and optical fusion perception, belongs to the technical field of intelligent marine equipment and aquaculture engineering, is used for net cage contamination identification and cleaning, and comprises an upper computer, a master mobile operation platform and a slave mobile operation platform, each mobile platform carries a multi-mode sensing unit, an intelligent processing unit and a cooperative control system; the method comprises the steps of multi-modal data synchronous acquisition and preprocessing, electromagnetic and optical fusion identification and tracking, accurate positioning and global mapping, and intelligent evaluation and collaborative decision making. Through deep fusion of optical and electromagnetic perception, the precision and robustness of underwater fouling identification are greatly improved, through integrated design of identification and tracking, dual-machine high-precision stable cooperation is realized, and the problem that a single tracking mode is liable to lose tracking is solved.

Description

Technical Field

[0001] This invention discloses an integrated device and method for identifying and tracking cage fouling based on electromagnetic and optical fusion sensing, belonging to the field of intelligent marine equipment and aquaculture engineering technology. Background Technology

[0002] Currently, cage cleaning is transitioning from high-risk, high-cost manual underwater cleaning to automated robotic cleaning. However, most existing automated cleaning technologies focus on optimizing the mechanical performance of cleaning actuators (such as brush rollers and high-pressure water jets), while in the crucial "perception-decision-execution" closed loop, there are generally shortcomings such as "inaccurate perception, unintelligent decision-making, and weak coordination," making it impossible to achieve intelligent, precise, and collaborative operations based on the actual contamination situation.

[0003] Existing technologies suffer from the following problems: single optical sensing schemes are extremely unreliable in underwater environments. Water turbidity, changes in lighting, shadows from adhering organisms, and the texture of the netting itself can all severely interfere with image quality, leading to misjudgments and missed detections of fouled organisms. It struggles to accurately distinguish between different fouled organisms with similar colors and textures (such as different types of algae or shellfish), and cannot precisely quantify the density of fouling coverage. Tracking schemes based on single optical markers are vulnerable to loss of tracking targets due to water turbidity, netting obstruction, or changes in the robot's posture, resulting in interruptions in collaborative operations and potentially damage to the netting or the robot due to erroneous actions. Furthermore, these schemes cannot correlate local sensing information with the overall state of the net cage. They either blindly traverse the cleaning process or perform reactive, localized cleaning based on a "what you see is what you get" approach, failing to remember which areas have been cleaned and which areas are heavily fouled and require priority treatment. This inevitably leads to lengthy cleaning paths, low efficiency, energy waste, and even blind spots in the cleaning process. It is impossible to adaptively adjust cleaning parameters (such as scrubbing intensity, travel speed, and number of cleaning cycles) according to the type and severity of fouling organisms. It may not clean hard barnacles adequately, while it may over-clean soft algae, resulting in inefficiency and energy waste, and may even damage the netting due to over-cleaning. Summary of the Invention

[0004] The purpose of this invention is to provide an integrated device and method for identifying and tracking cage contamination based on electromagnetic and optical fusion sensing, in order to solve the problems of weak sensing capabilities, low reliability of tracking coordination, lack of global basis for decision-making, and inability to achieve refined operations in the prior art.

[0005] An integrated device and method for identifying and tracking cage contamination based on electromagnetic and optical sensing, comprising: The system comprises a host computer, a main mobile work platform, and slave mobile work platforms, each equipped with a multimodal sensing unit, an intelligent processing unit, and a collaborative control system. The multimodal sensing unit includes an optical imaging module, an electromagnetic detection module, a positioning unit, and a central processing unit; The intelligent processing unit includes an optical recognition module, an electromagnetic detection module, a fusion recognition module, a tracking system, a sensor fusion positioning module, a mapping module, an evaluation module, and a decision-making system. A collaborative control system includes a control module and a communication system; The host computer issues commands, and the main mobile work platform and the slave mobile work platform begin operations. Data acquisition and positioning are performed through the multimodal sensing unit, optical imaging module acquires optical images, electromagnetic detection module acquires electromagnetic signals, and positioning unit and central processing unit acquire positioning data. The acquired data and positioning data are input into the intelligent processing unit, optical recognition module recognizes optical images, electromagnetic detection module recognizes electromagnetic signals, fusion recognition module fuses the optical image recognition results and electromagnetic signal recognition results, tracking system tracks optical data and electromagnetic signals, sensor fusion positioning module fuses all positioning sensor information, mapping module maps local recognition results to the global coordinate system, evaluation module analyzes the mapping module results, outputs the contamination level to the decision system, and the decision system generates work commands based on the contamination level. In the collaborative control system, the main mobile work platform is the control center. The main mobile work platform control module receives instructions from the host computer and coordinates the actions of the slave mobile work platforms. The communication system includes wired communication between the host computer and the main mobile work platform, and wireless communication between the main mobile work platform and the slave mobile work platforms.

[0006] The main mobile work platform and the front end of the secondary mobile work platform are equipped with underwater cameras. The underwater cameras are equipped with flat glass waterproof covers and LED supplementary lights. Multiple Optical markings and ID encoding; The electromagnetic sensing module consists of a high-frequency signal source, a drive coil, an induction coil, and a signal processing circuit. The high-frequency signal source is generated based on a direct digital frequency synthesizer chip. The drive coil is installed at the bottom of the main mobile work platform, and the induction coil is installed inside the slave mobile work platform. The signal processing circuit consists of a lock-in amplifier and a microcontroller.

[0007] S1. The optical recognition module performs optical recognition processing through an improved U-Net neural network. The electromagnetic detection module processes electromagnetic signals into feature vectors and inputs them into a support vector machine classifier for electromagnetic recognition processing. The fusion recognition module uses a dynamic weighting mechanism based on environmental adaptation for dynamic weight fusion recognition. The tracking system adopts an optical and electromagnetic complementary architecture and fuses the two tracking data through a Kalman filter. S2. The sensor fusion positioning module uses an extended Kalman filter to fuse all positioning sensor information and outputs the six-degree-of-freedom pose of the mobile work platform. The mapping module maps the local recognition results to the global coordinate system by establishing a two-dimensional grid map corresponding to the size of the cage, and generates a global dirt heat map. S3. The assessment module quantifies the degree of contamination based on the identification results and classifies the contamination level. The decision system generates adaptive cleaning decisions based on the contamination level. The control module adopts master-slave motion planning and establishes a safety protection mechanism.

[0008] S2 includes S2.1, where optical images are segmented at the pixel level using an improved U-Net neural network. The network structure includes an encoder, a decoder, and skip connections. The encoder includes 5 layers of convolutional downsampling to extract image depth features; the decoder includes 5 layers of deconvolutional upsampling to restore spatial resolution; and the skip connections concatenate encoder features with decoder features to preserve detail information. S2 includes, S2.2, processing the electromagnetic signal into a feature vector. , This is due to the resonant frequency shift. For quality factor changes, For phase difference, The input is fed into a support vector machine classifier, which outputs a probability distribution of the contamination category based on electromagnetic properties. S2 includes S2.3, and the fusion recognition module adopts a dynamic weighting mechanism based on environment adaptation. ; ; ; In the formula, To integrate the recognition results, For optical recognition results, Electromagnetic identification results For the dynamic weights of optical recognition results, Dynamic weights for electromagnetic identification results. It is an exponentially decaying function. α is the real-time water optical attenuation coefficient, estimated using an image sharpness algorithm; α is an empirical adjustment coefficient.

[0009] S2 includes S2.4, the tracking system adopts an optical and electromagnetic complementary architecture, when When, optical tracking is used; when At that time, electromagnetic induction tracking is used, and the two tracking data are fused by a Kalman filter. The tracking control algorithm adopts an adaptive PID controller, which dynamically adjusts the KP parameter according to the tracking error. ; In the formula, This is the adaptively adjusted proportional gain coefficient. The basic proportional gain coefficient, For adaptive adjustment coefficient, This is for tracking error.

[0010] S3 includes, S3.1, acquiring acceleration and angular velocity data through an IMU, acquiring relative pose changes through a visual odometry, acquiring displacement information through an encoder odometry, and acquiring relative position constraints through electromagnetic tracking; An extended Kalman filter is used to fuse acceleration, angular velocity, relative pose change, displacement information, and relative position constraints to output the six-degree-of-freedom pose of the mobile work platform. , For rolling, For looking up and down, To yaw, X-axis coordinates Y-axis coordinate This is the Z-axis coordinate.

[0011] S3 includes S3.2, the mapping module, which maps the local recognition results to the global coordinate system. This includes creating a two-dimensional grid map corresponding to the cage size. Each grid cell stores the probability distribution of the soiling type, soiling coverage density, comprehensive soiling index, timestamp, and confidence level. The soiling coverage density is set to a value... Finally, a global contamination heatmap is generated.

[0012] S4 includes, S4.1, establishing a comprehensive pollution index model: ; In the formula, To calculate the comprehensive pollution index, For the region China's comprehensive pollution index, For the region middle, For the first Hazard weighting factors for fouling organisms, number 1 Cover density of fouling organisms, For regional indexes, Index for categories of soiled organisms.

[0013] S4 includes, S4.2, according to Classify the level of soiling, with light soiling being: Moderate soiling is Severely soiled ; The decision-making system generates work instructions based on the level of contamination: for lightly contaminated areas, standard cleaning parameters are used; for moderately contaminated areas, cleaning intensity is increased; and for heavily contaminated areas, focused cleaning is performed.

[0014] S4 includes S4.3, a main mobile work platform and a slave mobile work platform that coordinate with each other via wireless communication. The main mobile work platform is the control center, which receives instructions from the host computer and coordinates the actions of the slave mobile work platforms. The slave mobile work platforms provide real-time feedback of status parameters, including battery power, motor temperature and water pressure. Maintain relative pose constraints between the master and slave mobile work platforms. When communication interruption exceeds 300ms, the slave mobile work platform enters safe mode.

[0015] Compared with existing technologies, the present invention has the following advantages: The present invention significantly improves the accuracy and robustness of underwater fouling identification through the deep integration of optical and electromagnetic sensing; through the integrated design of identification and tracking, it achieves high-precision and stable collaboration between two machines, solving the problem of easy loss of tracking in a single tracking method; the precise cleaning path generated based on the global fouling heat map improves cleaning efficiency, while avoiding cage deformation and fish disturbance, realizing a technological leap from blind cleaning to intelligent targeted cleaning. Attached Figure Description

[0016] Figure 1 This is a flowchart of the technology of this invention; Figure 2 It is a dynamically weighted adaptive adjustment graph; Figure 3 It is a real-time turbidity change detection graph; Figure 4 This is a comparison chart of tracking errors between two machines; Figure 5 This is a comparison chart of the stability of different tracking methods; Figure 6 This is a distribution map of fouling organism types; Figure 7 It is an optimized cleaning path diagram based on the contamination heat map; Figure 8 It is a heat map of dirt cover density; Figure 9 It is a map showing the distribution of soiling levels; Figure 10 This is a comparison chart showing the clearer images of the fish cages over time; Figure 11 This is a comparison chart of cleaning energy consumption; Figure 12 This is a comparison chart of mesh clothing washing coverage. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention are described clearly and completely below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0018] An integrated device and method for identifying and tracking cage contamination based on electromagnetic and optical sensing, comprising: The system consists of a host computer, a main mobile work platform, and slave mobile work platforms. Each mobile platform is equipped with a multimodal sensing unit, an intelligent processing unit, and a collaborative control system. The multimodal sensing unit includes an optical imaging module, an electromagnetic detection module, a positioning unit, and a central processing unit; The intelligent processing unit includes an optical recognition module, an electromagnetic detection module, a fusion recognition module, a tracking system, a sensor fusion positioning module, a mapping module, an evaluation module, and a decision-making system. A collaborative control system includes a control module and a communication system; The host computer issues commands, and the main mobile work platform and the slave mobile work platform begin operations. Data acquisition and positioning are performed through the multimodal sensing unit, optical imaging module acquires optical images, electromagnetic detection module acquires electromagnetic signals, and positioning unit and central processing unit acquire positioning data. The acquired data and positioning data are input into the intelligent processing unit, optical recognition module recognizes optical images, electromagnetic detection module recognizes electromagnetic signals, fusion recognition module fuses the optical image recognition results and electromagnetic signal recognition results, tracking system tracks optical data and electromagnetic signals, sensor fusion positioning module fuses all positioning sensor information, mapping module maps local recognition results to the global coordinate system, evaluation module analyzes the mapping module results, outputs the contamination level to the decision system, and the decision system generates work commands based on the contamination level. In the collaborative control system, the main mobile work platform is the control center. The main mobile work platform control module receives instructions from the host computer and coordinates the actions of the slave mobile work platforms. The communication system includes wired communication between the host computer and the main mobile work platform, and wireless communication between the main mobile work platform and the slave mobile work platforms.

[0019] The main mobile work platform and the front end of the secondary mobile work platform are equipped with underwater cameras. The underwater cameras are equipped with flat glass waterproof covers and LED supplementary lights. Multiple Optical markings and ID encoding; The electromagnetic sensing module consists of a high-frequency signal source, a drive coil, an induction coil, and a signal processing circuit. The high-frequency signal source is generated based on a direct digital frequency synthesizer chip. The drive coil is installed at the bottom of the main mobile work platform, and the induction coil is installed inside the slave mobile work platform. The signal processing circuit consists of a lock-in amplifier and a microcontroller.

[0020] S1. The optical recognition module performs optical recognition processing through an improved U-Net neural network. The electromagnetic detection module processes electromagnetic signals into feature vectors and inputs them into a support vector machine classifier for electromagnetic recognition processing. The fusion recognition module uses a dynamic weighting mechanism based on environmental adaptation for dynamic weight fusion recognition. The tracking system adopts an optical and electromagnetic complementary architecture and fuses the two tracking data through a Kalman filter. S2. The sensor fusion positioning module uses an extended Kalman filter to fuse all positioning sensor information and outputs the six-degree-of-freedom pose of the mobile work platform. The mapping module maps the local recognition results to the global coordinate system by establishing a two-dimensional grid map corresponding to the size of the cage, and generates a global dirt heat map. S3. The assessment module quantifies the degree of contamination based on the identification results and classifies the contamination level. The decision system generates adaptive cleaning decisions based on the contamination level. The control module adopts master-slave motion planning and establishes a safety protection mechanism.

[0021] S2 includes S2.1, where optical images are segmented at the pixel level using an improved U-Net neural network. The network structure includes an encoder, a decoder, and skip connections. The encoder includes 5 layers of convolutional downsampling to extract image depth features; the decoder includes 5 layers of deconvolutional upsampling to restore spatial resolution; and the skip connections concatenate encoder features with decoder features to preserve detail information. S2 includes, S2.2, processing the electromagnetic signal into a feature vector. , This is due to the resonant frequency shift. For quality factor changes, For phase difference, The input is fed into a support vector machine classifier, which outputs a probability distribution of the contamination category based on electromagnetic properties. S2 includes S2.3, and the fusion recognition module adopts a dynamic weighting mechanism based on environment adaptation. ; ; ; In the formula, To integrate the recognition results, For optical recognition results, Electromagnetic identification results For the dynamic weights of optical recognition results, Dynamic weights for electromagnetic identification results. It is an exponentially decaying function. α is the real-time water optical attenuation coefficient, estimated using an image sharpness algorithm; α is an empirical adjustment coefficient.

[0022] S2 includes S2.4, the tracking system adopts an optical and electromagnetic complementary architecture, when When, optical tracking is used; when At that time, electromagnetic induction tracking is used, and the two tracking data are fused by a Kalman filter. The tracking control algorithm adopts an adaptive PID controller, which dynamically adjusts the KP parameter according to the tracking error. ; In the formula, This is the adaptively adjusted proportional gain coefficient. The basic proportional gain coefficient, For adaptive adjustment coefficient, This is for tracking error.

[0023] S3 includes, S3.1, acquiring acceleration and angular velocity data through an IMU, acquiring relative pose changes through a visual odometry, acquiring displacement information through an encoder odometry, and acquiring relative position constraints through electromagnetic tracking; An extended Kalman filter is used to fuse acceleration, angular velocity, relative pose change, displacement information, and relative position constraints to output the six-degree-of-freedom pose of the mobile work platform. , For rolling, For looking up and down, To yaw, X-axis coordinates Y-axis coordinate This is the Z-axis coordinate.

[0024] S3 includes S3.2, the mapping module, which maps the local recognition results to the global coordinate system. This includes creating a two-dimensional grid map corresponding to the cage size. Each grid cell stores the probability distribution of the soiling type, soiling coverage density, comprehensive soiling index, timestamp, and confidence level. The soiling coverage density is set to a value... Finally, a global contamination heatmap is generated.

[0025] S4 includes, S4.1, establishing a comprehensive pollution index model: ; In the formula, To calculate the comprehensive pollution index, For the region China's comprehensive pollution index, For the region middle, For the first Hazard weighting factors for fouling organisms, number 1 Cover density of fouling organisms, For regional indexes, Index for categories of soiled organisms.

[0026] S4 includes, S4.2, according to Classify the level of soiling, with light soiling being: Moderate soiling is Severely soiled ; The decision-making system generates work instructions based on the level of contamination: for lightly contaminated areas, standard cleaning parameters are used; for moderately contaminated areas, cleaning intensity is increased; and for heavily contaminated areas, focused cleaning is performed.

[0027] S4 includes S4.3, a main mobile work platform and a slave mobile work platform that coordinate with each other via wireless communication. The main mobile work platform is the control center, which receives instructions from the host computer and coordinates the actions of the slave mobile work platforms. The slave mobile work platforms provide real-time feedback of status parameters, including battery power, motor temperature and water pressure. Maintain relative pose constraints between the master and slave mobile work platforms. When communication interruption exceeds 300ms, the slave mobile work platform enters safe mode.

[0028] The technical solution adopted by this invention to solve its technical problem is: to construct an integrated device and method based on electromagnetic-optical fusion sensing, identification, and tracking. This system consists of four main parts: a mobile operating platform, a multimodal sensing unit, an intelligent processing unit, and a collaborative control system, realizing integrated operation throughout the entire process from contamination identification and dual-machine tracking to intelligent cleaning. The overall technical architecture of this invention is as follows: Figure 1 As shown, the first step is the synchronous acquisition and preprocessing of multimodal data, including optical data acquisition, electromagnetic data acquisition, and pose data acquisition.

[0029] Optical data acquisition includes installing a 2-megapixel industrial-grade underwater camera on the front end of both the host and slave units, equipped with a flat glass waterproof cover and an adjustable-angle LED fill light; the camera acquires images of the mesh surface at a rate of 10 frames per second, with the image resolution set to 1920×1080; at the same time, multiple redundant Apriltag optical tags are arranged on the surface of the host unit, with a tag size of 30cm×30cm, and are distinguished by different ID codes.

[0030] The electromagnetic data acquisition module consists of a high-frequency signal source, a drive coil, and an induction coil. The high-frequency signal source is based on a direct digital frequency synthesizer chip, generating a 1-50MHz sine wave signal. The drive coil is wound with enameled wire, has a diameter of 15cm, and an inductance of approximately 10μH, and is mounted on the bottom of the main unit. The slave unit has two sets of induction coils, used to detect changes in the magnetic field in the horizontal and vertical directions, respectively. The signal processing circuit is based on a lock-in amplifier and a microcontroller, which measures the resonant frequency shift in real time. and quality factor changes .

[0031] Pose data acquisition includes installing an inertial measurement unit (IMU) at the center of gravity of both the master and slave units, which contains a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer; using the robot's thruster speed encoder to provide odometer information; and transmitting all sensor data to the central processing unit via a CAN bus.

[0032] The second step is electromagnetic-optical fusion recognition and tracking, which includes optical recognition processing, electromagnetic recognition processing, dynamic weighted fusion recognition, and dual-modal collaborative tracking.

[0033] Optical recognition processing includes pixel-level semantic segmentation of optical images using an improved U-Net neural network. The network structure includes an encoder, a decoder, and network connections. The encoder is a 5-layer convolutional downsampling layer that extracts image depth features; the decoder is a 5-layer deconvolutional upsampling layer that restores spatial resolution; skip connections concatenate encoder and decoder features to preserve detailed information; the network outputs the probability distribution of each pixel belonging to categories such as background, clean netting, barnacles, oysters, and algae.

[0034] Electromagnetic identification processing includes forming feature vectors from electromagnetic signals after preprocessing. The input is fed into a support vector machine classifier, which outputs a probability distribution of the contamination category based on electromagnetic properties.

[0035] The dynamic weighted fusion identification includes a fusion identification module that employs an environment-adaptive dynamic weighting mechanism for fusion. This invention's dynamic weighted fusion mechanism adjusts the confidence levels of optical and electromagnetic sensor data in real time based on water turbidity. For example... Figure 2 and Figure 3 As shown, when the turbidity of the water body When raised, the system automatically reduces optical weight. Accordingly increase electromagnetic weight This ensures the system's robustness in complex underwater environments.

[0036] Dual-modal cooperative tracking involves a tracking system employing an optical and electromagnetic complementary architecture: In clear water (c(t) < 0.3), Apriltag optical tracking is used preferentially, with an accuracy of ±2cm; in turbid water (c(t) ≥ 0.3), it automatically switches to electromagnetic induction tracking, with an accuracy of ±5cm; the two tracking data are fused by a Kalman filter to ensure the continuity and stability of tracking; the tracking control algorithm uses an adaptive PID controller to dynamically adjust the KP parameter according to the tracking error; the dual-modal redundant tracking method of this invention effectively solves the technical problem of easy tracking loss in a single tracking method. Figure 4 and Figure 5 As shown, the success rate of dual-modal fusion tracking reached 98%, which is significantly higher than that of single optical tracking (72%) and single electromagnetic tracking (85%), and the tracking error was stably controlled within ±3cm.

[0037] The third step is precise localization and global mapping, including multi-sensor fusion localization and global contamination distribution mapping. Multi-sensor fusion localization uses an extended Kalman filter to fuse the following localization information: acceleration and angular velocity data provided by the IMU (200Hz sampling rate); relative pose changes provided by visual odometry (ORB-SLAM3 algorithm); displacement information provided by encoder odometry; relative position constraints provided by electromagnetic tracking; and the output robot's six-DOF pose. The positioning accuracy reaches ±3cm; Global contamination distribution mapping includes a mapping module that maps local identification results to a global coordinate system: establishing a two-dimensional grid map corresponding to the cage size, with a grid resolution of 5cm×5cm; each grid stores the following information: probability distribution of contamination type and contamination coverage density. The invention combines a comprehensive fouling index, timestamp, and confidence level; it outputs a fouling distribution heatmap. The global fouling distribution heatmap constructed in this invention enables a quantitative assessment of fouling conditions. Figure 6 , Figure 7 , Figure 8 , Figure 9 As shown, the system can accurately identify the level of contamination (light, moderate, and heavy) in different areas and generate an optimized cleaning path based on the contamination heat map to achieve targeted cleaning.

[0038] The fourth step is intelligent assessment and collaborative decision-making, including quantitative assessment of fouling degree, adaptive cleaning decision-making, and dual-machine collaborative control. Quantitative assessment of fouling degree involves establishing a comprehensive fouling index model, in which the hazard weighting factor of fouling organisms is determined in advance through experiments: barnacles 0.7 (hard, significant thickening), oysters 0.6, and algae 0.3 (soft, easily detached). Then, the fouling level is classified according to the SDI value.

[0039] The adaptive cleaning decision-making process involves the decision-making system generating work instructions based on the contamination assessment results: for lightly contaminated areas, standard cleaning parameters are used (brush roller speed 30 rpm, travel speed 0.3 m / s); for moderately contaminated areas, the cleaning intensity is increased (brush roller speed 45 rpm, travel speed 0.2 m / s); and for heavily contaminated areas, focused cleaning is performed (brush roller speed 60 rpm, travel speed 0.1 m / s, repeated cleaning).

[0040] The dual-machine collaborative control includes master and slave machines coordinating via 433MHz wireless communication: the master machine acts as the control center, receiving instructions from the host computer and coordinating the actions of the slave machine; the slave machine provides real-time feedback on its own status (battery power, motor temperature, water pressure, etc.); master-slave motion planning is adopted to ensure that the two machines always maintain relative posture constraints; a safety protection mechanism is established, and when the communication interruption exceeds 300ms, the slave machine automatically enters the safety mode; System hardware configuration: The mobile operation platform is configured with a six-degree-of-freedom thruster layout for both the host and slave units, providing omnidirectional motion capability, a maximum diving depth of 50 meters, a payload capacity of more than 5 kg, and an endurance of 4 hours (standard operation mode). The communication system architecture includes TCP / IP wired communication between the host computer and the main unit (zero-buoyancy armored cable), and 433MHz wireless communication between the host and slave units (transmit power 20dBm, receiver sensitivity -120dBm). The communication protocol is a custom application layer protocol, including data verification and retransmission mechanisms.

[0041] The specific embodiments of this invention are as follows. Two underwater robots equipped with six degrees of freedom thrusters are selected as the master and slave mobile platforms. A panel is installed on the front of the robot, integrating the following modules: The optical imaging module uses a 2-megapixel underwater camera (IP68), with high-brightness LED supplementary lights symmetrically arranged on both sides. The electromagnetic detection module uses a custom PCB board to integrate a DDS chip (generating 1-50MHz sine waves) and a lock-in amplifier circuit. The electromagnetic characteristic detection module can also be implemented by combining a frequency response analyzer with a sensing coil, replacing the signal processing circuit based on the lock-in amplifier; in scenarios where high maneuverability is not required, the mobile platform can also use a crawling robot running along a preset track instead of an underwater ROV; the water turbidity input in the dynamic weight fusion mechanism can also be directly provided by an independent turbidity sensor, replacing the estimation algorithm based on image sharpness; a 15cm diameter drive coil (master) and induction coil (slave) are installed, and the coils are waterproof potted. The positioning unit uses an IMU fixed inside the robot and uses a thruster encoder to provide mileage information. The central processing unit uses an industrial computer (equipped with an NVIDIA RTX 3060 GPU) and is connected to the host via a zero-buoyancy armored cable to establish Ethernet communication.

[0042] Within the Robot Operating System (ROS) framework, a fusion recognition module was developed using an improved U-Net network, with a fully connected branch added at the encoder end to process electromagnetic feature vectors. The image depth features and electromagnetic features are concatenated and then input into the decoder. The mapping module starts the robot_pose_ekf package, fusing IMU data and visual odometry (ORB-SLAM3) information; it develops a global map building node, subscribes to the precise pose ( / odom topic) and recognition results, and renders them to a 2D grid map in real time. Dynamic weight control estimates the optical attenuation coefficient and calculates the weights using real-time image sharpness or dedicated turbidity sensor data.

[0043] The specific operation is as follows: The robot is deployed near the cleanroom mesh, and the IMU is initially aligned; the electromagnetic sensor reference values ​​(resonant frequency f0 and quality factor Q0) are recorded in the cleanroom mesh area. A zigzag path is set via the host computer. The robot acquires images and synchronous electromagnetic data at 10 frames per second; the central processing unit runs the fusion recognition model and outputs a soiling type segmentation map; the positioning module provides real-time pose, and the mapping node plots the recognition results on the global map. The system calculates the comprehensive soiling index (SDI) based on the global soiling heat map and automatically classifies the soiling level (light soiling is...). Moderate soiling is Severely soiled The system generates adaptive cleaning commands (such as adjusting the brush roller speed and travel speed). The host computer interface displays the operation progress in real time.

[0044] This invention achieves a cleaning efficiency increase of over 60% and a energy consumption reduction of 40%. For example... Figure 10 , Figure 11 , Figure 12 As shown, compared with the traditional preset procedure method, the fusion method of the present invention reduces the relative cleaning time from 100% to 40%, the energy consumption from 100% to 60%, and the cleaning coverage from 78% to 98%.

[0045] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An integrated device for identifying and tracking cage contamination based on electromagnetic and optical fusion sensing, characterized in that, include: The system consists of a host computer, a main mobile work platform, and slave mobile work platforms. Each mobile platform is equipped with a multimodal sensing unit, an intelligent processing unit, and a collaborative control system. The multimodal sensing unit includes an optical imaging module, an electromagnetic detection module, a positioning unit, and a central processing unit; The intelligent processing unit includes an optical recognition module, an electromagnetic detection module, a fusion recognition module, a tracking system, a sensor fusion positioning module, a mapping module, an evaluation module, and a decision-making system. A collaborative control system includes a control module and a communication system; The host computer issues commands, and the main mobile work platform and the slave mobile work platform begin operations. Data acquisition and positioning are performed through the multimodal sensing unit, optical imaging module acquires optical images, electromagnetic detection module acquires electromagnetic signals, and positioning unit and central processing unit acquire positioning data. The acquired data and positioning data are input into the intelligent processing unit, optical recognition module recognizes optical images, electromagnetic detection module recognizes electromagnetic signals, fusion recognition module fuses the optical image recognition results and electromagnetic signal recognition results, tracking system tracks optical data and electromagnetic signals, sensor fusion positioning module fuses all positioning sensor information, mapping module maps local recognition results to the global coordinate system, evaluation module analyzes the mapping module results, outputs the contamination level to the decision system, and the decision system generates work commands based on the contamination level. In the collaborative control system, the main mobile work platform is the control center. The main mobile work platform control module receives instructions from the host computer and coordinates the actions of the slave mobile work platforms. The communication system includes wired communication between the host computer and the main mobile work platform, and wireless communication between the main mobile work platform and the slave mobile work platforms.

2. The integrated device for identifying and tracking cage contamination based on electromagnetic and optical fusion sensing as described in claim 1, characterized in that, The main mobile work platform and the front end of the secondary mobile work platform are equipped with underwater cameras. The underwater cameras are equipped with flat glass waterproof covers and LED supplementary lights. Multiple Optical markings and ID encoding; The electromagnetic sensing module consists of a high-frequency signal source, a drive coil, an induction coil, and a signal processing circuit. The high-frequency signal source is generated based on a direct digital frequency synthesizer chip. The drive coil is installed at the bottom of the main mobile work platform, and the induction coil is installed inside the slave mobile work platform. The signal processing circuit consists of a lock-in amplifier and a microcontroller.

3. A method for integrated identification and tracking of cage fouling based on electromagnetic and optical fusion sensing, using the integrated device for integrated identification and tracking of cage fouling based on electromagnetic and optical fusion sensing as described in claim 1, characterized in that, include: S1. The optical recognition module performs optical recognition processing through an improved U-Net neural network. The electromagnetic detection module processes electromagnetic signals into feature vectors and inputs them into a support vector machine classifier for electromagnetic recognition processing. The fusion recognition module uses a dynamic weighting mechanism based on environmental adaptation for dynamic weight fusion recognition. The tracking system adopts an optical and electromagnetic complementary architecture and fuses the two tracking data through a Kalman filter. S2. The sensor fusion positioning module uses an extended Kalman filter to fuse all positioning sensor information and outputs the six-degree-of-freedom pose of the mobile work platform. The mapping module maps the local recognition results to the global coordinate system by establishing a two-dimensional grid map corresponding to the size of the cage, and generates a global dirt heat map. S3. The assessment module quantifies the degree of contamination based on the identification results and classifies the contamination level. The decision system generates adaptive cleaning decisions based on the contamination level. The control module adopts master-slave motion planning and establishes a safety protection mechanism.

4. The integrated method for identifying and tracking cage contamination based on electromagnetic and optical fusion sensing as described in claim 3, characterized in that, S2 includes S2.1, where optical images are segmented at the pixel level using an improved U-Net neural network. The network structure includes an encoder, a decoder, and skip connections. The encoder includes 5 layers of convolutional downsampling for extracting image depth features; the decoder includes 5 layers of deconvolutional upsampling for restoring spatial resolution. Skip connections concatenate encoder features with decoder features, preserving detailed information; S2 includes, S2.2, processing the electromagnetic signal into a feature vector. , This is due to the resonant frequency shift. For quality factor changes, For phase difference, The input is fed into a support vector machine classifier, which outputs a probability distribution of the contamination category based on electromagnetic properties. S2 includes S2.3, and the fusion recognition module adopts a dynamic weighting mechanism based on environment adaptation. ； ； ； In the formula, To integrate the recognition results, For optical recognition results, Electromagnetic identification results For the dynamic weights of optical recognition results, Dynamic weights for electromagnetic identification results. It is an exponentially decaying function. α is the real-time water optical attenuation coefficient, estimated using an image sharpness algorithm; α is an empirical adjustment coefficient.

5. The integrated method for identifying and tracking cage contamination based on electromagnetic and optical fusion sensing as described in claim 4, characterized in that, S2 includes S2.4, the tracking system adopts an optical and electromagnetic complementary architecture, when When, optical tracking is used; when At that time, electromagnetic induction tracking is used, and the two tracking data are fused by a Kalman filter. The tracking control algorithm adopts an adaptive PID controller, which dynamically adjusts the KP parameter according to the tracking error. ； In the formula, This is the adaptively adjusted proportional gain coefficient. The basic proportional gain coefficient, For adaptive adjustment coefficient, This is for tracking error.

6. The integrated method for identifying and tracking cage contamination based on electromagnetic and optical fusion sensing as described in claim 5, characterized in that, S3 includes, S3.1, acquiring acceleration and angular velocity data through an IMU, acquiring relative pose changes through a visual odometry, acquiring displacement information through an encoder odometry, and acquiring relative position constraints through electromagnetic tracking; An extended Kalman filter is used to fuse acceleration, angular velocity, relative pose change, displacement information, and relative position constraints to output the six-degree-of-freedom pose of the mobile work platform. , For rolling, For looking up and down, To yaw, X-axis coordinates Y-axis coordinate This is the Z-axis coordinate.

7. The integrated method for identifying and tracking cage contamination based on electromagnetic and optical fusion sensing as described in claim 6, characterized in that, S3 includes S3.2, the mapping module, which maps the local recognition results to the global coordinate system. This includes creating a two-dimensional grid map corresponding to the cage size. Each grid cell stores the probability distribution of the soiling type, soiling coverage density, comprehensive soiling index, timestamp, and confidence level. The soiling coverage density is set to a value... Finally, a global contamination heatmap is generated.

8. The integrated method for identifying and tracking cage contamination based on electromagnetic and optical fusion sensing as described in claim 7, characterized in that, S4 includes, S4.1, establishing a comprehensive pollution index model: ； In the formula, To calculate the comprehensive pollution index, For the region China's comprehensive pollution index, For the region middle, For the first Hazard weighting factors for fouling organisms, number 1 Cover density of fouling organisms, For regional indexes, Index for categories of contaminated organisms.

9. The integrated method for identifying and tracking cage contamination based on electromagnetic and optical fusion sensing as described in claim 8, characterized in that, S4 includes, S4.2, according to Classify the level of soiling, with light soiling being: Moderate soiling is Severely soiled ; The decision-making system generates work instructions based on the level of contamination: for lightly contaminated areas, standard cleaning parameters are used; for moderately contaminated areas, cleaning intensity is increased; and for heavily contaminated areas, focused cleaning is performed.

10. The integrated method for identifying and tracking cage contamination based on electromagnetic and optical fusion sensing as described in claim 9, characterized in that, S4 includes S4.3, a main mobile work platform and a slave mobile work platform that coordinate with each other via wireless communication. The main mobile work platform is the control center, which receives instructions from the host computer and coordinates the actions of the slave mobile work platforms. The slave mobile work platforms provide real-time feedback of status parameters, including battery power, motor temperature and water pressure. Maintain relative pose constraints between the master and slave mobile work platforms. When communication interruption exceeds 300ms, the slave mobile work platform enters safe mode.

Images