Unmanned ship cluster water surface partition collaborative classification salvage method based on deep learning

Abstract

The invention discloses a deep learning-based unmanned ship cluster water surface partition collaborative classification salvage method, and aims to solve the problem of large-area water area suspended matter recovery. A water area three-dimensional map is constructed by using an SLAM technology, environment perception is realized, areas are reasonably divided, overlapping compensation is performed, and the operation coverage rate and efficiency are improved; and performing full-coverage path planning on each partition by using an A * algorithm, and optimizing the path length. During cruising, the unmanned ship cluster obtains dynamic environment information, and obstacle avoidance is performed by using a DWA algorithm; and meanwhile, a deep learning algorithm is applied to monitor and classify suspended solids. And tracking the identified suspended matter through a DeepSORT algorithm, calculating an equivalent moving speed and a position offset of the suspended matter, predicting a future position, planning a shortest path by using a DWA algorithm according to a priority, and salvaging. After salvage, the operation state is judged according to hull data, and tasks are dynamically adjusted.

Description

Technical Field

[0001] The present invention belongs to the field of classification and recovery of suspended matter on water surfaces, and specifically provides a method for collaborative classification and salvage of water surface partitions by an unmanned boat cluster based on deep learning. Background Art

[0002] Garbage classification has now become an indispensable part of our lives. The country vigorously promotes garbage classification in order to reduce resource waste, protect the environment and achieve sustainable development. Through classification and processing, recyclables can be reused, landfill waste can be reduced, and environmental pollution can be reduced. However, with the acceleration of urbanization, in addition to garbage on land, garbage on the water surface has also become a problem that cannot be ignored. Suspended matter such as plastic bags, foam, and fallen leaves in lakes and rivers not only affects the image of the city, but also damages the aquatic environment. The traditional method of salvaging suspended matter on the water surface mainly relies on manual salvage, which is time-consuming, labor-intensive, and inefficient. In recent years, with the development of science and technology, unmanned salvage ships have been gradually introduced to salvage suspended matter on the water surface, but they can often only remove suspended matter from the water, and subsequent classification work still needs to be done manually. Therefore, how to achieve intelligent classification and salvage of suspended matter on the water surface has become a difficult problem that needs to be solved urgently.

[0003] At the same time, aquatic organisms in water bodies may suffer from certain degrees of disease and death due to changes in weather and water quality, which may pollute water quality, damage the ecological environment, and even lead to the spread of infectious diseases. Therefore, timely salvage and harmless treatment are an important task of surface salvage.

[0004] Chinese patent publication number CN109080780A proposes a salvage method for classifying and salvaging floating objects on the water surface. The method uses a surface search device to search for floating objects, uses a crushing device to salvage and crush them, then uses a property discrimination device to classify the salvaged objects, and finally uses different compression methods to store them according to the properties of the floating objects.

[0005] However, this patent has insufficient environmental adaptability and lacks dynamic obstacle avoidance capabilities. It may not be able to effectively respond to obstacles encountered in actual operations, which may cause the unmanned ship to collide or other safety hazards in complex water conditions, limiting its application effect and safety in changing environments.

[0006] Chinese patent publication number CN117522988A proposes an unmanned long-arm boat surface salvage and positioning system and method based on AI recognition. After shooting with a shooting module and performing image analysis in a learning module, the system uses a gyroscope to align the orientation and perform analysis and positioning. Finally, the system is transferred to the unmanned long-arm boat through a conveying module, and the unmanned long-arm boat is used to quickly go to the designated location and salvage the floating objects.

[0007] However, the method proposed in this patent cannot effectively adapt to changes in the water environment, especially under the influence of complex external factors such as wind direction, wind speed, and water current. Relying solely on identification and positioning technology is insufficient to ensure successful salvage of floating objects, and the lack of dynamic obstacle avoidance capabilities can lead to inefficient operations and salvage failures.

[0008] Chinese patent publication number CN118149822A proposes a fully automatic cyanobacteria salvage vessel navigation method and system based on remote sensing multispectral technology. To address the problem of cyanobacteria blooms, existing technologies for identifying cyanobacteria are affected by weather, resulting in low recognition rates. This paper proposes using thermodynamic image processing and machine learning to detect images and achieve real-time monitoring of cyanobacteria-infested areas in lakes.

[0009] Chinese patent publication number CN117037052A proposes a method for detecting dead fish on the water surface using a multi-scale feature fusion and attention mechanism. This method addresses the significant threat that floating dead fish on the water surface pose to aquatic ecology and drinking water safety. Traditional target detection methods are prone to missing or misdetecting small targets in dead fish detection. This paper proposes using computer vision-based target detection technology to improve the performance of dead fish detection on the water surface.

[0010] While the aforementioned existing patents propose effective detection methods for suspended pollutants on the water surface, such as cyanobacteria and dead fish, they still have some shortcomings. For one thing, most of them target a single type of suspended matter and lack the ability to identify, classify, and track multiple targets in complex water environments. Furthermore, these methods often focus solely on target detection while neglecting the overall optimization of the salvage and recovery mission, such as optimizing a fully covered salvage route to ensure that all suspended matter can be identified, classified, tracked, and recovered, thereby reducing voyage time and distance and extending the drone ship's endurance.

[0011] Chinese patent publication number CN115170982A proposes an artificial intelligence-based automatic surface garbage identification system. To address the low efficiency of manual salvage, the system uses an image visual recognition system to autonomously identify, locate, plan paths, and collect surface garbage, thus achieving unmanned and fully automated salvage operations for floating objects on the surface.

[0012] However, this patent uses image acquisition to collect garbage, but the proposed "after obtaining the location of the garbage, transmitting all garbage locations to the terminal control platform" has certain limitations. Obtaining the location of garbage in all waters requires a large number of camera units, which is costly. Moreover, when there is a lot of garbage in the water area, the image comparison unit requires higher computing resources for comparison and analysis to achieve garbage judgment.

[0013] Chinese patent publication number CN113359780A proposes an unmanned boat cleaning path planning method, device, computer equipment and storage medium. By obtaining the unmanned boat's area to be cleaned and a set of obstacles in the area to be cleaned, the area to be cleaned is divided, and a traversal cleaning path is generated for each sub-area to be cleaned. Each of the traversal cleaning paths is connected, so that the unmanned boat cleans according to the planned path, realizing autonomous cleaning path planning in complex water surface areas.

[0014] However, the patent uses a partitioned traversal path generation method, dividing the water area into several sub-areas and cleaning them one by one, which simplifies the static obstacle avoidance problem to a certain extent. However, it ignores possible dynamic obstacles (such as other ships), which may cause the unmanned boat to collide with dynamic obstacles, affecting the smooth progress of the cleaning task.

[0015] Chinese patent publication number CN112102369A proposes an autonomous inspection method, device, equipment and storage medium for floating targets on the water surface. A dynamic planning algorithm is used to calculate the inspection route, detect and track floating garbage on the water surface, and simultaneously identify and collect small targets on the water surface during the navigation of the unmanned boat, completing the water surface cleaning.

[0016] However, this patent only detects floating garbage on the water surface in real time. It lacks environmental perception capabilities, cannot cope with complex and changing water environments, and lacks dynamic obstacle avoidance capabilities. After collecting floating garbage, the unmanned boat returns to the preset inspection route, which can consume a lot of time and reduce overall inspection efficiency. In addition, when handling large areas of water, the inspection efficiency is low due to the limitations of a single unmanned boat. Summary of the Invention

[0017] To solve the above technical problems, the present invention proposes a deep learning-based unmanned boat cluster water surface partitioning collaborative classification and salvage method. Through the A* algorithm and the DWA algorithm, autonomous navigation and obstacle avoidance in complex water environments are realized; a deep learning model and the multi-target tracking DeepSORT algorithm are used to achieve efficient identification, classification and recovery of various suspended objects; through regional division, division of labor and cooperation are achieved, and automated classification and salvage of suspended objects by unmanned boat clusters in large water areas is realized; to address the problem of partition boundary effect, a partition overlapping mechanism is designed to improve overall operational efficiency.

[0018] To achieve the above object, the technical solution adopted by the present invention is:

[0019] A deep learning-based unmanned vessel cluster surface partitioning collaborative classification salvage method is characterized by: including;

[0020] S1: Deploy the deep learning model trained and verified with the self-built dataset to the unmanned vessel cluster edge computing platform;

[0021] Images of suspended debris on the water surface are captured in different waters, weather conditions, and time of day, and data augmentation techniques are used to expand the dataset. A visual interface is used to annotate the dataset images. After preprocessing, a deep learning model suitable for the task is selected for training. Model performance is evaluated using metrics such as mAP. Once the model reaches the expected accuracy, it is deployed on the unmanned vessel edge computing platform to achieve real-time detection and classification of suspended debris on the water surface.

[0022] S2: Use SLAM technology to build a three-dimensional map of the water area and store it as a global map to achieve environmental perception;

[0023] Using SLAM technology, a data system combining SLAM modules, sensor modules, and positioning modules is constructed. Based on the extracted obstacle speed, it is determined whether the obstacle is a static or dynamic object. This allows for the construction of a global 3D map containing the locations of static obstacles, providing spatial information for subsequent path planning and water area demarcation.

[0024] S3: Divide the water area into regions based on the global map and the complexity of the water area, and compensate for the overlap of the internal boundaries of the partitions;

[0025] S4: A* algorithm performs full coverage path planning for each sub-partition;

[0026] S5: When the unmanned boat cluster cruises along the planned path in each sub-area, it uses SLAM sensor data to update the map and uses the DWA algorithm to plan obstacle avoidance paths for obstacles identified by SLAM technology;

[0027] S6: During cruising, the image sensor collects real-time image information of the water surface around the unmanned vessel, and processes the image through a deep learning algorithm to monitor and classify suspended solids on the water surface;

[0028] S7: Use the DeepSORT algorithm to track the identified suspended objects. Through multi-sensor data fusion, the equivalent movement speed and position offset of the suspended objects are calculated based on the position coordinate information of multiple historical moments.

[0029] S8: Based on the equivalent moving speed and position offset, the possible position of the suspended object at a future moment is predicted, and according to the set salvage priority of the suspended object, the unmanned vessel is driven to the predicted position of the priority suspended object;

[0030] S9: using the DWA algorithm to plan the shortest path to the predicted location of the priority suspended object;

[0031] The DWA algorithm is used to plan the shortest path to the predicted location of the priority suspended object. By establishing a virtual force field in the overlapping area, possible conflicts can be avoided, thereby improving safety, reliability, and operational efficiency.

[0032] S10: Salvage operations;

[0033] S11: After the salvage operation, the operating status is judged based on the sensor data. When the classification recovery bin is not full and the battery is sufficient, the operation continues. If the current sub-partition has not been completely traversed, the A* algorithm is used to plan a full coverage path for the untraversed area of ​​the current sub-partition based on the current location of the unmanned ship, obstacle information and the cruise path. If the current sub-partition has been completely traversed, the A* algorithm is used again to plan a full coverage path for the current sub-partition based on the current location of the unmanned ship and obstacle information;

[0034] S12: If the classified recycling bin is full or the battery level is below a preset threshold, the A* algorithm is used to plan a route to the nearest dumping point or charging station;

[0035] If the classified recycling bin is full or the battery level is below a preset threshold, the A* algorithm is used to plan a route to the nearest dumping point or charging station;

[0036] S13: If there are other unmanned ships in the adjacent partition of the unmanned ship, the unfinished salvage task will be dynamically assigned to the unmanned ship with the lightest load in the adjacent partition, and the A* algorithm will be reused to perform full coverage path planning on the merged partition. If there are no other unmanned ships in the adjacent partition of the unmanned ship, the salvage will be terminated.

[0037] As a further improvement of the present invention, in S3, the water area is divided into multiple sub-partitions according to the distribution of obstacles, and the unmanned boats correspond one-to-one to the sub-partitions. By considering the shape of the water area and the distribution of obstacles, the sub-partitions are divided into different local partition shapes. Through partitioning, distributed task collaboration is achieved. At the same time, considering boundary effect problems such as limited flexibility of unmanned boats, difficulty in salvaging boundary areas, and the possibility of suspended objects moving from the current partition to another, moderate overlap between adjacent partitions is allowed. By designing overlapping areas, the coverage of boundary areas is increased to avoid omissions caused by boundary problems.

[0038] As a further improvement of the present invention, in S4, the A* algorithm is used to plan the full coverage path of each divided sub-partition, and the optimal solution of the full coverage path is quickly found by the A* algorithm to ensure that all suspended objects can be detected, tracked and salvaged, while reducing the navigation time and distance, thereby extending the endurance of the unmanned ship.

[0039] As a further improvement of the present invention, in S5, when the unmanned ship cluster cruises along the planned path in each sub-division, the feature matching and data association algorithm in the SLAM technology is used to process the point cloud data obtained by the SLAM sensor module in the perception and detection module of the unmanned ship to identify obstacles, and the movement speed of the obstacle is obtained according to the point cloud difference. According to the movement speed and position information of the obstacle, the local map and the global map containing the static obstacle position are updated. If the obstacle is a static object, the shortest safe distance between the unmanned ship and the obstacle is calculated according to the current position of the unmanned ship, and the optimal angle for bypassing the obstacle is determined. The DWA algorithm is used to plan the obstacle avoidance path for the obstacles identified by the SLAM technology; if the obstacle is a dynamic object, the obstacle position and speed information are updated in real time to ensure the safe operation of the unmanned ship.

[0040] As a further improvement of the present invention, in S6, image information of the water surface around the unmanned boat is collected in real time by an image sensor, and a deep learning algorithm is used to monitor and classify images extracted from collected photos and video frames. The position of the suspended objects on the water surface is marked with a detection bounding box, providing key information for subsequent target tracking, size measurement and robot grasping.

[0041] As a further improvement of the present invention, in S7, based on the bounding box and category obtained by the deep learning method, combined with the target state of the previous frame, the Kalman filter is used to predict the position of the current frame, and the Hungarian algorithm is used, combined with the appearance features and IOU extracted by deep learning, to associate the detection result of the current frame with the tracking target of the previous frame. Based on the nonlinear motion model, the least squares method is used to fit the motion trajectory of the target, and the equivalent moving speed and position offset are calculated. The Kalman filter is used to update the state of the target and predict its future trajectory.

[0042] As a further improvement of the present invention, in S8, the DeepSORT algorithm is used to track suspended objects on the water surface at multiple moments to obtain location information at multiple historical moments. Based on external factors such as wind direction, wind speed and water flow, a robust and accurate suspended object motion prediction system is constructed by combining image processing, nonlinear motion models and adaptive filtering algorithms. By assigning custom weights to each suspended object category, a weighted sum method is used to determine the salvage priority of different suspended objects, thereby giving priority to more important or urgent salvage tasks.

[0043] As a further improvement of the present invention, in S10, a robotic arm and collection net are used to classify and recover suspended solids. A PID control algorithm is used to precisely control the robotic arm, ensuring stability and accuracy when recovering and placing suspended solids into a recovery bin, thereby reducing operational errors. A fixed collection net is installed at the end of the robotic arm to salvage the suspended solids. The classified recovery bins include: a bin for recyclable suspended solids such as plastic bottles, metal beverage cans, plastic bags, and recycling bags; a bin for organic matter such as leaves, algae, and aquatic plants; and a bin for harmful suspended solids such as carcasses of aquatic animals.

[0044] As a further improvement of the present invention, in S11, after the salvage operation, the system will judge the full load status of the current classification recovery bin and the power level of the unmanned boat in real time based on the sensor data. When the classification recovery bin is not full and the power is sufficient, the operation will continue. If the current sub-partition has not been completely traversed, the A* algorithm will be used to plan a full coverage path for the untraversed area of ​​the current sub-partition based on the current position of the unmanned boat, obstacle information and the cruise path. If the current sub-partition has been fully traversed, the A* algorithm will be used again to plan a full coverage path for the current sub-partition based on the current position of the unmanned boat and obstacle information.

[0045] As a further improvement of the present invention, in S13, if there are other unmanned ships in the adjacent partition of the unmanned ship, the unfinished salvage task will be dynamically assigned to the unmanned ship with the lightest load in the adjacent partition based on the load balancing principle, and the A* algorithm will be reused to perform full coverage path planning on the merged partition. If there are no other unmanned ships in the adjacent partition of the unmanned ship, the salvage will be terminated.

[0046] Compared with the existing technology, the beneficial effects of the present invention are as follows: the method of the present invention can construct a three-dimensional map by itself, has the ability to perceive the environment, and realizes autonomous navigation and obstacle avoidance in complex water environments through the A* algorithm and the DWA algorithm. Through regional division, division of labor and cooperation are achieved, and overall work efficiency is improved. In response to the problem of partition boundary effect, a partition overlapping mechanism is designed to achieve an improvement in overall work efficiency. At the same time, the use of deep learning models and multi-target tracking DeepSORT algorithms enables unmanned ships to track multiple suspended objects in real time within sub-partitions and predict their future positions. It can also perform salvage operations based on the priority of suspended objects, thereby improving salvage efficiency. Through the task coordination mechanism, the automated classification and salvage of suspended objects by unmanned ship clusters in large areas of water is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of the unmanned boat cluster surface partitioning collaborative classification salvage method based on deep learning of the present invention;

[0048] Figure 2 This is the architecture diagram of the unmanned ship system;

[0049] Figure 3 This is a schematic diagram of the partition compensation of the unmanned boat cluster water surface partition collaborative classification salvage method based on deep learning of the present invention;

[0050] Figure 3 In the figure, 1 is the partition boundary line and 2 is the sub-compensation area. DETAILED DESCRIPTION

[0051] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0052] See also Figure 1 The embodiment of the present invention provides a deep learning-based unmanned vessel cluster surface partitioning collaborative classification salvage method, which specifically includes the following steps:

[0053] S1: Deploy the deep learning model trained and verified with the self-built dataset to the unmanned vessel cluster edge computing platform;

[0054] Specifically, in order for the unmanned vessel to realize the suspended matter identification and classification function, the following steps need to be implemented before step S1:

[0055] Using cameras, drones, surface robots, and unmanned boats, we captured suspended debris in various water types (including lakes, rivers, and oceans) across different seasons, weather conditions, time periods, and lighting conditions to construct a high-quality, diverse dataset of suspended debris on water surfaces. Dataset preprocessing, including denoising, brightness correction, and local adaptive histogram equalization, as well as data augmentation techniques (rotation, scaling, and flipping), increased dataset diversity and enriched the training set, thereby improving model robustness. Deep learning models such as YOLO, CenterNet, and EfficientDet were trained on the training set. The LabelImg annotation tool was used to annotate dataset images by name and select their locations through a visual interface, generating a VOC-formatted XML file. Annotation types included plastic bottles, plastic bags, glass bottles, metal beverage cans, and aquatic plants and animals. The dataset was divided into a training set (train), a validation set (valid), and a test set (test). Model performance was evaluated on the validation set, and parameter adjustments were made. Failure to achieve the expected validation accuracy may indicate overfitting, leading to reduced generalization. Optimization can be performed by increasing the number of training rounds and the number of data sets until the expected verification accuracy is achieved. If the expected verification accuracy is achieved, the model is deployed to the edge computing platform of the unmanned vessel processing and analysis module to achieve real-time detection of suspended solids on the water surface.

[0056] Denoising uses Gaussian filtering cv2.GaussianBlur to restore the details of the original image.

[0057] Brightness correction uses brightness correction based on Retinex theory to eliminate the impact of uneven lighting.

[0058] S(x,y)=R(x,y)·L(x,y)

[0059] Where:

[0060] S(x,y) is the original image captured;

[0061] R(x,y) is the reflected image, that is, the image we want to obtain;

[0062] L(x,y) is the brightness image, which is the part that causes the image brightness to be uneven.

[0063] log(S(x,y))=log(R(x,y))+log(L(x,y))

[0064] log(R(x,y))=log(S(x,y))-log(L(x,y))

[0065] =log(S(x,y))-log(S(x,y)*G(x,y))

[0066] Where:

[0067] * represents convolution;

[0068] G(x,y) is the Gaussian kernel;

[0069] Thus, the enhanced R(x,y) is obtained.

[0070] The dataset is preprocessed using rotation cv2.rotate, scaling cv2.resize, and local adaptive histogram equalization cv2.createCLAHE.

[0071] S2: Use SLAM technology to build a data system that combines SLAM modules, sensor modules, and positioning modules. Based on the extracted obstacle movement speed, it determines whether the obstacle is a static or dynamic object, and then constructs a global three-dimensional map containing the location of static obstacles, providing spatial information for subsequent path planning and water area division.

[0072] S=S free +S static +S dynamic

[0073] Where:

[0074] S is the surface area of ​​the water area;

[0075] S free It is a cruiseable area;

[0076] S static It is a static obstacle area;

[0077] S dynamic It is a dynamic obstacle area.

[0078] S3: Divide the water area into multiple sub-areas based on the distribution of obstacles, and the unmanned boats correspond one to one to the sub-areas. By considering the shape of the water area and the distribution of obstacles, the sub-areas can be of different local partition shapes (such as rectangles, hexagons, etc.) to improve the partitioning efficiency. Through partitioning, the efficiency of path planning and the salvage operation efficiency of unmanned boats can be improved, the probability of task conflicts can be reduced, and distributed task collaboration can be achieved. At the same time, taking into account boundary effect issues such as the limited flexibility of unmanned boats, the difficulty of salvaging boundary areas, and the possibility that suspended objects may move from the current partition to another partition, moderate overlap is allowed between adjacent partitions. By designing overlapping areas, the coverage of boundary areas is increased to avoid omissions caused by boundary problems, thereby improving the overall salvage effect. Figure 3 As shown in the partition compensation diagram, a partition overlap compensation mechanism is designed. Specifically, each partition extends an overlap area of ​​a certain width to the adjacent partition. The overlap area is obtained by offsetting the partition boundary by a fixed distance, which is the effective sensing distance of the sensor.

[0079] To address potential conflicts within overlapping areas, SLAM sensors and the position and attitude information of the unmanned vessels provide the basic data for conflict detection. The system monitors the distance between unmanned vessels and whether they are sharing the same target in real time. If proximity or mission conflict is detected, a conflict resolution mechanism is triggered. By establishing a virtual force field in the overlapping area, the unmanned vessels are treated as sources of repulsion and guided away from the target, effectively avoiding collisions and improving system safety, reliability, and operational efficiency.

[0080] S4: Utilize the A* algorithm to plan a full coverage path for each sub-area. This algorithm quickly finds the optimal solution for a full coverage path, ensuring that all suspended objects can be detected, tracked, and salvaged while reducing navigation time and distance, thereby extending the drone's endurance.

[0081] S5: As the unmanned vessel cluster cruises along the planned paths in each sub-area, it uses the feature matching and data association algorithms in SLAM technology to process the point cloud data acquired by the SLAM sensor module in the perception and detection module of the unmanned vessel to identify obstacles. Based on the point cloud difference, the movement speed of the obstacle is obtained, and the local map and global map containing the static obstacle position are updated based on the obstacle movement speed and position information. If the obstacle is a static object, the shortest safe distance between the unmanned vessel and the obstacle is calculated based on the current position of the unmanned vessel, and the optimal angle for bypassing the obstacle is determined. The DWA algorithm is used to plan an obstacle avoidance path for the obstacles identified by SLAM technology. If the obstacle is a dynamic object, the obstacle position and speed information are updated in real time to ensure the safe operation of the unmanned vessel.

[0082] According to the position of the unmanned ship and the obstacle, the Euclidean distance formula is used Calculate the distance between the unmanned vessel and obstacles.

[0083] Where:

[0084] d(t) is the distance between the unmanned ship and the obstacle at time t;

[0085] (x i (t),y i (t)) is the position coordinate of the unmanned ship at time t;

[0086] (x j (t),y j (t)) is the position coordinate of the obstacle at time t;

[0087]

[0088] Where:

[0089] d safe For safe distance;

[0090] d static is the minimum safe distance for static obstacles;

[0091] d dynamic It is the minimum safe distance for dynamic obstacles.

[0092] The unmanned boat can move forward, backward and turn by means of the directional propeller and the left and right drive motors in the power and drive module. By driving the unmanned boat to cruise at a constant speed along the planned route, real-time perception of the water surface environment can be achieved.

[0093] S6: The image sensor collects real-time image information of the water surface around the unmanned boat. The deep learning algorithm is used to monitor and classify suspended debris on the water surface from the captured photos and video frames. The location of suspended debris on the water surface is marked with a detection bounding box, providing key information for subsequent target tracking, size measurement, and robot grasping.

[0094] S7: Based on the bounding box and category obtained by deep learning methods, combined with the target state (position, velocity, etc.) in the previous frame, a Kalman filter is used to predict the position of the current frame. Using the Hungarian algorithm, combined with the appearance features and IOU extracted by deep learning, the detection results of the current frame are associated with the tracked target in the previous frame. Based on a nonlinear motion model, the least squares method is used to fit the target's motion trajectory and calculate the equivalent movement velocity and position offset. The Kalman filter is used to update the target's state and predict its future trajectory.

[0095] S8: Utilizing the DeepSORT algorithm, we track suspended debris on the water surface over multiple time periods, acquiring location information at multiple historical moments. Based on external factors such as wind direction, wind speed, and water current, we combine image processing, nonlinear motion models, and adaptive filtering algorithms to construct a robust and highly accurate suspended debris motion prediction system. By assigning custom weights to each suspended debris category and using a weighted summation method to prioritize the salvage of different suspended debris, we prioritize more important or urgent salvage tasks, such as the salvage of dead aquatic animals and hazardous suspended debris.

[0096] S9: Use the DWA algorithm to plan the shortest path to the predicted location of the priority suspended object. By establishing a virtual force field in the overlapping area, potential conflicts are avoided, improving safety, reliability, and operational efficiency.

[0097] S10: The recovery robot arm and collection net in the recovery and processing module are used to classify and recycle suspended solids. The PID control algorithm is used to precisely control the robot arm to ensure stability and accuracy when recovering and placing suspended solids in the recovery bin, thereby reducing operational errors. The end of the robot arm is equipped with a fixed collection net to salvage the suspended solids through the collection object. By precisely controlling the robot arm, different types of suspended solids can be classified and accurately placed. The collection net is designed with different materials and mesh sizes according to the type and size of the suspended solids to achieve efficient and accurate recovery results. The classified recycling bins include: bins for recyclable suspended solids such as plastic bottles, metal beverage cans, and plastic bag recycling bags; bins for aquatic plant organic matter such as leaves and algae; and bins for harmful suspended solids such as aquatic animal carcasses.

[0098] S11: After the salvage operation, the system will determine the fullness of the current classification recovery bin and the battery level of the unmanned vessel in real time based on sensor data. If the classification recovery bin is not full and the battery is sufficient, the operation will continue. If the current sub-region has not been fully traversed, the A* algorithm will be used to plan a full coverage path for the untraversed area of ​​the current sub-region based on the current location of the unmanned vessel, obstacle information, and the cruise path. If the current sub-region has been fully traversed, the A* algorithm will be used again to plan a full coverage path for the current sub-region based on the current location of the unmanned vessel and obstacle information.

[0099] S12: If the classified recycling bin is full or the battery level is below a preset threshold, the A* algorithm is used to plan a route to the nearest dumping point or charging station. The full load status is determined by the pressure sensor and infrared sensor of the sensing and detection module.

[0100] S13: If there are other unmanned ships in the adjacent partition of the unmanned ship, based on the load balancing principle, the unfinished salvage task will be dynamically assigned to the unmanned ship with the lightest load in the adjacent partition, and the A* algorithm will be reused to perform full coverage path planning on the merged partition. If there are no other unmanned ships in the adjacent partition of the unmanned ship, the salvage will be terminated.

[0101] See also Figure 2 , a deep learning-based unmanned vessel cluster surface partition collaborative classification salvage method, including communication and positioning module, perception and detection module, processing and analysis module, power and drive module, recovery and processing module.

[0102] The communication and positioning modules enable efficient collaboration and precise positioning of the unmanned vessel swarm. The wireless communication module transmits data and coordinates operations with other unmanned vessels to achieve swarm collaboration. The positioning module acquires the current latitude and longitude in real time to determine the unmanned vessel's position and posture within the environment. This is a prerequisite for the unmanned vessel to perform path planning, obstacle avoidance, and salvage missions.

[0103] The perception and detection module is used to achieve environmental perception and status monitoring of the unmanned vessel cluster. Among them: Image sensors are used to search for images of suspended objects on the water surface. SLAM sensors are used to build environmental maps and locate objects in real time, helping unmanned vessels to navigate autonomously in complex waters. Water quality sensors monitor the chemical and physical properties of water bodies and assess water quality. Wind direction and speed sensors are used to measure wind direction and speed. Ultrasonic Doppler flowmeters are used to measure water flow velocity. Pressure sensors are installed at the bottom of the recovery bin to measure internal pressure changes and determine the full load status. Infrared sensors are installed at the top of the recovery bin to detect the height of the filling material and determine whether it is fully loaded.

[0104] The processing and analysis module is used to implement real-time suspended debris identification, tracking, and unmanned vessel control. The edge computing platform is responsible for real-time suspended debris identification, tracking, and unmanned vessel control. Through efficient data processing and analysis, the edge computing platform not only reduces latency and improves response speed, but also supports decision-making for regional division and route generation. Data such as the latitude, longitude, and speed of the unmanned vessel cluster is centrally stored on the central server. Through a visual interface, the operating status of the unmanned vessel cluster can be monitored in real time, providing an overview of the overall operational status. Unified control commands are issued through the central server, enabling unified scheduling and control of the entire unmanned vessel cluster.

[0105] The power and drive module implements the propulsion system and motion control for the unmanned vessel swarm. The directional propellers control the vessel's steering, achieving precise course adjustment by adjusting the propeller angle and speed. The drive motors deliver power to the vessel, controlling the motor's start / stop and speed to meet varying sailing conditions. The power module provides a stable power supply to the power system.

[0106] The recovery and processing module is used to salvage, recover, and store suspended solids on the water surface. This includes a classified recovery bin for recyclable suspended solids such as plastic bottles, metal beverage cans, and plastic bag recycling bags; a bin for aquatic plant matter such as leaves and algae; and a bin for harmful suspended solids such as carcasses of aquatic animals. A fixed collection net is installed at the end of the robotic arm to salvage the suspended solids. Precise control of the robotic arm enables the classified capture and precise delivery of different types of suspended solids. The collection net is designed with different materials and mesh sizes based on the type and size of the suspended solids, ensuring efficient and accurate recovery.

[0107] The above description is merely a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent variation based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A deep learning-based unmanned boat cluster surface partitioning collaborative classification salvage method, characterized by: include; S1: Deploy the deep learning model trained and verified by the self-built dataset to the unmanned ship cluster edge computing platform; Take images of suspended matter on the water surface in different waters, weather conditions, and time conditions, and use data enhancement technology to expand the data set; use a visual operation interface to annotate the data set images, and after preprocessing, select a deep learning model suitable for the task for training, and evaluate the model performance through indicators such as mAP. When the model reaches the expected accuracy, deploy it on the unmanned boat edge computing platform to achieve real-time detection and classification of suspended matter on the water surface; S2: Use SLAM technology to build a three-dimensional map of the water area and store it as a global map to achieve environmental perception; Use SLAM technology to build a data system that combines SLAM modules, sensor modules, and positioning modules. According to the movement speed of the extracted obstacles, it is determined whether the obstacles are static or dynamic objects, and then a global three-dimensional map containing the location of static obstacles is constructed to provide spatial information for subsequent path planning and water area division. S3: Divide the water area into regions according to the global map and the complexity of the water area, and compensate for the overlap of the internal boundaries of the partitions; S4: A* algorithm performs full coverage path planning for each sub-partition; S5: When the unmanned ship cluster cruises along the planned path in each sub-area, it uses the SLAM sensor data to update the map and uses the DWA algorithm to plan the obstacle avoidance path for the obstacles identified by the SLAM technology; S6: When cruising, the image sensor is used to collect image information of the water surface around the unmanned ship in real time, and the image is processed through a deep learning algorithm to monitor and classify suspended matter on the water surface; S7: Use the DeepSORT algorithm to track the identified suspended objects, and calculate the equivalent moving speed and position offset of the suspended objects based on the position coordinate information at multiple historical moments through multi-sensor data fusion; S8: predicting the possible position of the suspended object at a future time according to the equivalent moving speed and the position offset, and driving the unmanned boat to the predicted position of the priority suspended object according to the set salvage priority of the suspended object; S9: using the DWA algorithm to plan the shortest path to the predicted location of the priority suspended object; The DWA algorithm is used to plan the shortest path to the predicted location of the priority suspended object, and a virtual force field is established in the overlapping area to avoid possible conflicts and improve safety, reliability and work efficiency. S10: Salvage operations; S11: After the salvage operation, the operating status is determined based on the sensor data. When the classification recovery bin is not full and the power is sufficient, the operation continues. If the current sub-partition has not been completely traversed, the A* algorithm is used to plan a full coverage path for the untraversed area of ​​the current sub-partition according to the current location of the unmanned ship, obstacle information and the cruise path. If the current sub-partition has been completely traversed, the A* algorithm is used again to plan a full coverage path for the current sub-partition according to the current location of the unmanned ship and obstacle information; S12: If the classified recycling bin is full or the power level is below a preset threshold, the A* algorithm is used to plan a path to the nearest dumping point or charging station; If the classified recycling bin is full or the battery level is below a preset threshold, the A* algorithm is used to plan a route to the nearest dumping point or charging station; S13: If there are other unmanned ships in the adjacent partition of the unmanned ship, the unfinished salvage task will be dynamically assigned to the unmanned ship with the lightest load in the adjacent partition, and the A* algorithm will be reused to perform full coverage path planning on the merged partition. If there are no other unmanned ships in the adjacent partition of the unmanned ship, the salvage will be terminated.

2. The deep learning-based unmanned boat cluster surface partition collaborative classification salvage method according to claim 1 is characterized by: In S3, the water area is divided into multiple sub-areas according to the distribution of obstacles, and the unmanned boats correspond to the sub-areas one by one. By considering the shape of the water area and the distribution of obstacles, the sub-areas can be of different shapes. Distributed task collaboration can be achieved through partitioning. At the same time, considering the boundary effect problems such as the limited flexibility of unmanned boats, the difficulty of salvaging boundary areas, and the possibility of suspended objects moving from the current partition to another partition, moderate overlap between adjacent partitions is allowed. By designing the overlapping area, the coverage of the boundary area is increased to avoid omissions caused by boundary problems. The overlapping area is obtained by offsetting the contour of the partition boundary by a fixed distance, and the offset distance is the effective perception distance of the sensor.

3. The deep learning-based unmanned boat cluster surface partition collaborative classification salvage method according to claim 1 is characterized by: In S4, the A* algorithm is used to plan full coverage paths for each sub-division after division. The A* algorithm is used to quickly find the optimal solution for the full coverage path, ensuring that all suspended objects can be detected, tracked and salvaged while reducing navigation time and distance, thereby extending the endurance of the unmanned ship.

4. The deep learning-based unmanned boat cluster surface partition collaborative classification salvage method according to claim 1 is characterized by: In S5, when the unmanned ship cluster cruises along the planned path in each sub-division, the feature matching and data association algorithms in the SLAM technology are used to process the point cloud data obtained by the SLAM sensor module in the perception and detection module of the unmanned ship to identify obstacles, obtain the movement speed of the obstacle based on the point cloud difference, and update the local map and global map containing the static obstacle position based on the obstacle movement speed and position information. If the obstacle is a static object, the shortest safe distance between the unmanned ship and the obstacle is calculated according to the current position of the unmanned ship, and the optimal angle to bypass the obstacle is determined. The DWA algorithm is used to plan the obstacle avoidance path for the obstacles identified by the SLAM technology; if the obstacle is a dynamic object, the obstacle position and speed information are updated in real time to ensure the safe operation of the unmanned ship.

5. The deep learning-based unmanned boat cluster surface partition collaborative classification salvage method according to claim 1 is characterized by: In S6, image sensors are used to collect image information of the water surface around the unmanned boat in real time. Deep learning algorithms are used to monitor and classify images extracted from collected photos and video frames, and the positions of suspended objects on the water surface are marked with detection bounding boxes, providing key information for subsequent target tracking, size measurement, and robot grasping.

6. The deep learning-based unmanned boat cluster surface partition collaborative classification salvage method according to claim 1 is characterized by: In S7, based on the bounding box and category obtained by the deep learning method and the target state of the previous frame, the Kalman filter is used to predict the position of the current frame. The Hungarian algorithm is used in combination with the appearance features and IOU extracted by deep learning to associate the detection result of the current frame with the tracked target of the previous frame. Based on the nonlinear motion model, the least squares method is used to fit the motion trajectory of the target, and the equivalent moving speed and position offset are calculated. The Kalman filter is used to update the state of the target and predict its future trajectory.

7. The deep learning-based unmanned boat cluster surface partition collaborative classification salvage method according to claim 1 is characterized by: In S8, the DeepSORT algorithm is used to track suspended matter on the water surface at multiple times to obtain location information at multiple historical times. Based on external factors such as wind direction, wind speed and water flow, a robust and accurate suspended matter motion prediction system is constructed by combining image processing, nonlinear motion model and adaptive filtering algorithm. By assigning custom weights to each suspended matter category, the weighted sum method is used to determine the salvage priority of different suspended matter, thereby giving priority to more important or urgent salvage tasks.

8. The deep learning-based unmanned boat cluster surface partition collaborative classification salvage method according to claim 1 is characterized by: In S10, a robotic arm and a collection net are used to classify and recycle suspended solids. The PID control algorithm is used to precisely control the robotic arm to ensure the stability and accuracy of salvaging and recycling suspended solids and placing them in the recycling bin, reducing operational errors. A fixed collection net is equipped at the end of the robotic arm to salvage suspended solids through the collection bin. The classified recycling bins include: a recycling bin for recyclable suspended solids such as plastic bottles, metal beverage cans, plastic bags, and recycling bags; a bin for organic matter such as leaves, algae, and aquatic plants; and a bin for harmful suspended solids such as carcasses of aquatic animals.

9. The deep learning-based unmanned boat cluster surface partition collaborative classification salvage method according to claim 1 is characterized by: In S11, after the salvage operation, the system will judge the full load status of the current classification and recovery bin and the power level of the unmanned boat in real time based on the sensor data. When the classification and recovery bin is not full and the power is sufficient, the operation will continue. If the current sub-partition has not been completely traversed, the A* algorithm will be used to plan a full coverage path for the untraversed area of ​​the current sub-partition based on the current position of the unmanned boat, obstacle information and the cruise path. If the current sub-partition has been fully traversed, the A* algorithm will be used again to plan a full coverage path for the current sub-partition based on the current position of the unmanned boat and obstacle information.

10. The deep learning-based unmanned boat cluster surface partition collaborative classification salvage method according to claim 1, characterized in that: In S13, if there are other unmanned ships in the adjacent partition of the unmanned ship, based on the load balancing principle, the unfinished salvage tasks are dynamically allocated to the unmanned ship with the lightest load in the adjacent partition, and the A* algorithm is reused for full coverage path planning for the merged partition. If there are no other unmanned ships in the adjacent partition of the unmanned ship, the salvage is terminated.

Images