Method for cooperatively salvaging sea floating garbage by unmanned ship based on reinforcement learning algorithm
By applying reinforcement learning algorithms to drive unmanned ship collaborative operations in seafloor garbage salvage, and combining with the LSTM model to predict the garbage diffusion path, the problems of low efficiency and high cost of seafloor garbage salvage in the existing technology are solved, and efficient and low-cost seafloor garbage cleaning is achieved.
Patent Information
- Application Number
- CN202510527978.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-25
- Publication Date
- 2025-06-24
AI Technical Summary
The existing seafloor garbage salvage methods rely on manual search, limited coverage, slow response speed and high labor costs. Especially under the narrow river channels and open sea surface transition areas and dynamic aggregation effects in the sea estuary area, the salvage efficiency is low and the cost is high.
The unmanned ship collaborative salvage method based on reinforcement learning algorithm is adopted, and the LSTM model integrating hydrological and meteorological data predicts the pollution diffusion path, automatically generates a pollution cleaning solution, and uses a multi-agent reinforcement learning algorithm to drive multiple unmanned ships to dynamically form a team to complete the containment and salvage of sea drift garbage.
The timely discovery and cleaning of seafloor garbage has been achieved, the efficiency of seafloor garbage has been improved, the labor cost of salvage operations has been reduced, and the allocation of salvage resources has been optimized.
Smart Images
Figure CN120191476A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of marine debris treatment, and particularly relates to a method for collaborative salvage of marine debris by unmanned ships based on a reinforcement learning algorithm. Background Art
[0002] Marine debris mainly refers to the debris floating on the sea surface, such as plastic bags, floating wooden blocks, buoys, and plastic bottles. They not only cause visual pollution but also cause water pollution, deteriorating water quality. The methods for removing these debris can be classified by region into coastal and beach collection methods and marine vessel collection methods, and the marine debris is salvaged by manual vessels, fixed salvage equipment, or simple remotely controlled unmanned ship equipment. However, these salvage methods have problems such as relying on manual search, limited coverage, slow response speed, and high labor costs. At the same time, when dealing with marine debris in the estuary area, greater difficulties will be encountered. For example, traditional salvage vessels are difficult to handle the salvage operations in the transition area between narrow channels and open seas, and the confluence of fresh and salt water causes water flow disorders and irregular distribution of debris. In addition, the marine debris in the estuary area has a dynamic aggregation effect. Affected by tides, ocean currents, and monsoons, the marine debris forms a temporary accumulation area in the estuary. The manual vessels and unmanned ship equipment for salvage cannot effectively predict the formation of the temporary accumulation area and conduct timely salvage treatment, resulting in a large amount of manpower and material resources being consumed for search and cleaning, seriously affecting the work efficiency of marine debris salvage. Summary of the Invention
[0003] In view of the above deficiencies, the present invention discloses a method for collaborative salvage of marine debris by unmanned ships based on a reinforcement learning algorithm, which integrates an LSTM model of hydrological and meteorological data to predict the pollution diffusion path, automatically generates a pollution cleaning plan, and drives multiple unmanned ships to form a dynamic team to complete operations such as the enclosure and salvage of marine debris based on a multi-agent reinforcement learning algorithm, realizing the timely discovery and cleaning of marine debris, and also improving the salvage and cleaning efficiency of marine debris and reducing the labor cost of salvage operations.
[0004] The present invention is implemented by the following technical solutions: A method for collaborative salvage of marine debris by unmanned ships based on a reinforcement learning algorithm, which includes the following steps: (1) Set a monitoring ship and more than three groups of salvage fleets in the monitored sea area. Each group of salvage fleets is provided with a mother ship and several unmanned salvage operation ships; (2) The monitoring ship is equipped with a hydrological data acquisition system A, a meteorological data acquisition system A, a remote sensing data processing system A, a garbage diffusion prediction system, and a salvage fleet route planning system. The garbage diffusion prediction system analyzes and processes the data information in the hydrological data acquisition system A, the meteorological data acquisition system A, and the remote sensing data processing system A to predict the aggregation areas and flow routes of marine floating garbage in the monitored sea area. The salvage fleet route planning system formulates the work processes of each salvage fleet according to the predicted aggregation areas and flow routes of marine floating garbage in the monitored sea area, and distributes the work processes to each salvage fleet. The work processes include work areas and work times. (3) The mother ship is equipped with a garbage storage bin for storing salvaged garbage, a mother ship control system, a hydrological data acquisition system B, a meteorological data acquisition system B, and an unmanned aerial vehicle cruising system. The mother ship control system includes a mother ship data transceiver module, a mother ship data parsing module, a mother ship image recognition module, and an unmanned ship route planning module. The mother ship data parsing module receives the work processes sent from the monitoring ship for parsing, determines the work areas and work times, and guides the salvage fleet to reach the work areas. Then, it collects the sea surface images of the area through the unmanned aerial vehicle cruising system. The mother ship image recognition module analyzes and processes the sea surface images to determine the garbage position information in the work areas. Next, the unmanned ship route planning module formulates the cruising processes for each unmanned salvage operation ship according to the garbage position information. The cruising processes include cruising routes, times, and frequencies, and then are sent to the unmanned salvage operation ships through the mother ship data transceiver module. For the hydrological data acquisition system B, the meteorological data acquisition system B, and the unmanned aerial vehicle cruising system, the hydrological data acquisition system B and the meteorological data acquisition system B collect the hydrological data and meteorological data of the work area where the mother ship is located and send them to the monitoring ship through the mother ship control system. At the same time, the unmanned aerial vehicle cruising system collects the sea surface images of the work area where the mother ship is located and sends them to the monitoring ship through the mother ship control system. The garbage diffusion prediction system on the monitoring ship corrects the input data according to the data collected by the hydrological data acquisition system B, the meteorological data acquisition system B, and the unmanned aerial vehicle cruising system. (4) The unmanned salvage operation ship is equipped with mechanical claws and / or electromagnetic suction cups for salvaging garbage, cameras for image acquisition, an unmanned ship control system, and a power system. The unmanned ship control system includes an unmanned ship positioning and navigation module, an unmanned ship data transceiver module, an unmanned ship image recognition module, and an unmanned ship data storage module. The unmanned ship positioning and navigation module receives and analyzes the cruise process sent by the mother ship, and then controls the unmanned salvage operation ship to cruise in the working area according to the specified route. During the cruise, image data on the cruise route is collected in real time through a camera. The unmanned ship image recognition module identifies and locates marine floating garbage based on the image data, then decelerates or pauses the cruise when approaching the marine floating garbage according to the positioning information, and controls the mechanical claw and / or electromagnetic suction cup to salvage the marine floating garbage in the diagonally front direction, and then returns to the mother ship to recycle the garbage into the garbage storage bin.
[0005] Further, in step (2), the hydrological data acquisition system A is used to collect hydrological data in real time. The hydrological data includes the flow velocity, flow direction, and salinity of the water flow; the meteorological data acquisition system A is used to collect meteorological data in real time. The meteorological data includes wind speed, wind direction, and rainfall; the remote sensing data processing system A is used to collect remote sensing data of the monitored sea area regularly.
[0006] Further, in step (2), the garbage diffusion prediction system is constructed based on a convolutional neural network (CNN) and a long short-term memory network (LSTM) to obtain a prediction model. The CNN includes a convolutional layer, a pooling layer, and a fully connected layer. The spatial features of the remote sensing data are extracted through the CNN network, and then the obtained spatial features are fused with the hydrological data and meteorological data at the corresponding time points as the input data of the LSTM to learn the long-term dependencies in the time series data and predict the garbage diffusion path in the next time window. The LTSM includes an input gate, a forget gate, a cell state, and an output gate.
[0007] Further, the hydrological data and meteorological data are preprocessed by cleaning, filling missing values, and normalizing in sequence, and then fused with the spatial features through splicing or weighted fusion.
[0008] Further, the normalization is a modality-specific normalization process based on Z-score. Since the numerical ranges of data sources such as hydrological data and meteorological data vary greatly, the modality-specific normalization process based on Z-score can be used to avoid the features being dominated after fusion and affecting the decision-making and performance of the model.
[0009] Further, in step (2), the garbage diffusion prediction system is trained using historical data in the monitored area for 3 to 5 years, and adversarial training is performed by generating extreme meteorological condition samples through DeepSeek. Selecting historical data for 3 to 5 years for training can provide sufficient training samples for the prediction model and improve the accuracy of model prediction. At the same time, using DeepSeek to generate extreme meteorological condition samples for adversarial training can improve the robustness of the model.
[0010] Further, an image data processing system for the monitoring ship is provided on the monitoring ship. The image data processing system of the monitoring ship selects the image data collected by the UAV cruise systems on each mother ship at the same time point, extracts their spatial features, and then combines them to obtain the overall spatial features of the monitored sea area. Then, the prediction model is optimized according to the overall spatial features.
[0011] The technical solution of the present invention has the following beneficial effects compared with the prior art: The present invention collects hydrological, meteorological and satellite remote sensing data in the monitoring area through a monitoring ship, constructs a prediction model based on CNN and LTSM to analyze and process the data, predicts the distribution and diffusion path of sea drift garbage in the monitoring area, and then formulates and assigns the salvage work area and working time to each salvage fleet based on the prediction results, etc. The salvage work process assigns more salvage resources to the garbage aggregation area, realizes the optimal allocation of salvage resources, and improves the work efficiency of the salvage operation.
[0012] The salvage fleet consists of a mother ship and several unmanned salvage operation ships. The salvage fleet moves to the work area according to the assigned work process for sea drift garbage salvage operations. Moreover, the mother ship of the salvage fleet will collect hydrological, meteorological and other information in the work area again and send it to the monitoring ship for data and model calibration and optimization. At the same time, the UAV cruise system is used to collect sea surface images, and then the garbage distribution is determined according to the sea surface images, and the salvage cruise route is set for the unmanned salvage operation ships. Finally, the unmanned salvage operation ships cruise according to the cruise route, and the sea drift garbage on the cruise route is identified by configuring an image recognition system and salvaged in time.
[0013] The mother ship can be equipped with several unmanned salvage operation ships, and mechanical claws or electromagnetic suction cups are provided on the unmanned salvage operation ships for different types of garbage, so that the sea drift garbage can be preliminarily classified during the salvage stage. Moreover, the mother ship can not only provide storage for the salvaged garbage, but also serve as an energy relay to supplement electric energy for the unmanned salvage operation ships to maintain the continuity of the salvage operation. Description of the Drawings
[0014] Figure 1 It is a flowchart of the method for collaborative salvage of sea drift garbage by unmanned ships based on the reinforcement learning algorithm in Embodiment 1. Detailed Embodiments
[0015] The present invention will be further described below through embodiments, but it is not intended to limit the present invention. The specific experimental conditions and methods not specified in the following embodiments are usually conventional means well-known to those skilled in the art.
[0016] Embodiment 1: A method for collaborative salvage of sea drift garbage by unmanned ships based on the reinforcement learning algorithm, which includes the following steps: (1)Set up a monitoring ship and more than three salvage fleets in the monitored sea area. Each salvage fleet consists of a mother ship and 8 unmanned salvage operation ships; (2)The monitoring ship is equipped with a hydrological data acquisition system A, a meteorological data acquisition system A, a remote sensing data processing system A, a garbage diffusion prediction system, and a salvage fleet route planning system; The garbage diffusion prediction system analyzes and processes the data information in the hydrological data acquisition system A, the meteorological data acquisition system A, and the remote sensing data processing system A to predict the aggregation area and flow route of sea drift garbage in the monitored sea area; The salvage fleet route planning system formulates the work processes of each salvage fleet according to the predicted aggregation area and flow route of sea drift garbage in the monitored sea area, and distributes the work processes to each salvage fleet. The work processes include work areas and work times; The hydrological data acquisition system A is used to collect hydrological data in real time. The hydrological data includes the flow velocity, flow direction, and salinity of the water flow; The meteorological data acquisition system A is used to collect meteorological data in real time. The meteorological data includes wind speed, wind direction, and rainfall; The remote sensing data processing system A is used to collect remote sensing data of the monitored sea area regularly; The garbage diffusion prediction system is constructed based on a convolutional neural network (CNN) and a long short-term memory network (LSTM) to obtain a prediction model. The CNN includes a convolutional layer, a pooling layer, and a fully connected layer. The spatial features of the remote sensing data are extracted through the CNN network, and then the obtained spatial features are fused with the hydrological data and meteorological data at the corresponding time points and used as the input data of the LSTM to learn the long-term dependence relationship in the time series data and predict the garbage diffusion path in the next time window. The LTSM includes an input gate, a forget gate, a cell state, and an output gate; (3)The mother ship is equipped with a garbage storage bin for storing the salvaged garbage, a mother ship control system, a hydrological data acquisition system B, a meteorological data acquisition system B, and an unmanned aerial vehicle cruise system; The mother ship control system includes a mother ship data transceiver module, a mother ship data parsing module, a mother ship image recognition module, and an unmanned ship route planning module; The mother ship data parsing module receives the work process sent from the monitoring ship for parsing, determines the work area and work time, and guides the salvage fleet to reach the work area. Then, it collects the sea surface images of the area through the unmanned aerial vehicle cruise system. The mother ship image recognition module analyzes and processes the sea surface images to determine the garbage position information in the work area. Then, the unmanned ship route planning module formulates the cruise processes for each unmanned salvage operation ship according to the garbage position information. The cruise processes include the cruise route, time, and number of times, and then sends them to the unmanned salvage operation ship through the mother ship data transceiver module; The hydrological data acquisition system B, the meteorological data acquisition system B, and the UAV cruise system. The hydrological data acquisition system B and the meteorological data acquisition system B collect hydrological data and meteorological data in the working area where the mother ship is located and send them to the monitoring ship through the mother ship control system. At the same time, the UAV cruise system collects sea surface images in the working area where the mother ship is located and sends them to the monitoring ship through the mother ship control system. The garbage diffusion prediction system on the monitoring ship corrects the input data according to the data collected by the hydrological data acquisition system B, the meteorological data acquisition system B, and the UAV cruise system; (4) Among the unmanned salvage operation ships, 6 are equipped with mechanical claws for salvaging garbage, 2 are equipped with electromagnetic suction cups for salvaging metal garbage, and each unmanned salvage operation ship is equipped with a camera for image acquisition, an unmanned ship control system, and a power system; The unmanned ship control system includes an unmanned ship positioning and navigation module, an unmanned ship data transceiver module, an unmanned ship image recognition module, and an unmanned ship data storage module; The unmanned ship positioning and navigation module receives the cruise process sent by the mother ship for parsing, and then controls the unmanned salvage operation ship to cruise in the working area according to the specified route. During the cruise, image data on the cruise route is collected in real time through the camera. The unmanned ship image recognition module identifies and locates the floating garbage according to the image data, then decelerates or pauses the cruise when approaching the floating garbage according to the positioning information, and controls the mechanical claw and / or the electromagnetic suction cup to salvage the floating garbage in the diagonally front direction, and then returns to the mother ship to recycle the garbage into the garbage storage bin.
[0017] Embodiment 2: The difference between the method for collaborative salvage of floating garbage by unmanned ships based on the reinforcement learning algorithm described in this embodiment and the method described in Embodiment 1 is only that the hydrological data and meteorological data are fused with the spatial features in a splicing or weighted fusion manner after being preprocessed by cleaning, filling missing values, and normalization in sequence, and the normalization is a modality-based normalization process based on Z-score; At the same time, in step (2), the garbage diffusion prediction system is trained using historical data in the monitoring area for 3 to 5 years, and extreme weather condition samples are generated through DeepSeek for adversarial training; The monitoring ship is equipped with a monitoring ship image data processing system. The monitoring ship image data processing system selects the image data collected by the UAV cruise system on each mother ship at the same time point, extracts their spatial features, and then combines them to obtain the overall spatial features of the monitored sea area, and then optimizes the prediction model according to the overall spatial features.
[0018] In addition, it should be understood that although this specification is described in terms of embodiments, not every embodiment contains only an independent technical solution. This narrative manner of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A method for unmanned boats to collaboratively salvage floating garbage at sea based on a reinforcement learning algorithm, characterized in that: The following steps are involved: (1) A monitoring ship and three or more salvage fleets are deployed in the monitoring sea area, and each salvage fleet is equipped with a mother ship and several unmanned salvage operation vessels; (2) The monitoring ship is equipped with a hydrological data collection system A, a meteorological data collection system A, a remote sensing data processing system A, a garbage diffusion prediction system, and a salvage fleet route planning system; the garbage diffusion prediction system analyzes and processes the data information in the hydrological data collection system A, the meteorological data collection system A, and the remote sensing data processing system A to predict the gathering area and flow route of marine floating garbage in the monitored sea area; The salvage fleet route planning system formulates the work flow of each salvage fleet according to the predicted gathering area and flow route of the marine debris in the monitored sea area, and distributes the work flow to each salvage fleet, wherein the work flow includes the work area and the work time; (3) The mother ship is equipped with a garbage storage bin for storing salvaged garbage, a mother ship control system, a hydrological data collection system B, a meteorological data collection system B, and a drone cruise system; The mother ship control system includes a mother ship data transceiver module, a mother ship data analysis module, a mother ship image recognition module, and an unmanned ship route formulation module; the mother ship data analysis module receives and analyzes the work flow sent by the monitoring ship, determines the working area and working time, and guides the salvage fleet to the working area, and then collects the sea surface image of the area through the unmanned aerial vehicle cruise system, and the mother ship image recognition module analyzes and processes the sea surface image to determine the garbage location information in the working area, and then the unmanned ship route formulation module formulates a cruise process for each unmanned salvage operation ship according to the garbage location information, and the cruise process includes the cruise route, time, and number of times, and then sends it to the unmanned salvage operation ship through the mother ship data transceiver module; The hydrological data collection system B, the meteorological data collection system B, and the drone cruise system, the hydrological data collection system B and the meteorological data collection system B collect the hydrological data and meteorological data of the working area of the mother ship and send them to the monitoring ship through the mother ship control system, while the drone cruise system collects the sea surface image of the working area of the mother ship and sends it to the monitoring ship through the mother ship control system, and the garbage diffusion prediction system on the monitoring ship corrects the input data according to the data collected by the hydrological data collection system B, the meteorological data collection system B and the drone cruise system; (4) The unmanned salvage operation vessel is equipped with a mechanical claw and / or electromagnetic suction cup for salvaging garbage, a camera for image acquisition, an unmanned ship control system, and a power system; the unmanned ship control system includes an unmanned ship positioning and navigation module, an unmanned ship data transceiver module, an unmanned ship image recognition module, and an unmanned ship data storage module; The unmanned ship positioning and navigation module receives the cruising process sent from the mother ship and analyzes it, and then controls the unmanned salvage operation ship to cruise in the working area according to the specified route. During the cruising process, the camera collects image data on the cruising route in real time. The unmanned ship image recognition module identifies and locates the floating garbage at sea based on the image data, and then slows down or pauses the cruising when approaching the floating garbage at sea based on the positioning information, and controls the mechanical claw and / or electromagnetic suction cup to salvage the floating garbage at the front side, and then returns to the mother ship to recycle the garbage into the garbage storage bin.
2. The method for unmanned boat collaborative salvaging of floating garbage based on reinforcement learning algorithm according to claim 1 is characterized in that: In step (2), the hydrological data acquisition system A is used to collect hydrological data in real time, and the hydrological data includes the flow velocity, flow direction, and salinity of the water flow; the meteorological data acquisition system A is used to collect meteorological data in real time, and the meteorological data includes wind speed, wind direction, and rainfall; the remote sensing data processing system A is used to regularly collect remote sensing data of the monitored sea area.
3. The method for unmanned boat collaborative salvaging of floating garbage based on reinforcement learning algorithm according to claim 1 is characterized in that: In step (2), the garbage spread prediction system is constructed based on a convolutional neural network and a long short-term memory network to obtain a prediction model. The CNN includes a convolutional layer, a pooling layer and a fully connected layer. The spatial features of the remote sensing data are extracted through the CNN network, and then the obtained spatial features are fused with the hydrological data and meteorological data at the corresponding time point as the input data of the LSTM, and the long-term dependencies in the time series data are learned to predict the garbage spread path of the next time window. The LTSM includes an input gate, a forget gate, a cell state and an output gate.
4. The method for unmanned boat collaborative salvaging of floating garbage based on reinforcement learning algorithm according to claim 1 is characterized in that: The hydrological data and meteorological data are sequentially cleaned, pre-processed by filling missing values and normalization, and then fused with the spatial features by splicing or weighted fusion.
5. The method for unmanned boat collaborative salvaging of floating garbage based on reinforcement learning algorithm according to claim 4 is characterized in that: The normalization is a sub-modal normalization process based on Z-score.
6. The method for unmanned boat collaborative salvaging of floating garbage based on reinforcement learning algorithm according to claim 3 is characterized by: In step (2), the garbage spread prediction system is trained using historical data from the monitoring area within 3 to 5 years, and extreme weather condition samples are generated through DeepSeek for adversarial training.
7. The method for unmanned boat collaborative salvaging of floating garbage based on reinforcement learning algorithm according to claim 3 is characterized by: The monitoring ship is provided with a monitoring ship image data processing system, which selects image data collected by the drone cruise systems on each mother ship at the same time point and extracts their spatial features and then combines them to obtain the overall spatial features of the monitored sea area, and then optimizes the prediction model according to the overall spatial features.