Multi-unmanned ship cooperative task allocation algorithm based on deep learning optimization
By combining deep learning and the CBAA algorithm, and utilizing the DQN network model to automatically mine the inherent correlations in task allocation data, the shortcomings of manual parameter setting in traditional methods are solved, and the allocation of multi-unmanned vessel collaborative patrol tasks is optimized, thereby improving the robustness and efficiency of the system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA STATE SHIPBUILDING CORP NO 707 RES INST
- Filing Date
- 2026-03-31
- Publication Date
- 2026-07-24
AI Technical Summary
Traditional methods for allocating patrol missions among multiple unmanned vessels rely on manual parameter settings, which are not robust and make it difficult to achieve optimal results in complex and ever-changing real-world scenarios.
By combining deep learning algorithms with CBAA algorithms, and through the DQN network model and transfer learning, the inherent correlations in task allocation data are automatically discovered, and the optimal parameters are dynamically output, replacing the traditional manual setting.
It significantly improves the environmental adaptability and overall efficiency of multi-unmanned vessel collaborative patrol systems, reduces task conflicts and resource waste, and enhances robustness and global optimality of task allocation.
Smart Images
Figure CN122452979A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of unmanned vessel swarm cooperative control technology, and in particular to a multi-unmanned vessel cooperative task allocation algorithm based on deep learning optimization, which aims to improve the robustness and intelligence level of multi-agent cooperative task allocation. Background Technology
[0002] With the continuous growth of global trade and the deepening development of marine resources, the importance of maritime security and management is becoming increasingly prominent. Against this backdrop, collaborative patrols by multiple unmanned surface vessels (USVs) play a crucial role in ensuring maritime safety, safeguarding maritime rights, and improving the efficiency of maritime management. Traditional single-vessel patrol models, limited by factors such as the number of vessels, endurance, and monitoring range, are no longer sufficient to meet the increasingly complex marine environment and diverse patrol needs. Collaborative patrols by multiple USVs, by integrating the resources of multiple vessels, can significantly expand the patrol range, improve monitoring accuracy, and enhance the ability to respond to emergencies.
[0003] However, achieving efficient multi-unmanned surface vessel (USV) collaborative area patrols faces numerous challenges. One key challenge is how to rationally allocate patrol tasks among the vessels to maximize overall patrol efficiency and effectiveness. CBAA (Consensus-Based Auction Algorithm) has demonstrated good performance in task allocation in distributed systems. Through information exchange and consensus among nodes, it effectively solves the task allocation problem in multi-agent systems. However, due to limitations in its algorithmic principles, CBAA often requires manually setting parameters such as distance loss thresholds, time loss thresholds, and maximum depth thresholds based on the task scenario. Different task scenarios, such as varying numbers of USVs and task distributions, have significantly different requirements for these parameters. Manually setting parameters not only requires extensive experience and numerous experimental adjustments but also makes it difficult to guarantee optimal task allocation results in various complex and changing real-world scenarios.
[0004] In recent years, with the rapid development of artificial intelligence technology, deep learning algorithms have achieved remarkable application results in many fields due to their powerful feature learning and complex model building capabilities. Deep learning algorithms can automatically extract features and recognize patterns from large amounts of high-dimensional task allocation data, thus providing more accurate and intelligent decision support for multi-unmanned vessel (UAV) collaborative task allocation. Combining deep learning algorithms with the CBAA algorithm provides a new solution for task allocation in multi-unmanned vessel collaborative area patrols. By using deep learning algorithms to perform in-depth analysis and prediction of task allocation data and vessel status data, and then utilizing the CBAA algorithm to achieve efficient task allocation decisions, it is expected to overcome the limitations of traditional methods, optimize multi-unmanned vessel collaborative area patrol task allocation, and significantly improve the performance and efficiency of multi-unmanned vessel collaborative patrol systems. Summary of the Invention
[0005] To address the shortcomings of existing technologies, this invention proposes a multi-unmanned vessel collaborative task allocation algorithm based on deep learning optimization, which solves the problems of existing methods such as reliance on manual parameter settings and poor robustness.
[0006] The above-mentioned objective of this invention is achieved through the following technical solution: A multi-unmanned surface vessel cooperative task allocation algorithm based on deep learning optimization includes the following steps: Step 1: Construct a multi-unmanned vessel collaborative task allocation dataset based on actual task scenarios to provide data support for network model training; Step 2: The DQN algorithm is used as the core model, and the multilayer perceptron is used as the optimized network. The training is completed by mining the inherent correlation of allocation strategies under different scenarios. By combining pre-training and transfer learning, the model can quickly adapt to new scenarios and output key parameters such as time threshold, distance threshold, and maximum depth. Step 3: Use the parameters output by the DQN algorithm in Step 2 as input to the CBAA algorithm to complete the multi-unmanned vessel task allocation and path planning.
[0007] Moreover, step 1 includes: Step 1.1 Based on the actual mission scenario, determine the mission area and the ship's status information, including the mission area coordinates, the ship's coordinates, and the mission speed. Step 1.2 Randomly generate task scenarios. For each task scenario, record the task allocation results of the CBAA algorithm under different parameter configurations and the corresponding task configurations. Each parameter configuration is allocated at least three times. Manually select the better-performing samples and remove unreasonable data. Save the task allocation results and scenario information in xlsx format to construct a custom dataset. The task configuration includes the coordinate information of the task area, ship, and task point, as well as the effective time period of the task point.
[0008] Furthermore, in step 1.1, the task area includes 1 1 and 1 2. Two different task areas of varying sizes; for each task area, 1 to 4 virtual location coordinates are randomly generated each time to simulate a multi-unmanned vessel scenario, and all locations are within the task area; secondly, the task point information in the scenario is usually selected randomly, requiring the number of task points to be greater than the number of vessels but less than 100, and the distance between task points to be greater than 50 meters.
[0009] Furthermore, in step 1.2, the saved naming format follows the naming convention of "Task Area Ratio_Number of Ships_Number of Task Points_Number of Times Index.xlsx".
[0010] Furthermore, step 2 includes: Step 2.1: Build a deep Q-network model The DQN network adopts a direct output optimal target parameter structure and uses a multilayer perceptron as the main network. It includes an input layer, a hidden layer, and an output layer. The input layer consists of three types of data: environmental information, task list information, and unmanned surface vessel list information. The environmental information is a vector set composed of the latitude and longitude of the task area and the start and end time segments of the task. The hidden layer uses a multilayer perceptron and contains three sub-hidden layers, with corresponding output channels of 64, 128, and 64, respectively. The output layer is the target result, which includes the optimal time threshold, distance threshold, and maximum depth threshold for the current sample. Step 2.2: Perform model pre-training The Google Research Football dataset was selected as the pre-training dataset, and information such as player match status, ball position, and initial position were extracted from it and used as input to the model, corresponding to environmental information, task list information, and unmanned boat list information, respectively. Reward values were calculated based on player actions and match results, serving as feedback signals for reinforcement learning. Step 2.3: Perform model training and validation Based on the pre-training, the training set of the custom dataset in step 1 is used as the input data for model training, and the network model is trained a second time to optimize the parameters; and the validation set is further input into the trained network model to statistically analyze the model allocation results.
[0011] Furthermore, in step 2.1, SoftMax is selected as the activation function, and the cross-entropy function is used as the loss function.
[0012] The advantages and positive effects of this invention are as follows: 1. This invention employs a DQN network model as the parameter adaptive generator for the CBAA algorithm. Through deep learning of historical task data and similar dynamic decision-making scenario data, it can automatically uncover the potential correlations between environmental features, ship performance, and task requirements, replacing the parameter setting process that relies on manual experience in traditional methods. This mechanism enables the algorithm to dynamically output optimal parameters adapted to the current scenario when facing different ship cluster configurations and task combinations, significantly improving the algorithm's environmental adaptability.
[0013] 2. This invention introduces transfer learning, significantly reducing model training costs in new scenarios. It enables rapid algorithm deployment without requiring large amounts of labeled data, solving the data scarcity problem faced by traditional deep learning models in practical applications. Furthermore, the collaborative working mode of the DQN and CBAA algorithms retains the advantage of CBAA in efficiently achieving consensus in distributed systems while optimizing the global optimality of task allocation through deep reinforcement learning capabilities. This effectively reduces task conflicts and resource waste, improving the overall efficiency and robustness of multi-unmanned vessel collaborative operations. Attached Figure Description
[0014] Figure 1 This is a schematic diagram of the DQN model of the present invention. Figure 2 This is the optimized network of the DQN model of this invention; Figure 3 This is a comparison chart of the traditional CBAA allocation results and the improved CBAA allocation results of this invention. In the chart, This represents the i-th ship. 3a represents the traditional CBAA allocation result, and 3b represents the improved CBAA allocation result.
[0015] Figure 4 This is a schematic diagram of an actual mission scenario for an embodiment of the present invention. The diagram includes four unmanned vessels (top left corner), the mission area (orange background rectangle), 33 patrol points (yellow markers), and some threat / obstacle areas (red background). Figure 5 This invention is aimed at Figure 4 The diagram illustrates the optimal task allocation and path planning results for the given scenario. The blue flight paths represent the path planning results for each unmanned vessel. Detailed Implementation
[0016] The structure of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0017] This invention combines Deep Q-Network (DQN) with the CBAA algorithm. Utilizing the idea of transfer learning, it trains a DQN model suitable for regional patrol tasks using complex environmental data with similar dynamic decision-making characteristics to multi-unmanned surface vessel (USV) collaborative scenarios and historical USV task allocation cases. The model's prediction results are then used as input parameters for the CBAA algorithm to achieve adaptive parameter setting.
[0018] This invention is based on a deep learning-optimized multi-unmanned surface vessel cooperative task allocation algorithm. Please refer to [link / reference]. Figures 1-3 Its inventive point is: including the following steps: Step 1: Construct a multi-unmanned surface vessel (USV) collaborative task allocation dataset based on actual task scenarios to provide data support for network model training. This includes: Step 1.1 Based on the actual mission scenario, determine the mission area and the ship's status information, including the mission area coordinates, the ship's coordinates, and the mission speed. The mission area includes 1... 1 and 1 2. Two different task areas of varying sizes; for each task area, 1 to 4 virtual location coordinates are randomly generated each time to simulate a multi-unmanned vessel scenario, and all locations are within the task area; secondly, the task point information in the scenario is usually selected randomly, requiring the number of task points to be greater than the number of vessels but less than 100, and the distance between task points to be greater than 50 meters. Step 1.2 Randomly generate task scenarios, which include the coordinates of multiple patrol points and the effective time information for each patrol point, simulating real-world application scenarios. For each task scenario, manually set parameters such as distance threshold (range 0.0–1.0), time threshold (range 0.0–1.0), and maximum depth (range 1–total number of tasks). Record the task allocation results of the CBAA algorithm and the corresponding task configurations under different parameter configurations. Perform at least three allocations for each parameter configuration, record the best-performing examples, and save the corresponding task allocation results and scenario information in xlsx format. The saved naming format follows the naming convention of "task area ratio_number of ships_number of task points_number of times index.xlsx" to construct a custom dataset. The task configuration includes the coordinate information of the task area, ships, and task points, as well as the effective time period of the task points.
[0019] Step 1.3 divides the dataset into training and test sets. The custom dataset constructed in Step 1.2 is divided into training and test sets in an 8:2 ratio.
[0020] Step 2: Employ the DQN algorithm as the core model and a multilayer perceptron as the optimized network. Training is completed by exploring the inherent correlations between allocation strategies in different scenarios. Combining pre-training and transfer learning enables the model to quickly adapt to new scenarios and output key parameters such as time threshold, distance threshold, and maximum depth. This includes: Step 2.1: Build a deep Q-network model Common DQN networks often employ two structures: the first uses the input state information to train and output the Q-value for each action corresponding to the current sample data; the second directly allows the network to output the optimal action. This invention chooses the second structure, directly outputting the optimal target parameters, and uses a multilayer perceptron as the main network to improve the model's computation speed, forming the following structure: Figure 1As shown, the input layer consists of three types of data: environmental information, task list information, and unmanned surface vessel (USV) list information. The environmental information is a vector set composed of the latitude and longitude of the task area and the start and end times of the task. The hidden layer uses a multilayer perceptron, containing three sub-hidden layers with corresponding output channels of 64, 128, and 64, respectively. The output layer represents the target result, including the optimal time threshold, distance threshold, and maximum depth threshold for the current sample. Furthermore, this project selects SoftMax as the activation function and cross-entropy as the loss function.
[0021] Step 2.2: Perform model pre-training This invention selects the Google Research Football dataset as a pre-training dataset and extracts information such as player match status, ball position, and initial position from it. This information is used as input to the model, corresponding to environmental information, task list information, and unmanned boat list information, respectively. Reward values are calculated based on player actions and match results, serving as feedback signals for reinforcement learning.
[0022] Step 2.3: Perform model training and validation Based on the pre-training, the training set of the custom dataset in step 1 is used as the input data for model training, and the network model is trained a second time to optimize the parameters; and the validation set is further input into the trained network model to statistically analyze the model allocation results.
[0023] Step 3: Automatic parameter tuning task allocation for the CBAA algorithm based on DQN Step 3.1 Input the actual mission scenario information (including mission area location coordinates, ship position coordinates, mission speed, patrol point coordinates, and effective time information for each patrol point) into the trained DQN algorithm model, and the model outputs the optimal parameter configuration (including distance threshold, time threshold, and maximum depth parameter value) adapted to the current mission scenario.
[0024] Step 3.2 Directly configure the optimal parameters output by the DQN model as the running parameters of the CBAA algorithm to complete the automatic parameter adjustment of the CBAA algorithm.
[0025] Step 3.3 Input the same task scenario information into the CBAA algorithm with completed parameter configuration, execute multi-unmanned vessel cooperative task allocation and path planning, and obtain the final optimal task allocation result.
[0026] See the examples. Figure 4 and Figure 5 .
[0027] Although embodiments and drawings of the present invention have been disclosed for illustrative purposes, those skilled in the art will understand that various substitutions, variations and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the scope of the present invention is not limited to the contents disclosed in the embodiments and drawings.
Claims
1. A multi-unmanned surface vessel cooperative task allocation algorithm based on deep learning optimization, characterized in that, Includes the following steps: Step 1: Construct a multi-unmanned vessel collaborative task allocation dataset based on actual task scenarios to provide data support for network model training; Step 2: The DQN algorithm is used as the core model, and the multilayer perceptron is used as the optimized network. The training is completed by mining the inherent correlation of allocation strategies under different scenarios. By combining pre-training and transfer learning, the model can quickly adapt to new scenarios and output key parameters such as time threshold, distance threshold, and maximum depth. Step 3: Use the parameters output by the DQN algorithm in Step 2 as input to the CBAA algorithm to complete the multi-unmanned vessel task allocation and path planning.
2. The multi-unmanned vessel cooperative task allocation algorithm based on deep learning optimization according to claim 1, characterized in that, Step 1 includes: Step 1.1 Based on the actual mission scenario, determine the mission area and the ship's status information, including the mission area coordinates, the ship's coordinates, and the mission speed. Step 1.2 Randomly generate task scenarios. For each task scenario, record the task allocation results of the CBAA algorithm under different parameter configurations and the corresponding task configurations. Each parameter configuration is allocated at least three times. Manually select the better-performing samples and remove unreasonable data. Save the task allocation results and scenario information in xlsx format to construct a custom dataset. The task configuration includes the coordinate information of the task area, ship, and task point, as well as the effective time period of the task point.
3. The multi-unmanned vessel cooperative task allocation algorithm based on deep learning optimization according to claim 2, characterized in that: In step 1.1, the task area includes 1 1 and 1 2. Two different task areas of varying sizes; for each task area, 1 to 4 virtual location coordinates are randomly generated each time to simulate a multi-unmanned vessel scenario, and all locations are within the task area; secondly, the task point information in the scenario is usually selected randomly, requiring the number of task points to be greater than the number of vessels but less than 100, and the distance between task points to be greater than 50 meters.
4. The multi-unmanned vessel cooperative task allocation algorithm based on deep learning optimization according to claim 2, characterized in that: In step 1.2, the saved naming format follows the naming convention of "Task Area Ratio_Number of Ships_Number of Task Points_Number of Times Index.xlsx".
5. The multi-unmanned vessel cooperative task allocation algorithm based on deep learning optimization according to claim 1, characterized in that, Step 2 includes: Step 2.1: Build a deep Q-network model The DQN network adopts a direct output optimal target parameter structure and uses a multilayer perceptron as the main network. It includes an input layer, a hidden layer, and an output layer. The input layer consists of three types of data: environmental information, task list information, and unmanned surface vessel list information. The environmental information is a vector set composed of the latitude and longitude of the task area and the start and end time segments of the task. The hidden layer uses a multilayer perceptron and contains three sub-hidden layers, with corresponding output channels of 64, 128, and 64, respectively. The output layer is the target result, which includes the optimal time threshold, distance threshold, and maximum depth threshold for the current sample. Step 2.2: Perform model pre-training The Google Research Football dataset was selected as the pre-training dataset, and information such as player match status, ball position, and initial position were extracted from it and used as input to the model, corresponding to environmental information, task list information, and unmanned boat list information, respectively. Reward values were calculated based on player actions and match results, serving as feedback signals for reinforcement learning. Step 2.3: Perform model training and validation Based on the pre-training, the training set of the custom dataset in step 1 is used as the input data for model training, and the network model is trained a second time to optimize the parameters; and the validation set is further input into the trained network model to statistically analyze the model allocation results.
6. The multi-unmanned vessel cooperative task allocation algorithm based on deep learning optimization according to claim 5, characterized in that: In step 2.1, SoftMax is selected as the activation function, and cross-entropy is used as the loss function.