Dynamic task allocation method and system for offshore unmanned ship cluster based on ADM-MADDPG

Abstract

The invention discloses an ADM-MADDPG-based dynamic task allocation method and system for an offshore unmanned ship cluster, and the method comprises the following steps: firstly, taking the real-time state information of a detected task target and the dynamic state information of the unmanned ship cluster as input data; the constructed feature extraction and fusion module performs feature compression and correlation modeling on the multi-source information so as to form unified high-dimensional feature representation; thirdly, constructing a self-adaptive judgment mechanism based on an evaluation network, performing comprehensive analysis and real-time evaluation on the input environment situation, and when an evaluation result exceeds a preset confidence threshold value, judging that a current task allocation scheme has an optimization requirement, and triggering a task allocation adjustment process; and then, inputting the state information processed by the feature extraction and fusion module into the trained Actor network module, and giving an unmanned ship cluster task allocation scheme with optimal performance in the current environment. According to the invention, the autonomous decision-making level and cooperation efficiency of the unmanned ship cluster are improved.

Description

Technical Field

[0001] This invention relates to a dynamic task allocation method for unmanned surface vessel (USV) swarms, specifically to a dynamic task allocation method and system for maritime USV swarms based on ADM-MADDPG. Background Technology

[0002] Unmanned surface vehicles (USVs), as intelligent unmanned platforms, are characterized by their small size, high autonomy, and high flexibility. In the complex and ever-changing marine environment, a single USV cannot guarantee the successful completion of a mission. Collaborative cooperation among USV swarms can compensate for the shortcomings of a single vessel, demonstrating great application potential in areas such as environmental monitoring, intelligence reconnaissance, and situational awareness.

[0003] Task allocation, a key technology for collaborative unmanned surface vessel (USV) swarms, is the core and guarantee of USV swarm cooperation. However, the rapidly changing maritime situation renders task allocation no longer a static process, requiring rapid dynamic adjustments and on-the-spot decisions based on constantly changing real-time information in the environment. Traditional task allocation methods, such as heuristic algorithms and objective function-based optimization algorithms, can stably complete task allocation through simple behavioral rules and method settings, but they struggle to adapt flexibly to complex tasks and environments, and are prone to getting trapped in local optima in complex problems. Maritime missions are complex, with constantly changing real-time situations, necessitating dynamic adjustments to tasks while considering the opportune moment for such adjustments.

[0004] Reinforcement learning, as an important branch of artificial intelligence, can proactively learn optimal strategies through interaction with the environment. It has significant advantages in decision-making for complex, dynamic, and uncertain problems, and has also found numerous applications in task allocation. With the development of AI, multi-agent reinforcement learning (MARL) algorithms are increasingly being applied to the field of unmanned swarms. This method simulates the interaction between agents and the environment, as well as the interaction between agents themselves, to achieve task allocation and resource optimization. However, when dealing with complex tasks in maritime environments, MARL-based methods suffer from problems such as unclear collaboration methods between individuals and the need to improve the convergence and robustness of the algorithm. Summary of the Invention

[0005] The purpose of this invention is to provide a dynamic task allocation method and system for unmanned surface vessel (USV) swarms based on ADM-MADDPG, thereby improving the autonomous decision-making level and collaborative efficiency of USV swarms.

[0006] The technical solution to achieve the purpose of this invention is as follows:

[0007] A dynamic task allocation method for unmanned surface vessel (USV) swarms based on ADM-MADDPG includes:

[0008] An ADM-MADDPG network based on the Actor-Critic framework was constructed and trained. In the ADM-MADDPG, the Critic network adopts an adaptive discrimination mechanism. The Actor network is designed as a three-head output structure including an index head, an action head, and a quantity head. The index head is responsible for outputting the task target allocation result, while the action head and the quantity head output the specific action type and the number of unmanned surface vessels executing the task, respectively.

[0009] The real-time status information of the detection mission target and the dynamic status information of the unmanned surface vessel cluster itself are used to compress and correlate the status information through the constructed feature extraction and fusion module to form a unified high-dimensional feature representation, which serves as the input of the ADM-MADDPG network.

[0010] The Critic network performs comprehensive analysis and real-time evaluation of the input state information. When the evaluation result exceeds the preset threshold, it determines that the current unmanned surface vessel cluster task allocation scheme needs optimization and triggers the Actor network.

[0011] Based on the input state information, the Actor network provides the optimal task allocation scheme for the unmanned surface vessel swarm under the current environment.

[0012] Furthermore, the feature extraction and fusion module uses a multilayer perceptron structure with shared weights for feature extraction and achieves feature fusion through a one-dimensional convolutional kernel combined with an attention mechanism. The multilayer perceptron structure adopts a normalization mechanism, and cross-layer residual channels are introduced when the one-dimensional convolutional kernel is combined with the attention mechanism to achieve feature fusion.

[0013] Furthermore, the evaluation model for the Critic network is as follows:

[0014] S eval =ρ1·m1+ρ2·m2+ρ3·m3+ρ4·m4

[0015] Where m1 represents the number of surviving task executors, m2 represents the number of available task executors, m3 represents the number of surviving targets, m4 represents the target release batch, and ρ1, ρ2, ρ3, and ρ4 represent the corresponding weights.

[0016] S eval To evaluate the confidence value of the network output.

[0017] Furthermore, the confidence threshold is 0.55.

[0018] Furthermore, the ADM-MADDPG employs a centralized training and distributed execution approach during training, separating the training and execution processes. During training, the Critic network shares global information, evaluates the value of actions based on the current state, and updates the target value and network parameters according to the current reward. The Actor network continuously optimizes the policy network parameters using a policy gradient algorithm. During execution, the Actor network uses only its locally observed information and executes the policy in parallel. Each unmanned surface vessel (USV) formation takes actions based on its own policy, then obtains the state information for the next time step according to the state transition equation, obtains the reward for the current time step according to the state-behavior equation, and simultaneously obtains the observation value for the next time step based on the state information for the next time step.

[0019] Furthermore, the gradient of the policy gradient algorithm is:

[0020] For a stochastic policy, the policy gradient of the i-th unmanned surface vessel formation is:

[0021]

[0022] in, For the expected value of unmanned surface vessel i, π i (a i |o i ) is the random strategy for the i-th unmanned surface vessel formation. Let a be the action value function of unmanned surface vessel formation i, where a = [a1, ..., a2]. n ] is a combined action, θ i Represents the strategy parameters of unmanned surface vessel formation i;

[0023] For a deterministic policy, the policy gradient of the i-th unmanned surface vessel formation is:

[0024]

[0025] Where D represents the experience pool, The policy network represents the policy network with respect to θ i gradient calculation, It is an action function with respect to action a. i The gradient, J(θ) i ), J(μ) i ) represents θ i μ i The cumulative expected reward for the i-th unmanned surface vessel formation.

[0026] Furthermore, the cumulative expected reward is:

[0027]

[0028] In the formula, p πLet a be a state distribution, where a = [a1, ..., a2] n This is a joint action. Let r be the action strategy, γ be the discount factor, and r be the discount factor. i,t The reward for the i-th unmanned surface vessel formation at time t.

[0029] Furthermore, the index header adopts a classification-based Softmax structure and introduces an adaptive target embedding module. By mapping within the target representation space, a unified index expression under different task scales is achieved.

[0030] Furthermore, the quantity head introduces a parameterized integer constraint mechanism to predict the allocation ratio in a continuous space, and then obtains the accurate quantity through discretization mapping.

[0031] A dynamic task allocation system for unmanned surface vessels (USVs) swarms based on ADM-MADDPG includes:

[0032] The ADM-MADDPG network construction and training unit is used to construct and train an ADM-MADDPG network based on the Actor-Critic framework. In the ADM-MADDPG, the Critic network adopts an adaptive discrimination mechanism, and the Actor network is designed as a three-head output structure including an index head, an action head, and a quantity head. The index head is responsible for outputting the task target allocation result, and the action head and quantity head output the specific action type and the number of unmanned surface vessels executing the task, respectively.

[0033] The real-time data acquisition unit detects the real-time status information of the mission target and the dynamic status information of the unmanned surface vessel cluster itself. Through the constructed feature extraction and fusion module, the status information is compressed and correlated to form a unified high-dimensional feature representation, which serves as the input to the ADM-MADDPG network.

[0034] The dynamic task allocation unit, Critic Network, performs comprehensive analysis and real-time evaluation of the input state information. When the evaluation result exceeds a preset threshold, it determines that the current unmanned surface vessel (USV) cluster task allocation scheme needs optimization and triggers Actor Network. Based on the input state information, Actor Network provides the optimal USV cluster task allocation scheme under the current environment.

[0035] Compared with existing technologies, the significant advancements of this invention are as follows: First, it proposes a multilayer perceptron structure based on shared weights for feature extraction, and combines one-dimensional convolutional kernels with an attention mechanism to achieve efficient feature fusion. This not only significantly improves the efficiency of task allocation and action decision-making, but also accelerates the integration process of coupled information between targets. Second, it designs a three-head output structure based on functional decoupling, which effectively reduces the dimensionality of the action space, thereby accelerating model convergence and improving decision accuracy. Third, it designs an adaptive discrimination mechanism based on an evaluation network, which can quickly determine the timing of task allocation scheme adjustments based on real-time situation, thereby ensuring that unmanned surface vessel swarms have greater flexibility and adaptability in complex and dynamic battlefield environments. Attached Figure Description

[0036] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention.

[0037] In the attached diagram:

[0038] Figure 1 This is an overall flowchart of the dynamic task allocation method for marine unmanned surface vessel swarms based on ADM-MADDPG.

[0039] Figure 2 This document details the implementation process of a dynamic task allocation method for unmanned surface vessel (USV) swarms based on ADM-MADDPG.

[0040] Figure 3 This describes the specific structure of the ADM-MADDPG algorithm. Detailed Implementation

[0041] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments represent only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0042] like Figures 1-3 As shown, a dynamic task allocation method for unmanned surface vessel (USV) swarms based on ADM-MADDPG includes the following steps:

[0043] S1. Using the real-time status information of the detected mission target and the dynamic status information of the unmanned surface vessel cluster itself as input data, the constructed feature extraction and fusion module performs feature compression and correlation modeling on multi-source information, thereby forming a unified high-dimensional feature representation.

[0044] S2. Construct an adaptive discrimination mechanism based on an evaluation network to comprehensively analyze and evaluate the input environmental situation in real time. When the evaluation result exceeds a preset confidence threshold, it is determined that the current task allocation scheme needs optimization, triggering the task allocation adjustment process.

[0045] S3. Input the state information processed by the feature extraction and fusion module into the ADM-MADDPG Actor network module to provide efficient and structured environmental perception information for subsequent task decision-making;

[0046] S4. The Actor network is designed as a three-head output structure, where the index head is responsible for outputting the task objective allocation result, and the action head and quantity head output the specific action type and the number of unmanned surface vessels executing the task, respectively, to achieve hierarchical decoupling of task allocation and action decision-making.

[0047] S5. Through multiple rounds of training and policy iteration, optimize model parameters and improve convergence performance, and finally output the best-performing unmanned surface vessel swarm task allocation scheme under the current environment.

[0048] In step S1, the real-time status information of the mission target includes the target's position, speed, target type, target survival status, and target release batch; the dynamic status information of the unmanned surface vessel (USV) swarm includes the position, speed, payload quantity, survival status, and allocation status of a single USV; the processed number of USVs surviving, USV allocation status, target survival quantity, and target release batch are used as inputs to the evaluation network, and the processed real-time status information of the mission target and the dynamic status information of the USV swarm are used as inputs to the Actor network.

[0049] In this embodiment, the ADM-MADDPG model is based on a multi-agent reinforcement learning method. It improves the efficiency of target allocation and action decision-making through feature extraction and fusion modules, accelerates algorithm convergence using a multi-head output structure, and employs an adaptive discrimination mechanism based on an evaluation network to determine when to adjust the task allocation scheme according to the real-time situation. This task allocation method can determine the timing of adjustments based on actual conditions under complex sea conditions and make adjustments accordingly, thereby improving task execution efficiency and ensuring that the unmanned surface vessel swarm can flexibly cope with complex environments.

[0050] The ADM-MADDPG adopts an Actor-Critic framework. The Actor network provides a deterministic action policy based on the current state information of the agent, enabling the agent to execute the optimal action. At the same time, the policy network parameters are continuously optimized through a policy gradient algorithm. The Critic network evaluates the value of the agent's actions based on the current state and updates the target value and network parameters according to the agent's reward, thereby improving the accuracy of the estimation.

[0051] MADDPG employs a centralized training and distributed execution approach during training, separating the training and execution processes. Specifically, during the training phase, the agent's Critic network shares global information, evaluates the current state, and updates the Actor and Critic network parameters. During the execution phase, the agent's Actor network uses only its own locally observed information to execute the policy in parallel. Each unmanned surface vessel (USV) formation takes actions based on its own policy, then obtains the next-time state information according to the state transition equation, obtains the current-time reward according to the state-behavior equation, and simultaneously obtains the next-time observation value based on the next-time state information.

[0052] In the mission scenario, the unmanned surface vessel (USV) swarm consists of n agents. In the MADDPG algorithm, π = [π1, ..., π]. n ] represents the strategy of each unmanned surface vessel formation, θ = [θ1, ..., θ2] n Let ] represent the strategy parameters for each unmanned surface vessel (USV) formation. Then, the cumulative expected reward for the i-th USV formation is:

[0053]

[0054] In the formula, p π Let a be a state distribution, where a = [a1, ..., a2] n This is a joint action. Let r be the action strategy, γ be the discount factor, and r be the discount factor. i,t The reward for the i-th unmanned surface vessel formation at time t.

[0055] For a stochastic policy, the policy gradient of the i-th unmanned surface vessel formation is:

[0056]

[0057] For a deterministic policy, the policy gradient of the i-th unmanned surface vessel formation is:

[0058]

[0059] Each unmanned surface vessel (USV) formation takes action based on its own strategy, then obtains the state information for the next moment according to the state transition equation, obtains the reward for the current moment according to the state-behavior equation, and obtains the observation value for the next moment according to the state information for the next moment.

[0060] In this invention, the data required for training is obtained through interaction with the simulation environment, rather than pre-constructed static data. During training, the agent interacts with the simulation environment, continuously collecting state information of the unmanned surface vessel (USV) and the mission target, executing actions, and receiving reward feedback, thereby generating a better policy network to guide the agent's action generation. The agent's input is all state information of the USV and the mission target, including position, velocity, and survival status. The policy network outputs the optimal task allocation strategy at the current moment and provides the timing for adjusting the task allocation scheme based on an adaptive discrimination mechanism. In this embodiment, the model is developed based on Python 3.9 and the accompanying PyTorch 1.13.1 as a deep learning development package, and adopts the ADM-MADDPG-based dynamic task allocation method for maritime USV swarms as the task allocation scheme.

[0061] To address the challenges of highly coupled and complex state information from multiple targets and platforms in complex battlefield environments, a hierarchical feature extraction and fusion module is proposed in step S1. This module consists of a feature extraction unit and a feature fusion unit, which work together to achieve efficient compression and semantic enhancement of state information.

[0062] In this embodiment, in step S1, the real-time state information of the detected mission target and the state information of the unmanned surface vessel (USV) cluster itself are used as input data. A multi-layer perceptron structure with shared weights is employed for feature extraction, and feature fusion is achieved through a one-dimensional convolutional kernel combined with an attention mechanism. This allows for feature compression and correlation modeling of multi-source information, forming a unified high-dimensional feature representation to support subsequent decision-making. Since the state vector structures of the USV platform and the combat target are identical, traditional methods are prone to feature redundancy and low extraction efficiency. Therefore, this invention innovatively adopts a multi-layer perceptron (MLP) structure with shared weights, performing parameter-sharing convolutional processing for different platforms and targets. Compared to independent modeling, this method significantly reduces the number of parameters while extracting high-dimensional feature representations with consistent semantic expression, aligning the USV and the target in a unified feature space for easier subsequent fusion. To further enhance the feature discriminative ability, a hierarchical normalization mechanism (LayerNormalization + Group Normalization) is introduced to ensure that features from different scales and sources have consistent distribution characteristics upon input, thereby avoiding gradient vanishing and feature bias problems.

[0063] After obtaining the extracted multi-source features, the feature fusion unit utilizes a one-dimensional convolutional kernel combined with an attention mechanism (Conv1D+Attention) to achieve cross-target information interaction. One-dimensional convolution is used to model the local interaction relationships between different targets, focusing on capturing the coupling features between neighboring targets; simultaneously, the embedded attention mechanism selectively amplifies the feature weights of key targets, thereby achieving a dynamic optimization effect of "highlighting important targets and suppressing irrelevant targets." Furthermore, this invention introduces a cross-layer residual channel in the feature fusion unit to ensure that the fused global information still retains the detailed expression of local features, improving both the model's generalization ability and its sensitivity to sudden situations.

[0064] This feature extraction and fusion module enables unmanned surface vessel swarms to rapidly characterize multi-target, multi-platform environments with limited computing resources, ensuring real-time performance while improving the perception accuracy and robustness of the agent in complex situations.

[0065] In this embodiment, step S2 proposes an adaptive discrimination mechanism based on an evaluation network. This mechanism comprehensively analyzes multiple factors such as the number of surviving task executors, the number of available platforms, the number of surviving targets, and the number of target release batches through the evaluation network, and sets a threshold of 0.55; when the output result of the evaluation network exceeds this threshold, the dynamic adjustment of the task allocation scheme is triggered.

[0066] To better cope with dynamic mission environments and achieve adaptive adjustment of task allocation for unmanned surface vessel (USV) swarms, an adaptive discrimination mechanism based on an evaluation network was designed. This mechanism utilizes an evaluation network to comprehensively analyze multiple factors to assess the current field situation, analyze the environmental situation in real time, and indicate when to adjust the task allocation scheme.

[0067] To make the evaluation network more comprehensive, the discrimination mechanism fully considers various factors, including the number of surviving task executors (m1), the number of available task executors (m2), the number of surviving targets (m3), and the target release batches (m4). These factors are assigned different weights (ρ1, ρ2, ρ3, ρ4) and comprehensively evaluated through an algorithm model. The evaluation network outputs a confidence value (S) between 0 and 1. eval ), representing the probability of task executor reallocation under the current conditions. The evaluation formula for the network is shown below:

[0068] S eval =ρ1·m1+ρ2·m2+ρ3·m3+ρ4·m4

[0069] Specifically, the adaptive discrimination mechanism based on the evaluation network evaluates the current task allocation scheme by setting a confidence threshold. After the evaluation network calculates and evaluates the existing allocation scheme, if its output exceeds the preset confidence threshold, it determines that the current task allocation deviates from the environmental situation or its efficiency has decreased. In this way, the discrimination mechanism is immediately triggered, initiating a dynamic correction process for the task allocation scheme. In this manner, the system can intelligently identify the critical moments when adjustments are truly needed without frequent global updates, thus avoiding the timeliness issues caused by over-computation and improving the targeting of decisions and resource utilization.

[0070] After triggering the discrimination mechanism, the ADM-MADDPG model proposed in this invention comprehensively considers multiple factors such as the spatial distribution of unmanned surface vessels (USVs), remaining payload, platform status, and operational target priority to adaptively partition and reconfigure the swarm formation. Unlike traditional methods based on fixed allocation rules, this invention focuses not only on globally optimal allocation efficiency during the repartitioning process but also on local collaborative execution capabilities, enabling closer and more efficient task cooperation among the agents. Specifically, spatial coordinate information is used to ensure the rationality of task coverage in the geographical distribution of USVs, payload information is used to ensure the executability and continuity of task allocation, and target priority and platform status further optimize the dynamic weights of the allocation. Based on this, the system can generate more refined task division and action strategies, improving overall collaborative combat effectiveness.

[0071] In this embodiment, in step S3, the state information processed by the feature extraction and fusion module, including the unmanned surface vessel's own position, speed, load status, and the target's current position and survival status, is used as the input to the ADM-MADDPG Actor network module.

[0072] In this embodiment, in step S4, the ADM-MADDPG Actor network is designed as a three-head output structure based on functional decoupling, which is used to output the allocation result of the task objective, as well as the specific action type and the number of unmanned surface vessels executing the task, thereby reducing the dimensionality of the action space.

[0073] To avoid the "action space dimension explosion" problem faced by traditional single-head Actor networks in large-scale task allocation problems, this invention proposes a three-head output structure based on functional decoupling, which innovatively separates the modeling of "target selection" and "action decision", thereby significantly reducing the action dimension and improving convergence efficiency.

[0074] 1) Index Header (Target Allocation Subnetwork)

[0075] It specifically outputs index information of task targets and adopts a classification-based Softmax structure, which can efficiently complete assignment among multiple candidate targets. To avoid index instability caused by dynamic changes in the number of targets, an adaptive target embedding module is introduced, which achieves a unified index expression under different task scales by mapping within the target representation space.

[0076] 2) Action Head (Action Type Subnetwork)

[0077] This is used to output the specific action type of the unmanned surface vessel (USV) under the assigned target, such as "attack" or "encirclement". A Gumbel-Softmax-based sampling method is employed, which ensures both the discreteness of action selection and the trainability of gradients, making the action learning process more stable.

[0078] 3) Quantity Header (Resource Allocation Sub-network)

[0079] This subnetwork is responsible for outputting the number of unmanned surface vessels (USVs) performing the current task, avoiding the exponential dimensionality expansion caused by combinatorial encoding in traditional schemes. It introduces a parameterized discrete allocation mechanism, which can predict the allocation ratio in a continuous space and then obtain the precise number through discretization mapping, thus balancing the flexibility and executability of the allocation.

[0080] Compared to traditional single-output modes, the three-output structure of this invention achieves hierarchical decoupling of "target selection—action execution—resource allocation," significantly reducing the complexity of the action space. Simultaneously, through the collaborative optimization of the three sub-networks, rapid convergence and improved policy stability can be achieved, ensuring that the unmanned surface vessel swarm can achieve efficient and robust task allocation and action decision-making in dynamic environments.

[0081] In this embodiment, step S5 employs the MADDPG optimization strategy for gradient updates to improve training stability. During optimization, the initial learning rate is set to 0.001, and an entropy adjustment mechanism is used to enhance strategy exploration. During training, each agent interacts with the environment at every moment, iterating continuously until convergence to obtain the optimal task allocation strategy, thereby improving task completion efficiency and combat effectiveness.

[0082] In summary, this invention improves decision-making efficiency through feature extraction and fusion modules, optimizes task planning using a multi-head output structure, and introduces an adaptive discrimination mechanism based on an evaluation network for accurate environmental analysis, thereby enhancing the algorithm's adaptability to dynamic environments and improving the autonomous decision-making level and collaborative efficiency of unmanned surface vessel swarms.

[0083] This embodiment also provides a dynamic task allocation system for a swarm of unmanned surface vessels (USVs) based on ADM-MADDPG, including:

[0084] The ADM-MADDPG network construction and training unit is used to construct and train an ADM-MADDPG network based on the Actor-Critic framework. In the ADM-MADDPG, the Critic network adopts an adaptive discrimination mechanism, and the Actor network is designed as a three-head output structure including an index head, an action head, and a quantity head. The index head is responsible for outputting the task target allocation result, and the action head and quantity head output the specific action type and the number of unmanned surface vessels executing the task, respectively.

[0085] The real-time data acquisition unit detects the real-time status information of the mission target and the dynamic status information of the unmanned surface vessel cluster itself. Through the constructed feature extraction and fusion module, the status information is compressed and correlated to form a unified high-dimensional feature representation, which serves as the input to the ADM-MADDPG network.

[0086] The dynamic task allocation unit, Critic Network, performs comprehensive analysis and real-time evaluation of the input state information. When the evaluation result exceeds a preset threshold, it determines that the current unmanned surface vessel (USV) cluster task allocation scheme needs optimization and triggers Actor Network. Based on the input state information, Actor Network provides the optimal USV cluster task allocation scheme under the current environment.

[0087] In this document, terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily indicate an actual relationship or order between them. Furthermore, the use of terms such as “comprising,” “including,” or other similar terms implies non-exclusive inclusion, meaning that a process, method, article, or apparatus that covers a range of elements includes not only those explicitly listed but also other elements not explicitly listed, or elements inherent to the process, method, article, or apparatus.

[0088] Although embodiments of the invention have been shown and described, those skilled in the art will recognize that various changes, modifications, substitutions, and alterations can be made to these embodiments without departing from the principles and spirit of the invention. The scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A dynamic task allocation method for maritime unmanned surface vessel swarms based on ADM-MADDPG, characterized in that, include: An ADM-MADDPG network based on the Actor-Critic framework was constructed and trained. In the ADM-MADDPG, the Critic network adopts an adaptive discrimination mechanism. The Actor network is designed as a three-head output structure including an index head, an action head, and a quantity head. The index head is responsible for outputting the task target allocation result, while the action head and the quantity head output the specific action type and the number of unmanned surface vessels executing the task, respectively. The real-time status information of the detection mission target and the dynamic status information of the unmanned surface vessel cluster itself are used to compress and correlate the status information through the constructed feature extraction and fusion module to form a unified high-dimensional feature representation, which serves as the input of the ADM-MADDPG network. The Critic network performs comprehensive analysis and real-time evaluation of the input state information. When the evaluation result exceeds the preset threshold, it determines that the current unmanned surface vessel cluster task allocation scheme needs optimization and triggers the Actor network. Based on the input state information, the Actor network provides the optimal task allocation scheme for the unmanned surface vessel swarm under the current environment.

2. The method for dynamic task allocation of unmanned surface vessel swarms based on ADM-MADDPG as described in claim 1, characterized in that, The feature extraction and fusion module uses a multilayer perceptron structure with shared weights for feature extraction and achieves feature fusion through a one-dimensional convolutional kernel combined with an attention mechanism. The multilayer perceptron structure adopts a normalization mechanism, and cross-layer residual channels are introduced when the one-dimensional convolutional kernel is combined with the attention mechanism to achieve feature fusion.

3. The method for dynamic task allocation of unmanned surface vessel swarms based on ADM-MADDPG according to claim 1, characterized in that, The evaluation model for the Critic network is as follows: S eval =ρ1·m1+ρ2·m2+ρ3·m3+ρ4·m4 Where m1 represents the number of surviving task executors, m2 represents the number of available task executors, m3 represents the number of surviving targets, m4 represents the target release batch, ρ1, ρ2, ρ3, and ρ4 represent the corresponding weights, and S eval To evaluate the confidence value of the network output.

4. The method for dynamic task allocation of unmanned surface vessel swarms based on ADM-MADDPG as described in claim 1, characterized in that, The confidence threshold is 0.

55.

5. The method for dynamic task allocation of unmanned surface vessel swarms based on ADM-MADDPG according to claim 1, characterized in that, The ADM-MADDPG employs a centralized training and distributed execution approach during training, separating the training and execution processes. During training, the Critic network shares global information, evaluates the value of actions based on the current state, and updates the target value and network parameters according to the current reward. The Actor network continuously optimizes the policy network parameters using a policy gradient algorithm. During execution, the Actor network uses only its locally observed information and executes the policy in parallel. Each unmanned surface vessel (USV) formation takes actions based on its own policy, then obtains the state information for the next time step according to the state transition equation, obtains the reward for the current time step according to the state-behavior equation, and simultaneously obtains the observation value for the next time step based on the state information for the next time step.

6. The method for dynamic task allocation of unmanned surface vessel swarms based on ADM-MADDPG according to claim 5, characterized in that, The gradient of the policy gradient algorithm is: For a stochastic policy, the policy gradient of the i-th unmanned surface vessel formation is: in, For the expected value of unmanned surface vessel i, π i (a i |o i ) is the random strategy for the i-th unmanned surface vessel formation. Let a be the action value function of unmanned surface vessel formation i, where a = [a1, ..., a2]. n ] represents a combined action, θ i This represents the strategy parameters of the unmanned surface vessel (USV) formation i. For a deterministic policy, the policy gradient of the i-th unmanned surface vessel formation is: Where D represents the experience pool, The policy network represents the policy network with respect to θ i gradient calculation, It is an action function with respect to action a. i The gradient, J(θ) i ), J(μ) i ) represents θ i μ i The cumulative expected reward for the i-th unmanned surface vessel formation.

7. A dynamic task allocation method for a maritime unmanned surface vessel swarm based on ADM-MADDPG as described in claim 6, characterized in that, The cumulative expected reward is: In the formula, p π Let a be a state distribution, where a = [a1, ..., a2] n This is a joint action. Let r be the action strategy, γ be the discount factor, and r be the discount factor. i,t The reward for the i-th unmanned surface vessel formation at time t.

8. A method for dynamic task allocation in a swarm of unmanned surface vessels based on ADM-MADDPG as described in claim 1, characterized in that, The index header adopts a classification-based Softmax structure and introduces an adaptive target embedding module. By mapping within the target representation space, it achieves a unified index expression for different task scales.

9. A dynamic task allocation method for a maritime unmanned surface vessel swarm based on ADM-MADDPG as described in claim 1, characterized in that, The quantity head introduces a parameterized integer constraint mechanism to predict the allocation ratio in a continuous space, and then obtains the accurate quantity through discretization mapping.

10. A dynamic task allocation system for a swarm of unmanned surface vessels (USVs) implementing the method of any one of claims 1-9, characterized in that, include: The ADM-MADDPG network construction and training unit is used to construct and train an ADM-MADDPG network based on the Actor-Critic framework. In the ADM-MADDPG, the Critic network adopts an adaptive discrimination mechanism, and the Actor network is designed as a three-head output structure including an index head, an action head, and a quantity head. The index head is responsible for outputting the task target allocation result, and the action head and quantity head output the specific action type and the number of unmanned surface vessels executing the task, respectively. The real-time data acquisition unit detects the real-time status information of the mission target and the dynamic status information of the unmanned surface vessel cluster itself. Through the constructed feature extraction and fusion module, the status information is compressed and correlated to form a unified high-dimensional feature representation, which serves as the input to the ADM-MADDPG network. The dynamic task allocation unit, Critic Network, performs comprehensive analysis and real-time evaluation of the input state information. When the evaluation result exceeds a preset threshold, it determines that the current unmanned surface vessel (USV) cluster task allocation scheme needs optimization and triggers Actor Network. Based on the input state information, Actor Network provides the optimal USV cluster task allocation scheme under the current environment.

Images