Migratable task planning method and system under coupling of unmanned system cluster environment

Abstract

The application discloses a migratable task planning method and system under an unmanned system cluster environment coupling, relates to the unmanned system cluster control technical field, and comprises the following steps: acquiring unmanned system cluster environment information, performing task allocation by using a greedy strategy, obtaining a task allocation result set corresponding to a current iteration number, representing the task allocation result as a graph structure, acquiring a feature matrix, a relationship adjacency matrix and a corresponding real reward value of the graph structure, and pre-training a differentiable pooling graph neural network model. When the environment coupling occurs, the pre-trained model is migrated to the changed environment, fine-tuning training is performed, the fine-tuned differentiable pooling graph neural network model is used to predict the best task planning result under the environment coupling, and the task planning efficiency is improved at a small cost. The application is suitable for the migration problem of unmanned cluster task planning under the environment coupling that the number and / or position of unmanned systems, tasks and obstacles change.

Description

Technical Field

[0001] This invention relates to the field of unmanned system cluster control technology, and in particular to a portable task planning method and system under the coupling of unmanned system cluster environment. Background Technology

[0002] In recent years, with the development of urbanization and changes in the international situation, urban warfare has become increasingly frequent. With the development of autonomous technologies, unmanned system swarm technology is playing an increasingly important role in urban missions. Unmanned system swarms have advantages such as low cost, high flexibility, and large scale, enabling them to accomplish many complex tasks. However, in urban warfare, the constantly changing urban environment brings about a strong environmental coupling problem. Environmental coupling refers to the impact of environmental changes on the original task allocation results of unmanned system swarms, leading to lower-than-expected accuracy and efficiency. For example, when obstacles are destroyed, mission locations change, or dangerous bombing zones are designated on the battlefield, the original mission planning results become inapplicable. This problem poses challenges to the search, rescue, and military operations of unmanned system swarms.

[0003] Unmanned systems tend to execute tasks in swarms. Task planning is a top-level technique for achieving high performance in unmanned system swarms, mainly including task allocation and path planning. The task allocation problem requires the unmanned system swarm to generate task allocation and scheduling plans at a high logical level, while the path planning problem requires the unmanned system swarm to search for motion trajectories that satisfy kinematic constraints at a low physical level. Unfortunately, traditional task allocation algorithms do not consider the impact of environmental coupling on task allocation in unmanned system swarms. Since the result of task allocation in unmanned system swarms largely depends on the task environment, to avoid its influence, most algorithms need to be re-run in the new environment, which leads to low generalization and low efficiency.

[0004] Deep learning has recently achieved great success in the field of artificial intelligence, and it has also been applied to research related to unmanned system swarms. Deep learning has already achieved great success with Euclidean data. However, in an increasing number of real-world problems, data does not have a regular arrangement or order. It cannot be abstracted into Euclidean structure data, and therefore cannot be processed by traditional deep learning models. Graph neural networks, as a special type of deep neural network, are mainly used to process irregular graph structure data and can be used to process non-Euclidean structure data. Currently, graph neural networks are mainly applied in natural language processing, image classification, biochemistry, and other fields. For the task allocation problem of unmanned system swarms, the unmanned system and the task can be regarded as nodes. The allocation connection relationship between them can be regarded as edges. Graph neural networks can also be applied to the task allocation problem of unmanned system swarms. For deeper information features, differentiable pooled graph neural network models can handle them better. The input of differentiable pooled graph neural network models requires the design of special feature vectors for the target problem, which has a significant impact on the effectiveness of the algorithm; therefore, designing feature vectors is also a challenging task. Considering the characteristics of environmental coupling problems, no one has yet applied differentiable pooled graph neural network models considering environmental coupling problems to the transferable task planning of unmanned system swarms. Summary of the Invention

[0005] The purpose of this invention is to provide a transferable task planning method and system for unmanned system cluster environment coupling. It can predict the best task planning results through differentiable pooling graph neural network, thereby improving planning efficiency. Furthermore, when environmental coupling problems occur, the differentiable pooling graph neural network model can be adapted to specific environments through transfer and fine-tuning. That is, the trained differentiable pooling graph neural network model is used as the base model and training can continue, which can achieve good task planning results at a lower cost.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] This invention provides a portable task planning method under coupled unmanned system cluster environment, comprising:

[0008] S1: Obtain unmanned system cluster environment information; the environment information includes the number and location of unmanned systems, tasks, and obstacles in the unmanned system cluster environment;

[0009] S2: Based on the number of unmanned systems, the number of tasks, and the current iteration number, a greedy strategy is used to allocate tasks, resulting in a task allocation result set corresponding to the current iteration number. The task allocation result set includes several task allocation results. Each task allocation result is an allocation result in which at least one of the tasks is allocated to an unmanned system. The allocation objects include unmanned systems and tasks. The labeling category is either an unmanned system or a task.

[0010] S3: For each task allocation result in the task allocation result set, obtain the graph structure corresponding to the task allocation result; the nodes of the graph structure correspond to the allocation objects; obtain the feature matrix, relational adjacency matrix, and the actual reward value corresponding to the graph structure; the graph structure is a task allocation result graph represented using recursive graph syntax; the feature matrix includes path planning parameters, the positions and ranges of all obstacles, the position coordinates, initial angle, and label category of each allocation object; the relational adjacency matrix includes the allocation relationship between each unmanned system and each task; the actual reward value represents the length of the task planning path corresponding to the task allocation result;

[0011] S4: Using the feature matrix and relational adjacency matrix of the graph structure corresponding to the task allocation result as input, and the true reward value corresponding to the graph structure as label, train the differentiable pooling graph neural network model;

[0012] S5: Determine if the iteration stopping condition is met. If yes, stop the iteration and obtain the trained differentiable pooled graph neural network model. Determine the best task planning result based on the true reward value corresponding to the graph structure of all the task allocation results. If no, take the next iteration number as the current iteration number and return to S2 until the iteration stopping condition is met.

[0013] S6: Determine if an environmental coupling problem has occurred. If yes, use the trained differentiable pooling graph neural network model as the new environment after the environmental coupling problem occurs, and return to S1 for fine-tuning training. If no, complete the task planning. The environmental coupling problem refers to the change in the number and / or position of the unmanned system, task, and obstacles in the unmanned system cluster environment, causing the original task planning results to fail. Fine-tuning training refers to the adaptation process of the model in the new environment after migration.

[0014] Optionally, the method for obtaining the actual reward value corresponding to the graph structure is as follows:

[0015] Using the breadth-first search algorithm and the Dubins trajectory method, path planning is performed based on the position coordinates of all the unmanned systems in the graph structure, the position coordinates of all the tasks, and the allocation relationship between the unmanned systems and the tasks, to obtain the task planning path length corresponding to the graph structure;

[0016] The actual reward value corresponding to the task allocation result is calculated based on the length of the task planning path.

[0017] Optionally, the actual reward value is calculated as follows.

[0018]

[0019] Among them, R T f1 and f2 represent the actual reward value; f1 and f2 represent the calculation factors; and l represents the length of the task planning path.

[0020] Optionally, S4 specifically includes...

[0021] The feature matrix and relational adjacency matrix of the graph structure corresponding to the task allocation result are input into a differentiable pooled graph neural network model to obtain the predicted reward value corresponding to the graph structure.

[0022] Calculate the loss function value based on the predicted reward value and the actual reward value corresponding to the graph structure; train the differentiable pooled graph neural network model based on the loss function value.

[0023] Optionally, the loss function value is the mean squared loss function value.

[0024] Optionally, the optimal task planning result is determined based on the actual reward value corresponding to the graph structure of all the task allocation results, specifically including:

[0025] The task allocation result of the graph structure corresponding to the largest true reward value is determined as the optimal task planning result.

[0026] This invention also provides a portable task planning system coupled in an unmanned system cluster environment, comprising:

[0027] The environmental information acquisition module is used to acquire environmental information of the unmanned system cluster; the environmental information includes the number and location of unmanned systems, tasks, and obstacles in the unmanned system cluster environment;

[0028] The task allocation module is used to allocate tasks using a greedy strategy based on the number of unmanned systems, the number of tasks, and the current iteration number, to obtain a task allocation result set corresponding to the current iteration number; the task allocation result set includes several task allocation results; the task allocation result is an allocation result in which at least one of the tasks is allocated to an unmanned system; the allocation objects include unmanned systems and tasks; the labeling category is unmanned system or task.

[0029] The training data acquisition module is used to acquire, for each task allocation result in the task allocation result set, the graph structure corresponding to the task allocation result; the nodes of the graph structure correspond to the allocation objects; acquire the feature matrix, the relational adjacency matrix, and the true reward value corresponding to the graph structure; the graph structure is a task allocation result graph represented using recursive graph syntax; the feature matrix includes path planning parameters, the positions and ranges of all obstacles, the position coordinates, initial angle, and label category of each allocation object; the relational adjacency matrix includes the allocation relationship between each unmanned system and each task; the true reward value represents the task planning path length corresponding to the task allocation result;

[0030] The training module is used to train a differentiable pooling graph neural network model with the feature matrix and relational adjacency matrix of the graph structure corresponding to the task allocation result as input and the real reward value corresponding to the graph structure as the label.

[0031] The iteration module is used to determine whether the iteration stopping condition is met. If it is, the iteration stops and a trained differentiable pooled graph neural network model is obtained. The optimal task planning result is determined based on the true reward value corresponding to the graph structure of all the task allocation results. If not, the next iteration number is used as the current iteration number, and the process is returned to the "task allocation module" until the iteration stopping condition is met.

[0032] The environment coupling module is used to determine whether an environment coupling problem has occurred. If so, the trained differentiable pooling graph neural network model is used as the new environment coupling problem model, and the trained differentiable pooling graph neural network model is transferred to the environment after the environment coupling problem occurs. The module then returns to the "environment information acquisition module" for fine-tuning training. If not, the task planning is completed. The environment coupling problem refers to the change in the number and / or position of the unmanned system, task, and obstacles in the unmanned system cluster environment, which renders the original task planning results invalid. The fine-tuning training refers to the adaptation process of the transferred model in the new environment.

[0033] According to specific embodiments provided by the present invention, the following technical effects are disclosed: The present invention provides a transferable task planning method and system under the coupling of an unmanned system cluster environment. First, the number and location of unmanned systems, tasks, and obstacles in the unmanned system cluster environment are obtained. A greedy strategy is used for task allocation, resulting in a task allocation result graph set corresponding to the current iteration number. For each task allocation result graph in the set, the graph structure corresponding to the task allocation result is obtained. The feature matrix, relational adjacency matrix, and true reward value corresponding to the graph structure are obtained. Using the feature matrix and relational adjacency matrix of the graph structure corresponding to the task allocation result as input, and the true reward value corresponding to the graph structure as a label, a differentiable pooling graph neural network model is trained to obtain a trained differentiable pooling graph neural network model. When an environmental coupling problem occurs, the trained differentiable pooling graph neural network model is used as a differentiable pooling graph neural network model for fine-tuning training. This invention improves planning efficiency by predicting optimal task planning results using a differentiable pooling graph neural network (PGN). Furthermore, when environmental coupling problems occur, fine-tuning allows the PGN model to adapt to specific environments. Specifically, it uses a pre-trained PGN model as the base model and retrains it, achieving good task planning results at a relatively low cost. This invention designs a special feature matrix for effective transfer of the algorithm under environmental coupling. Effective transfer refers to the generalization ability of the differentiable pooling PGN model to changed environments. This invention is applicable to the transfer problem of unmanned swarm task planning under environmental coupling where the number and / or location of unmanned systems, tasks, and obstacles change. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of the portable task planning method under the coupling of an unmanned system cluster environment provided in Embodiment 1 of the present invention;

[0036] Figure 2 This is a schematic diagram of the overall training process of the differentiable pooled graph neural network model provided in Embodiment 1 of the present invention;

[0037] Figure 3 The initial information of the drone, mission, and obstacles provided in Embodiment 1 of the present invention;

[0038] Figure 4 This is the design form of the feature vector provided in Embodiment 1 of the present invention;

[0039] Figure 5 The design process of obstacle features provided in Embodiment 1 of the present invention;

[0040] Figure 6 The recursive graph syntax rules and generated task assignment sequences provided in Embodiment 1 of the present invention;

[0041] Figure 7 This is a diagram showing the optimal task planning results for the original environment provided in Embodiment 1 of the present invention.

[0042] Figure 8 This is a schematic diagram of environmental information after environmental coupling occurs, provided in Embodiment 1 of the present invention;

[0043] Figure 9 This is a diagram showing the task planning results after environmental coupling occurs, as provided in Embodiment 1 of the present invention.

[0044] Figure 10 This is a block diagram of a portable task planning system under the coupling of an unmanned system cluster environment provided in Embodiment 2 of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] The purpose of this invention is to provide a transferable task planning method and system for unmanned system cluster environments with coupling. By predicting optimal task planning results using a differentiable pooling graph neural network (PGN), planning efficiency is improved. Furthermore, when environmental coupling problems occur, fine-tuning is performed to adapt the differentiable pooling PGN model to specific environments. Specifically, the trained PGN model is used as the base model, and retraining is performed, achieving good task planning results at a relatively low cost. This invention uses fine-tuning to adapt the differentiable pooling PGN model to specific environments. Then, the model parameters can be transferred to similar environments. Therefore, good results can be obtained at a relatively low cost.

[0047] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] Example 1:

[0049] like Figure 1 and Figure 2As shown, this embodiment provides a portable task planning method under the coupled environment of an unmanned system cluster, including:

[0050] S1: Obtain unmanned system cluster environment information; the environment information includes the number and location of unmanned systems, tasks, and obstacles in the unmanned system cluster environment.

[0051] S2: Based on the number of unmanned systems, the number of tasks, and the current iteration number, a greedy strategy is used to allocate tasks, resulting in a task allocation result set corresponding to the current iteration number. The task allocation result set includes several task allocation results. Each task allocation result is an allocation result in which at least one of the tasks is allocated to an unmanned system. The allocation objects include unmanned systems and tasks. The labeling category is either an unmanned system or a task.

[0052] S3: For each task allocation result in the task allocation result set, obtain the graph structure corresponding to the task allocation result; the nodes of the graph structure correspond to the allocation objects; obtain the feature matrix, relational adjacency matrix, and the actual reward value corresponding to the graph structure; the graph structure is a task allocation result graph represented using recursive graph syntax; the feature matrix includes path planning parameters, the positions and ranges of all obstacles, the position coordinates, initial angle, and label category of each allocation object; the relational adjacency matrix includes the allocation relationship between each unmanned system and each task; the actual reward value represents the task planning path length corresponding to the task allocation result.

[0053] S4: Using the feature matrix and relational adjacency matrix of the graph structure corresponding to the task allocation result as input, and the true reward value corresponding to the graph structure as label, train the differentiable pooling graph neural network model.

[0054] S5: Determine if the iteration stopping condition is met. If yes, stop the iteration and obtain the trained differentiable pooled graph neural network model. Determine the optimal task planning result based on the true reward value corresponding to the graph structure of all the task allocation results. If no, take the next iteration number as the current iteration number and return to S2 until the iteration stopping condition is met.

[0055] S6: Determine if an environmental coupling problem has occurred. If yes, use the trained differentiable pooling graph neural network model as the new environment after the environmental coupling problem occurs, and return to S1 for fine-tuning training. If no, complete the task planning. The environmental coupling problem refers to the change in the number and / or position of the unmanned system, task, and obstacles in the unmanned system cluster environment, causing the original task planning results to fail. Fine-tuning training refers to the adaptation process of the model in the new environment after migration.

[0056] This embodiment is applied to unmanned system clusters and can be specifically divided into the following steps:

[0057] I. Based on the characteristics of the UAV swarm environment, a special feature vector is designed for the environmental coupling problem caused by changes in the unmanned system (in this embodiment, the unmanned system is a UAV), the target task, and scene obstacles. The solution capability of the differentiable pooled graph neural network model for this problem is optimized.

[0058] Initialize drones, tasks, and obstacles; construct an initial graph; initial information for drones, tasks, and obstacles in the designed drone swarm environment is as follows: Figure 3 As shown in the diagram, u represents a drone, u1, u2, and u3 represent different drones, t represents a mission, and t1, t2...t10 represent different missions. Coordinates represent the positions of the drones and missions (the coordinate system is an axis with the lower left corner as the origin, e.g., [example missing]). Black bars represent the obstruction range of obstacles. Figure 3 The three black pillars in the middle represent three obstacles.

[0059] Specifically, this embodiment designs feature vectors for environmental coupling problems. The feature vectors mainly include features of the UAV, the mission, and obstacles. The features of the UAV and the mission consider the impact of changes in their number and location on mission allocation, while the features of obstacles consider the impact of their obstruction range on mission allocation. The design form of the feature vectors is as follows: Figure 4 As shown, the feature vector of a drone or a mission is defined as x = (M x ,P x G x O x M x This indicates the coordinates and initial angle of the assigned object (drone or mission). Where (x, y) are the position coordinates of the UAV or mission. This is the initial angle for the drone or mission; the initial angle can be set according to the user's preferences. x To represent some fixed parameters in path planning in, Path key points (path key points are the outward extension points of obstacles on this path; the outward extension points are the vertices of the obstacles within the path planning area after extending a set distance outward from the obstacles, such as...) Figure 3 As shown, there are 3 obstacles and 6 key points on the path. The average x-coordinate is... This represents the average y-coordinate of the key points on the path. G represents the initial average angle of the path's key points, n is the number of path key points, l is the path length of the current path planning after Dubin curve optimization (used to calculate the actual reward value), θ1 is the clockwise rotation angle (in radians) of the current Dubins path, θ2 is the counterclockwise rotation angle of the current Dubins path, and r is the Dubins turning radius. x The one-hot encoding indicates the category of the label. The encoding results differ for drones and tasks: for example, if the assigned object is a drone, its one-hot encoding is "010", and if the assigned object is a task, its one-hot encoding is "001". x It is a pooled representation of the obstruction range of the obstacle. In this embodiment, O x It is a 12-dimensional vector, but can be set to other dimensions depending on the specific environment and size. x =(γ1,γ2,γ3,γ4,γ5,γ6,γ7,γ8,γ9,γ 10 ,γ 11 ,γ 12 ). Figure 5 This demonstrates the formation process of obstacle features. First, the original obstacle information map is transformed into a 17×13 0-1 matrix, where 1 represents the obstruction range and 0 represents other areas. Then, the 0-1 matrix is ​​averaged using pooling to obtain a pooling matrix. Finally, the pooling matrix is ​​flattened into a one-dimensional vector matrix, from γ1 to γ... 12 The values ​​are (0.40, 0.40, 0, 0.16, 0.16, 0, 0.40, 0, 0.40, 0, 0.40, 0).

[0060] Second, by using recursive graph grammar, the task allocation process and results of UAV swarms can be represented by an irregular graph structure, from which the graph input of a differentiable pooled graph neural network model can be obtained, thereby transforming such problems into solvable objectives of a differentiable pooled graph neural network model.

[0061] S2 specifically includes:

[0062] In the process of generating a task assignment result using recursive graph grammar, the probability ∈ is calculated using a greedy strategy based on the current iteration number. n The process determines whether the next task in the current task assignment graph is randomly assigned or predicted by a graph neural network, ultimately resulting in a completed task assignment graph. The attributes of the nodes and edges in this graph contain information about the task assignment results. Multiple intermediate task assignment graphs are also generated, each representing a task assignment where at least one task has been assigned to the unmanned system. The final completed task assignment graph represents a task assignment where all tasks have been randomly assigned to the unmanned system. This process is generated using a recursive graph syntax, and the syntax rules and the generated task assignment process are as follows: Figure 6 As shown. Probability ∈ n The calculation formula is as follows:

[0063]

[0064] in, This is the preset number of iterations, where n is the current nth iteration number, ∈ start ,∈ end These are the start probability and end probability of the greedy strategy, respectively. In this embodiment, ∈ start =1,∈ end =0.1, In this embodiment, each time it is determined whether a task is randomly assigned or predicted by a graph neural network, a predetermined probability value is generated. This predetermined probability value ranges from 0 to 1. When the probability corresponding to the current iteration number is calculated, ∈ n When, ∈ n Compare with the set probability value, if ∈ n If the probability is greater than a set probability value, the task is randomly assigned to any drone. When determining the adjacency matrix of the task assignment result graph, the matrix element corresponding to the task and the matched drone is "1", indicating that the task is assigned to this drone. If the probability is not greater than a set probability value, the task is predicted and assigned by the graph neural network. Therefore, as the greedy probability decreases with the increase of iterations, more and more tasks will be predicted and assigned by the graph neural network (because the graph neural network is trained better and better with the increase of iterations), instead of being randomly assigned. When determining the adjacency matrix of the task assignment result graph, all elements corresponding to unassigned tasks are "0".

[0065] Third, by training the network model through the mapping relationship between graph input and reward value, excellent task allocation results can be predicted. When encountering environmental coupling problems, the parameters can be inherited from the previous training and then fine-tuned to adapt to the changed environment through parameter transfer, without the need to retrain the network, which has adaptability and high efficiency.

[0066] For each task allocation result, the task allocation result is represented using recursive graph syntax to obtain the graph structure corresponding to the task allocation result. The graph nodes of the graph structure correspond to the allocation objects, which are unmanned systems or tasks. The edges of the task allocation result graph represent the allocation relationship between the allocation objects corresponding to the two graph nodes connected by the edge.

[0067] The graph input I = (A, X) of the differentiable pooling graph neural network model is obtained from the graph structure corresponding to the final task allocation result. A is a relational adjacency matrix representing the relationship between unmanned systems and the tasks they execute. In relational adjacency matrix A, the rows represent unmanned systems and the columns represent tasks. If the m-th unmanned system matches the n-th task, that is, the m-th unmanned system executes the n-th task, then the value of the mn-th element (the element in the m-th row and n-th column) in relational adjacency matrix A is "1". If the m-th unmanned system does not match the n-th task, then the value of the mn-th element (the element in the m-th row and n-th column) in relational adjacency matrix A is "0". It is a feature matrix composed of the feature vectors of unmanned systems and the feature vectors of tasks. M and N are the number of unmanned systems and tasks, respectively. Then M+N is the total number of unmanned systems and tasks.

[0068] The method for obtaining the actual reward value corresponding to the graph structure is as follows:

[0069] Using the breadth-first search algorithm and the Dubins trajectory method, path planning is performed based on the position coordinates of all unmanned systems in the graph structure, the position coordinates of all tasks, and the allocation relationship between the unmanned systems and the tasks to obtain the task planning path length corresponding to the graph structure; the actual reward value corresponding to the task allocation result is calculated based on the task planning path length.

[0070] Specifically, the breadth-first search algorithm performs a path search on the map based on the position coordinates of all unmanned systems, all tasks, all obstacles, and the assignment relationships between all unmanned systems and tasks, to obtain the shortest path. Specifically, using a queue data structure, for example, starting from a specific UAV and defining it as the starting point, it sequentially traverses all unvisited adjacent points on the map, repeating the starting point traversal method for each adjacent point. During the traversal, if the destination (the next task position assigned to the UAV) is reached, this path is recorded. This process continues until all points on the map have been visited, resulting in multiple paths to the destination. The task planning path length is the length of the shortest path among all paths to the destination. The shortest path obtained by the breadth-first search algorithm is a polyline path. The Dubins curve optimizes the polyline path based on the shortest path nodes to obtain the shortest path that satisfies kinematic constraints. Specifically, the process involves treating the polyline path as a path composed of straight line segments and circular arc segments. The starting and ending points of each straight line segment are determined, generating all path nodes for that segment. Similarly, the center of each circular arc segment is determined, generating all path nodes for that arc segment. The actual reward value is calculated as follows.

[0071]

[0072] Among them, RT f1 and f2 represent the actual reward value (label); l represents the calculation factor and the task planning path length (the total length of the path planning for all tasks assigned to all drones).

[0073] S4 specifically includes...

[0074] The feature matrix and relational adjacency matrix of the graph structure corresponding to the task allocation result are input into a differentiable pooled graph neural network model to obtain the predicted reward value corresponding to the graph structure.

[0075] Calculate the loss function value based on the predicted reward value and the actual reward value corresponding to the graph structure; train the differentiable pooled graph neural network model based on the loss function value.

[0076] Compare I with the corresponding real reward value label R T Package the data into the training set, which will have the following format: I1, I2... represent the task allocation results in the task allocation result set. Let I1, I2, ... be the labels for the actual reward values. Set the training set size to 5000; if it exceeds this size, replace historical data. Set f1 = 100 and f2 = -0.05.

[0077] The training set is used to sequentially extract small batches of data from the training set according to the set training epochs to train a differentiable pooling graph neural network model. The differentiable pooling graph neural network model predicts the reward value R of the input I. R and R T Through mean square loss function Calculate the loss L for this training round, and update the parameters using the Adam optimizer to complete one training round. Set each training round to 25 iterations, and each training round extracts batches of data of size 32 from the training set.

[0078] If the preset iteration count of 500 has been reached, proceed to the end of this training iteration and save the model parameters. Otherwise, increment the current iteration count by 1 and return to step S2. The optimal task planning result obtained in this iteration is as follows: Figure 7 As shown, it was obtained in the 18th iteration.

[0079] The optimal task planning result is determined based on the actual reward values ​​corresponding to the graph structure of all the task allocation results, specifically including:

[0080] The task allocation result of the graph structure corresponding to the largest true reward value is determined as the optimal task planning result.

[0081] Specifically, because each iteration generates many task allocation result graphs, only the graph structure corresponding to the final task allocation result after all tasks have been allocated in this iteration is used to calculate the true reward value. Only that graph structure has completely allocated all tasks. The graph structures corresponding to the intermediate task allocation results use the true reward value of the graph structure corresponding to the final task allocation result as labels for training. That is: the task allocation result graph set generated in one iteration is: {I1,I2,I3,…,I...} d}, I d To assign the complete final task allocation diagram. I1->I d It is a task allocation chain. d The real label is The content added to the dataset in this iteration is: Therefore, in this embodiment, the optimal task planning result is determined only from the actual reward values ​​corresponding to the graph structures of all final task allocation results.

[0082] If an environmental coupling problem occurs, it will affect the original mission planning. The impact will be that the number and / or location of unmanned systems, missions, and obstacles will change in the unmanned system cluster environment.

[0083] First, determine the changed environmental information. In this embodiment, the changed environmental information is as follows: Figure 8 As shown, the slashed area represents the designated bombing zone, where passage is prohibited. Therefore, missions t5 and t7 cannot be executed. At this point, the trained model parameters are transferred to a new model, specifically the trained differentiable pooling graph neural network model. The process proceeds to step S1, where the trained differentiable pooling graph neural network model is fine-tuned to obtain and save the mission planning results after environmental coupling. This completes the mission planning, and the final mission planning graph is shown below. Figure 9 As shown, the optimal task planning result is obtained in the first iteration after parameter transfer. However, without transferring and retraining the network, the optimal task planning result is obtained in the 14th iteration. Compared with the prior art, this embodiment uses fine-tuning, which is more efficient and adaptable.

[0084] Example 2:

[0085] To execute the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a portable task planning system coupled in an unmanned system cluster environment is provided below, such as... Figure 10 As shown, the portable task planning system coupled in an unmanned system cluster environment includes:

[0086] The environmental information acquisition module T1 is used to acquire environmental information of the unmanned system cluster; the environmental information includes the number and location of unmanned systems, tasks, and obstacles in the unmanned system cluster environment.

[0087] The task allocation module T2 is used to allocate tasks using a greedy strategy based on the number of unmanned systems, the number of tasks, and the current iteration number, to obtain a task allocation result set corresponding to the current iteration number; the task allocation result set includes several task allocation results; the task allocation result is an allocation result in which at least one of the tasks is allocated to an unmanned system; the allocation object includes unmanned systems and tasks; the labeling category is unmanned system or task.

[0088] The training data acquisition module T3 is used to acquire the graph structure corresponding to each task allocation result in the task allocation result set; the nodes of the graph structure correspond to the allocation objects; acquire the feature matrix, relational adjacency matrix, and the true reward value corresponding to the graph structure; the graph structure is a task allocation result graph represented using recursive graph syntax; the feature matrix includes path planning parameters, the positions and ranges of all obstacles, the position coordinates, initial angle, and label category of each allocation object; the relational adjacency matrix includes the allocation relationship between each unmanned system and each task; the true reward value represents the task planning path length corresponding to the task allocation result.

[0089] Training module T4 is used to train a differentiable pooling graph neural network model with the feature matrix and relational adjacency matrix of the graph structure corresponding to the task allocation result as input and the real reward value corresponding to the graph structure as label.

[0090] The iteration module T5 is used to determine whether the iteration stopping condition is met. If so, the iteration stops and a trained differentiable pooled graph neural network model is obtained. The optimal task planning result is determined based on the true reward value corresponding to the graph structure of all the task allocation results. If not, the next iteration number is used as the current iteration number, and the process is returned to the "task allocation module" until the iteration stopping condition is met.

[0091] The environment coupling module T6 is used to determine whether an environment coupling problem has occurred. If so, the trained differentiable pooling graph neural network model is used as the new environment coupling problem, and the trained differentiable pooling graph neural network model is transferred to the environment after the environment coupling problem occurs. The module then returns to the "environment information acquisition module" for fine-tuning training. If not, the task planning is completed. The environment coupling problem refers to the change in the number and / or position of unmanned systems, tasks, and obstacles in the unmanned system cluster environment, which renders the original task planning results invalid. The fine-tuning training refers to the adaptation process of the transferred model in the new environment.

[0092] The present invention has the following beneficial effects:

[0093] 1) This invention proposes a transferable task planning method for unmanned system clusters in coupled environments based on differentiable pooled graph neural networks. Using recursive graph grammar, the task allocation process of the unmanned system cluster can be abstracted into an irregular graph structure, making task planning the solution objective of the differentiable pooled graph neural network model. The representation of an undirected acyclic graph structure also makes the task allocation process and results of the unmanned system cluster clear and easy to understand.

[0094] 2) Traditional task planning algorithms do not adequately consider environmental coupling issues, resulting in poor adaptability and low efficiency. This invention enables efficient transfer of task planning algorithms and effectively solves the environmental coupling problem.

[0095] 3) This invention designs a special feature vector (feature matrix of the graph) to better solve the environmental coupling problem.

[0096] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0097] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A portable task planning method under coupled unmanned system cluster environment, characterized in that, The method includes: S1: Obtain unmanned system cluster environment information; the environment information includes the number and location of unmanned systems, tasks, and obstacles in the unmanned system cluster environment; S2: Based on the number of unmanned systems, the number of tasks, and the current iteration number, a greedy strategy is used to allocate tasks to obtain a task allocation result set corresponding to the current iteration number; the task allocation result set includes several task allocation results; the task allocation result is an allocation result in which at least one of the tasks is allocated to an unmanned system. S3: For each task allocation result in the task allocation result set, obtain the graph structure corresponding to the task allocation result; the nodes of the graph structure correspond to the allocation objects; obtain the feature matrix, relational adjacency matrix, and the true reward value corresponding to the graph structure; the graph structure is a task allocation result graph represented using recursive graph syntax; the feature matrix includes path planning parameters, the positions and ranges of all obstacles, the position coordinates of each allocation object, the initial angle, and the label category; the relational adjacency matrix includes the allocation relationship between each unmanned system and each task; the true reward value represents the task planning path length corresponding to the task allocation result; the allocation objects include unmanned systems and tasks; the label category is unmanned system or task; the method for obtaining the true reward value corresponding to the graph structure is as follows: using the breadth-first search algorithm and the Dubins trajectory method, path planning is performed based on the position coordinates of all unmanned systems, the position coordinates of all tasks, and the allocation relationship between the unmanned systems and tasks in the graph structure to obtain the task planning path length corresponding to the graph structure; the true reward value corresponding to the task allocation result is calculated based on the task planning path length. S4: Using the feature matrix and relational adjacency matrix of the graph structure corresponding to the task allocation result as input, and the true reward value corresponding to the graph structure as label, train the differentiable pooling graph neural network model; S5: Determine if the iteration stopping condition is met. If yes, stop the iteration and obtain the trained differentiable pooled graph neural network model. Determine the best task planning result based on the true reward value corresponding to the graph structure of all the task allocation results. If no, take the next iteration number as the current iteration number and return to S2 until the iteration stopping condition is met. S6: Determine whether an environmental coupling problem has occurred. If so, migrate the trained differentiable pooling graph neural network model to the environment after the environmental coupling problem occurs, and return to S1 for fine-tuning training. If not, complete the task planning. The environmental coupling problem is that the number and / or position of the unmanned system, task, and obstacle changes in the unmanned system cluster environment, and the original task planning result becomes invalid. The fine-tuning training refers to the adaptation process of the model in the new environment after migration.

2. The portable task planning method under the coupled environment of an unmanned system cluster as described in claim 1, characterized in that, The actual reward value is calculated as follows: ； in, Indicates the actual reward value; and Indicates the calculation factor. Indicates the length of the task planning path.

3. The portable task planning method under the coupled environment of an unmanned system cluster according to claim 1, characterized in that, S4 specifically includes: The feature matrix and relational adjacency matrix of the graph structure corresponding to the task allocation result are input into a differentiable pooled graph neural network model to obtain the predicted reward value corresponding to the graph structure. Calculate the loss function value based on the predicted reward value and the actual reward value corresponding to the graph structure; train the differentiable pooled graph neural network model based on the loss function value.

4. The portable task planning method under the coupled environment of an unmanned system cluster according to claim 3, characterized in that, The loss function value is the mean squared loss function value.

5. The portable task planning method under the coupled environment of an unmanned system cluster according to claim 1, characterized in that, The optimal task planning result is determined based on the actual reward values ​​corresponding to the graph structure of all the task allocation results, specifically including: The task allocation result of the graph structure corresponding to the largest true reward value is determined as the optimal task planning result.

6. A portable task planning system coupled in an unmanned system cluster environment, characterized in that, The system includes: The environmental information acquisition module is used to acquire environmental information of the unmanned system cluster; the environmental information includes the number and location of unmanned systems, tasks, and obstacles in the unmanned system cluster environment; The task allocation module is used to allocate tasks using a greedy strategy based on the number of unmanned systems, the number of tasks, and the current iteration number, to obtain a task allocation result set corresponding to the current iteration number; the task allocation result set includes several task allocation results; the task allocation result is an allocation result in which at least one of the tasks is allocated to an unmanned system. The training data acquisition module is used to acquire the graph structure corresponding to each task allocation result in the task allocation result set; the nodes of the graph structure correspond to the allocation objects; acquire the feature matrix, relational adjacency matrix, and true reward value corresponding to the graph structure; the graph structure is a task allocation result graph represented using recursive graph syntax; the feature matrix includes path planning parameters, the positions and ranges of all obstacles, the position coordinates, initial angle, and label category of each allocation object; the relational adjacency matrix includes the allocation relationship between each unmanned system and each task; the true reward value represents the task planning path length corresponding to the task allocation result; the allocation objects include unmanned systems and tasks; the label category is unmanned system or task; the method for acquiring the true reward value corresponding to the graph structure is as follows: using the breadth-first search algorithm and the Dubins trajectory method, path planning is performed based on the position coordinates of all unmanned systems, the position coordinates of all tasks, and the allocation relationship between the unmanned systems and tasks in the graph structure to obtain the task planning path length corresponding to the graph structure; the true reward value corresponding to the task allocation result is calculated based on the task planning path length. The training module is used to train a differentiable pooling graph neural network model with the feature matrix and relational adjacency matrix of the graph structure corresponding to the task allocation result as input and the real reward value corresponding to the graph structure as the label. The iteration module is used to determine whether the iteration stopping condition is met. If it is, the iteration stops and a trained differentiable pooled graph neural network model is obtained. The optimal task planning result is determined based on the true reward value corresponding to the graph structure of all the task allocation results. If not, the next iteration number is used as the current iteration number, and the process is returned to the "task allocation module" until the iteration stopping condition is met. The environment coupling module is used to determine whether an environment coupling problem has occurred. If so, the trained differentiable pooled graph neural network model is transferred to the environment after the environment coupling problem occurs, and the system is returned to the "environment information acquisition module" for fine-tuning training. If not, the task planning is completed. The environment coupling problem is that the number and / or position of the unmanned system, task, and obstacles change in the unmanned system cluster environment, and the original task planning results become invalid. The fine-tuning training refers to the adaptation process of the model in the new environment after the model is transferred.

Images