Unmanned ship formation method based on heterogeneous graph and dimension adaptive network
By adopting heterogeneous graphs and dimensional adaptive networks in the control of unmanned boat fleets, the problems of low learning efficiency, slow convergence speed and low multi-task coordination efficiency in the existing technology are solved, and a more robust, accurate and efficient unmanned boat fleet generation is achieved.
Patent Information
- Application Number
- CN202510086256.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-20
- Publication Date
- 2025-05-23
AI Technical Summary
In the prior art, the unmanned boat formation control method has problems such as low learning efficiency, slow convergence speed and low coordination efficiency in handling multitasking, especially in complex marine environments, it is difficult to achieve rapid convergence and efficient formation.
The unmanned boat formation method based on heterogeneous graph and dimensional adaptive network is adopted to express multiple node types and their relationships through heterogeneous graphs. The input features are processed using the dimensional adaptive module to make their dimensions consistent, and trained in combination with the reinforcement learning network to optimize formation generation.
It improves the robustness and accuracy of unmanned boat formation generation, solves the problem of dimensional inconsistency caused by changes in USV counts in the training and prediction stages, and enhances the stability and efficiency of formation generation.
Smart Images
Figure CN120029274A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of unmanned boat formation control, and in particular to an unmanned boat formation method based on a heterogeneous graph and a dimensional adaptive network. Background Art
[0002] As an emerging ocean exploration platform, unmanned underwater vehicles (USVs) have the advantages of low cost, high intelligence and flexibility, and have been widely used in underwater exploration, marine environment monitoring, maritime rescue and other tasks. With the development of technology, the operating capabilities and application scenarios of a single unmanned vehicle are gradually limited, especially in complex marine environments. The operating efficiency, robustness and fault tolerance of a single boat cannot meet the needs of multi-tasks and complex scenarios. Therefore, the USV group system (USV group formation) came into being, which has stronger collaboration and efficiency.
[0003] USV group formation technology, as one of the core research directions of unmanned watercraft technology, mainly includes key tasks such as formation generation, formation transformation and navigation. Formation generation is the basis of USV group cooperation. Existing USV formation control methods include centralized control and distributed control. Centralized control relies on the flagship and has poor robustness, while distributed control has better fault tolerance but higher communication requirements.
[0004] In order to improve the efficiency and robustness of USV formation generation, bionic optimization algorithms and multi-task allocation strategies have become important directions in formation generation research in recent years. For example, bionic algorithms such as ant colony optimization and particle swarm optimization have been widely used in USV formation generation, aiming to optimize formation morphology and path planning by simulating group behavior in nature. However, these traditional optimization algorithms often face problems such as slow algorithm convergence and low computational efficiency when dealing with complex marine environments, especially in dynamically changing marine environments, and cannot fully meet the needs of real-time tasks.
[0005] In this context, reinforcement learning, as an adaptive learning algorithm, has been widely used in path planning and task assignment of USVs. Reinforcement learning can autonomously learn and make decisions in complex environments, and gradually optimize decision-making strategies based on system feedback, with good adaptability and decision-making capabilities. However, existing reinforcement learning methods still face certain challenges in complex marine environments, such as low learning efficiency, slow convergence speed, and how to effectively handle multi-task collaboration. In addition, the network also has problems such as fixed input dimensions and inability to change after training. Therefore, improving existing reinforcement learning algorithms so that they can achieve rapid convergence and efficient formation in marine environments is still a technical challenge. Summary of the invention
[0006] In view of the shortcomings of the prior art, the present invention provides an unmanned boat formation method based on heterogeneous graphs and dimensional adaptive networks to solve the technical problems of low learning efficiency, slow convergence speed, and low efficiency in processing multi-task collaboration in the prior art.
[0007] The present invention provides an unmanned boat formation method based on a heterogeneous graph and a dimension adaptive network, comprising the following steps:
[0008] Step 1: Set the feature standard dimension;
[0009] Step 2: Randomly set the number of unmanned boats and formation information in the unmanned boat formation, and drive the unmanned boat formation to move towards the target;
[0010] Step 3: Obtain the current position information of the unmanned boat formation, and obtain the high-dimensional feature information of the nodes representing the unmanned boats in the current position information through the heterogeneous graph;
[0011] Step 4: Perform dimension adaptive processing on the high-dimensional feature information of the current position information, so that the dimension of the high-dimensional feature information in the current position information is consistent with the feature standard dimension;
[0012] Step 5: Use the high-dimensional feature information in the current position information after dimensionality unification to train the reinforcement learning network of the unmanned boat formation and obtain the next action.
[0013] Repeat steps 3-5 until the current unmanned boat formation is completed;
[0014] Repeat steps 2-5 until the reinforcement learning network training is completed;
[0015] Step 6: Use the trained reinforcement learning network to control the unmanned boat formation.
[0016] Furthermore, in step 3, the output of the heterogeneous graph is:
[0017]
[0018] Where N i is the input feature of the i-th node; is the high-dimensional feature information of the i-th node; σ represents the activation function; ⊕ is the concatenation operation; Represents the result on the edge between the i-th and j-th nodes at time t.
[0019] Furthermore, the formula for obtaining the result on the edge of the two nodes at time t is:
[0020]
[0021] Where N iis the input feature of the i-th node; N j is the input feature of the jth node.
[0022] Furthermore, in step 4, the dimension adaptive processing method includes: unifying the dimensions through a dimension adaptive module, wherein the dimension adaptive module includes two multi-layer perceptron layers and a convolutional layer connected in sequence, and each multi-layer perceptron layer includes several multi-layer perceptrons.
[0023] Furthermore, the specific method of the dimension adaptation module to unify the dimensions is:
[0024] The high-dimensional feature information of the current position information of all unmanned boats first passes through the first multi-layer perceptron layer to internally fuse the high-dimensional feature information of each unmanned boat. The fused data then passes through the second multi-layer perceptron layer to fuse the same high-dimensional feature information between the unmanned boats, and finally passes through the convolution layer to reduce the dimension to the standard feature dimension.
[0025] Beneficial effects of the present invention:
[0026] The unmanned vehicle (USV) formation generation method provided by the present invention based on heterogeneous graph, dimensionality adaptive algorithm and reinforcement learning network construction shows significant advantages in multi-type node information processing, feature fusion and multi-objective task execution.
[0027] The present invention expresses the various node types and their relationships in the USV formation by constructing a heterogeneous graph. It can assign different weights to each edge type through heterogeneous edges, thereby accurately modeling the interactions and influences between different nodes and improving the robustness and accuracy of formation generation.
[0028] The present invention performs horizontal and vertical fusion processing on the input features of different numbers of unmanned boats through a dimensionality adaptation module, thereby ensuring the dimensional consistency of the input features at different stages. This not only solves the dimensional inconsistency problem caused by changes in the number of USVs in the training and prediction stages, but also improves the stability and processability of the input features, thereby ensuring the stability and efficiency of formation generation. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:
[0030] Figure 1 is a flow chart of a specific embodiment of the present invention;
[0031] Figure 2 is a schematic diagram of the HG-PPO actor network structure in a specific embodiment of the present invention;
[0032] Figure 3 is a schematic diagram of the HG-PPO reviewer network structure in a specific embodiment of the present invention;
[0033] Figure 4 It is a schematic diagram of the convergence effect of the method of a specific embodiment of the present invention;
[0034] Figure 5 is a schematic diagram of the overall processing process of a heterogeneous graph in a specific embodiment of the present invention;
[0035] Figure 6 is a schematic diagram of a node encoding process of a heterogeneous graph in a specific embodiment of the present invention;
[0036] Figure 7 is a schematic diagram of a heterogeneous edge aggregation process of a heterogeneous graph in a specific embodiment of the present invention;
[0037] Figure 8 is a schematic diagram of a node decoding process of a heterogeneous graph in a specific embodiment of the present invention;
[0038] Fig. 9 It is a schematic diagram of a dimension adaptation module in a specific embodiment of the present invention. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0040] The present invention is further illustrated below in conjunction with specific embodiments. Those skilled in the art should understand that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention, and modifications to various equivalent forms of the present invention fall within the scope defined by the appended claims of this application.
[0041] like Figure 1 As shown, the present invention provides an unmanned boat formation method based on heterogeneous graph and dimension adaptive network, comprising the following steps:
[0042] Step 1: Setting the feature standard dimension; the setting of the standard feature dimension can be determined as a moderate dimension according to the range of variation of the number of unmanned boats in the unmanned boat formation that needs to be adapted;
[0043] Step 2: Randomly set the number of unmanned boats and formation information in the unmanned boat formation, and drive the unmanned boat formation to move towards the target;
[0044] Step 3: Obtain the current position information of the unmanned boat formation, and obtain the high-dimensional feature information of the nodes representing the unmanned boats in the current position information through the heterogeneous graph;
[0045] like Figure 5-8 As shown in the figure, the heterogeneous graph consists of multiple types of nodes, such as USV, target, obstacle, and multiple types of edges, such as USV and target, USV and obstacle. When there are n unmanned boats and m obstacles, the total number of nodes in each layer is n*2+m. Node data includes feature information such as position, speed, acceleration, etc. Edge types include: USV-target edge, USV-obstacle edge, USV-neighbor USV edge, and self-loop edge, etc. Each edge type is represented by a different value. An example of an adjacency matrix is as follows:
[0046]
[0047] The heterogeneous graph goes through four steps at each level, namely node encoding, retaining encoded information, edge aggregation, and node encoding. The heterogeneous graph connects the data in the nodes to encode them, and then repeats four times to correspond to four different types of edges. After that, it retains the same type of encoded information and makes other edges 0, so that each edge has its own encoded information:
[0048]
[0049] Where N i is the input feature of the i-th node; N j is the input feature of the jth node;
[0050] The edge aggregation operation sums the edge information of each node and puts it into the activation function. Finally, the convergence information is concatenated with the node information to form a residual structure and decode it, and the final result is:
[0051]
[0052] Where N i is the input feature of the i-th node; is the high-dimensional feature information of the i-th node; σ represents the activation function; ⊕ is the concatenation operation; Represents the result on the edge between the i-th and j-th nodes at time t;
[0053] Step 4: Perform dimension adaptive processing on the high-dimensional feature information of the current position information, so that the dimension of the high-dimensional feature information in the current position information is consistent with the feature standard dimension;
[0054] The dimension adaptive processing method comprises: unifying the dimension through a dimension adaptive module, wherein the dimension adaptive module comprises two multi-layer perceptron layers and a convolution layer connected in sequence, each multi-layer perceptron layer comprises a plurality of multi-layer perceptrons,
[0055] The specific process includes:
[0056] The high-dimensional feature information of the current position information of all unmanned boats first passes through the first multi-layer perceptron layer to internally fuse the high-dimensional feature information of each unmanned boat. The fused data then passes through the second multi-layer perceptron layer to fuse the same high-dimensional feature information between the unmanned boats, and finally passes through the convolution layer to reduce the dimension to the feature standard dimension (MLP).
[0057] In order to deal with the problem of inconsistent input feature dimensions, the present invention designs a dimension adaptation module (DAB). Fig. 9 As shown in the figure, assuming that there are n unmanned boats in the initial state, each with m-dimensional features, DAB first performs horizontal MLP on the m-dimensional input features of each unmanned boat, so that they exchange features within each other, and the feature dimension is n*k at this time; then, vertical MLP is performed to fuse all unmanned boat features, and feature exchange is performed between unmanned boats, and the feature dimension remains unchanged; finally, the convolutional layer of MLP dimension reduction is used to reduce the dimension and obtain a fixed-dimensional feature representation, and the feature dimension is 1*k at this time. After passing through the DAB module, the input of the network is no longer related to the number of unmanned boats, thereby achieving dimensional adaptation. DAB is particularly suitable for situations where the number of USVs in the training phase and the prediction phase is different, and can ensure the stability of the algorithm.
[0058] Step 5: Use the high-dimensional feature information in the current position information after dimensionality unification to train the reinforcement learning network of the unmanned boat formation and obtain the next action.
[0059] Repeat steps 3-5 until the current unmanned boat formation is completed;
[0060] Repeat steps 2-5 until the reinforcement learning network training is completed;
[0061] Step 6: Use the trained reinforcement learning network to control the unmanned boat formation.
[0062] In the whole model, the actor network is as follows Figure 2 As shown in Figure 1, the USV action decision is output based on the local state, such as speed, heading, path planning, etc.; the commentator network is as follows Figure 3 As shown in Figure 1, an evaluation value is output based on the global state and the joint actions of all USVs to guide the optimization of the decision.
[0063] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A method for unmanned boat formation based on heterogeneous graph and dimension adaptive network, characterized in that: The steps include: Step 1: Set the feature standard dimension; Step 2: Randomly set the number of unmanned boats and formation information in the unmanned boat formation, and drive the unmanned boat formation to move towards the target; Step 3: Obtain the current position information of the unmanned boat formation, and obtain the high-dimensional feature information of the nodes representing the unmanned boats in the current position information through the heterogeneous graph; Step 4: Perform dimension adaptive processing on the high-dimensional feature information of the current position information, so that the dimension of the high-dimensional feature information in the current position information is consistent with the feature standard dimension; Step 5: Use the high-dimensional feature information in the current position information after dimensionality unification to train the reinforcement learning network of the unmanned boat formation and obtain the next action. Repeat steps 3-5 until the current unmanned boat formation is completed; Repeat steps 2-5 until the reinforcement learning network training is completed; Step 6: Use the trained reinforcement learning network to control the unmanned boat formation.
2. The unmanned boat formation method based on heterogeneous graph and dimension adaptive network as claimed in claim 1, characterized in that: In step 3, the output of the heterogeneous graph is: Where N i is the input feature of the i-th node; is the high-dimensional feature information of the i-th node; σ represents the activation function; For splicing operation; Represents the result on the edge between the i-th and j-th nodes at time t.
3. The unmanned boat formation method based on heterogeneous graph and dimension adaptive network as claimed in claim 2, characterized in that: The formula for obtaining the result on the edge of the two nodes at time t is: Where N i is the input feature of the i-th node; N j is the input feature of the jth node.
4. The unmanned boat formation method based on heterogeneous graph and dimension adaptive network as claimed in claim 1, characterized in that: In step 4, the dimension adaptive processing method includes: unifying the dimensions through a dimension adaptive module, wherein the dimension adaptive module includes two multi-layer perceptron layers and a convolutional layer connected in sequence, and each multi-layer perceptron layer includes several multi-layer perceptrons.
5. The unmanned boat formation method based on heterogeneous graph and dimension adaptive network as claimed in claim 4, characterized in that: The specific method for the dimension adaptation module to unify the dimensions is: The high-dimensional feature information of the current position information of all unmanned boats first passes through the first multi-layer perceptron layer to internally fuse the high-dimensional feature information of each unmanned boat. The fused data then passes through the second multi-layer perceptron layer to fuse the same high-dimensional feature information between the unmanned boats, and finally passes through the convolution layer to reduce the dimension to the standard feature dimension.