Method, device and electronic device for unmanned cluster distributed intelligent routing

Abstract

The present invention provides a method, device and electronic device for unmanned cluster distributed intelligent routing, which belongs to the technical field of networking communication. In the method, through information sharing and dynamic exchange between intelligent agents (i.e., network nodes) in the unmanned cluster network, the perception ability of the intelligent agents to the network status is comprehensively improved, thereby enhancing the global optimization ability of routing decisions, so that the network can still maintain efficient collaboration in a highly dynamic environment. In addition, each node can intelligently predict the future network status and formulate routing forwarding strategies based on its own historical local state vector and combined with real-time network changes (i.e., local state information), fully tapping the potential value of historical information, and greatly improving the accuracy and foresight of nodes in making routing decisions in complex network environments. In addition, the above process can perform the above distributed synchronous calculations locally on the current intelligent agent, and the network communication has good flexibility and high efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of networking communications, and in particular to a method, device and electronic equipment for unmanned cluster distributed intelligent routing. Background Art

[0002] Unmanned cluster network is an intelligent network system dynamically formed by various types of unmanned equipment through wireless communication technology, such as Figure 1 As shown in the figure. The node types of this network system are rich and diverse, covering three major areas: air, ground and sea, including: aerial drones (such as multi-rotor drones and fixed-wing drones), ground unmanned vehicles (such as autonomous vehicles and unmanned delivery vehicles) and marine unmanned ships (such as unmanned speedboats and unmanned submarines). Each node can play a specific role in different mission scenarios according to its functions and characteristics. Through self-organization and collaborative communication, these nodes of the unmanned cluster network can quickly form a dynamic distributed network without fixed infrastructure, flexibly adapt to complex environments and application requirements, and form a cross-domain interconnected structure covering land, sea and air. In this way, the network can realize multi-task collaboration and information sharing, support real-time information transmission and efficient task allocation. Relying on efficient communication technology and distributed computing capabilities, the unmanned cluster network can quickly respond to task requirements and dynamically adjust during execution to adapt to the changing environment, showing a high degree of flexibility, autonomy and intelligence, and is widely used in military, emergency rescue, logistics and transportation, scientific exploration and other fields.

[0003] Unmanned swarm networks have the following significant characteristics and unique advantages: First, the types of network nodes are rich and diverse, covering three domains: land, sea and air, including aerial drones, ground unmanned vehicles and surface unmanned ships, etc. Different nodes have their own unique movement modes, communication capabilities and mission characteristics. For example, aerial drones have high maneuverability and wide coverage, ground unmanned vehicles are suitable for performing high-precision ground tasks, and surface unmanned ships can work efficiently in water environments. Secondly, the topological structure of the network is highly dynamic. Since the nodes are usually in a high-speed mobile state, especially in air and surface scenes, their communication links are easily affected by node movement or environmental changes and change frequently. In addition, unmanned swarm networks have strong self-organizing capabilities. They can quickly form distributed networks in unfamiliar or harsh environments through collaboration and communication between nodes without the support of fixed infrastructure. This capability enables it to flexibly adapt to complex working scenarios, such as areas with dense obstacles in cities, harsh climatic conditions in the ocean, or complex terrain environments in mountainous areas. However, this network also faces many challenges, including unstable channel quality, increased communication interference, and restrictions on signal propagation due to complex terrain. Finally, due to the limited computing resources and battery life of unmanned devices, network performance is difficult to guarantee. These characteristics make unmanned cluster networks a highly flexible, intelligent, and adaptable network form that can meet the diverse needs in complex environments.

[0004] Nowadays, the application scenarios of unmanned swarm networks are very extensive. In the military field, unmanned swarm networks can be used for battlefield reconnaissance, target tracking, communication relay and tactical coordination. In emergency rescue, the flexibility and high coverage efficiency of unmanned equipment can be used to quickly monitor disasters, locate disaster-stricken areas and deliver materials. In logistics and transportation, the collaborative operation of drones and unmanned vehicles can significantly improve distribution efficiency and realize multimodal transport. In scientific exploration, unmanned swarm networks can complete data collection and environmental monitoring in oceans, polar regions or dangerous areas. In smart cities, the network can be used for intelligent services such as traffic management, security patrols, and unmanned delivery. This multi-domain collaborative network model provides important technical support for the future intelligent development of unmanned systems.

[0005] Focusing on the field of routing control, the existing unmanned cluster network routing methods can be mainly divided into the following categories: routing methods based on topology structure, routing methods based on geographic location, and routing methods based on intelligent learning. Specifically, the routing method based on topology structure is a relatively mature method at present, and typical representatives include OLSR (Optimized Link State Routing) and AODV (Ad hoc On-Demand Distance Vector Routing). These methods achieve data forwarding by maintaining the topology information in the network. Among them, OLSR is a table-driven routing protocol that maintains the global routing table by periodically exchanging link state information; AODV is an on-demand routing protocol that establishes paths only through routing requests and routing replies when needed. This type of routing method can be widely used in various scenarios in self-organizing networks.

[0006] The location-based routing method uses the location information of the node to make routing decisions. A typical example is GPSR (Greedy Perimeter Stateless Routing). This type of method does not need to maintain global network topology information, but selects the next hop node through a greedy strategy to achieve the gradual forwarding of data packets to the destination. When the path is blocked, GPSR can also perform route recovery through the perimeter traversal algorithm. The location-based routing method usually relies on the GPS information of the node and is suitable for distributed communication scenarios in large-scale networks.

[0007] Intelligent learning-based routing methods have gradually attracted attention in recent years. This type of method optimizes routing decisions by introducing machine learning models (such as reinforcement learning algorithms) and using historical communication data and real-time status information in the network. Unlike traditional methods, intelligent learning routing can continuously adjust its strategy according to the dynamic changes of the environment and has strong adaptability and global optimization capabilities. This method is particularly suitable for unmanned cluster networks in complex environments, such as multi-task collaboration and highly dynamic node scenarios.

[0008] The above methods provide a variety of solutions for the communication of unmanned cluster networks and show their respective advantages in different application scenarios. However, the above routing methods generally have certain limitations when facing the scenarios of high dynamics and topological instability of unmanned cluster networks:

[0009] First, although the topology-based routing method is currently widely used, in unmanned cluster networks, due to the frequent movement of nodes and the drastic changes in the topology structure, the overhead of routing table updates will increase significantly, resulting in delayed routing decisions and frequent communication interruptions. This method performs well in small and medium-sized static networks, but it is difficult to ensure routing efficiency in highly dynamic environments.

[0010] Secondly, the routing method based on geographic location may fail when the node distribution density is low or there are communication barriers. In addition, such methods rely heavily on the precise positioning of nodes. When the positioning accuracy is reduced or the signal interference is serious, the network performance will be significantly reduced.

[0011] On this basis, the routing method based on group behavior in intelligent learning has a high algorithm complexity, and control information needs to be frequently exchanged between nodes, resulting in high communication overhead, making it difficult to run efficiently on unmanned equipment with limited resources.

[0012] In summary, when network topology changes frequently and unmanned equipment resources are limited, how to achieve flexibility and optimality of network communication has become a technical problem that needs to be solved urgently. Summary of the invention

[0013] In view of this, the purpose of the present invention is to provide a method, device and electronic device for unmanned cluster distributed intelligent routing to alleviate the technical problem that the prior art cannot achieve flexibility and optimality of network communication when the network topology changes frequently and the resources of unmanned equipment are limited.

[0014] In a first aspect, an embodiment of the present invention provides a method for unmanned cluster distributed intelligent routing, including:

[0015] Performing a nonlinear transformation on the local state information of the current agent to obtain a local state vector of the current agent, and performing a nonlinear transformation on the local state information of the neighboring agents of the current agent to obtain a local state vector of the neighboring agents, wherein the local state information includes: location information, distance to the neighboring agents and queue load, and the local state vector includes: a query vector, a key vector and a value vector;

[0016] Compressing the key vector of the neighbor agent and the key vector of the current agent into an aggregate key vector, and compressing the value vector of the neighbor agent and the value vector of the current agent into an aggregate value vector;

[0017] Performing weighted fusion on the query vector of the current agent and the aggregate key vector to obtain an attention vector;

[0018] Performing a relative reasoning operation on the aggregate value vector and the historical local state vector of the current agent to obtain a state change value of the current agent;

[0019] Converting the state change value into a new state representation, and fusing the new state representation with the attention vector to obtain an updated historical local state vector;

[0020] The routing forwarding strategy of the current intelligent agent is determined according to the updated historical local state vector and the local state information of the current intelligent agent.

[0021] Furthermore, the method further comprises:

[0022] Determine whether the routing forwarding strategy points to a destination node, wherein the destination node is carried in the routing request, and the routing request also carries a source node;

[0023] If it does not point to the destination node, then executing the step of performing nonlinear transformation on the local state information of the current agent;

[0024] If it points to the destination node, the routing forwarding strategy is executed to form a multi-hop routing path.

[0025] Further, weighted fusion of the query vector of the current agent and the aggregate key vector includes:

[0026] The attention vector is obtained by weightedly merging the query vector of the current agent and the aggregate key vector through an attention mechanism.

[0027] Further, converting the state change value into a new state representation includes:

[0028] The state change value is converted into the new state representation through a feedforward neural network.

[0029] Further, determining the routing forwarding strategy of the current agent according to the updated historical local state vector and the local state information of the current agent includes:

[0030] The updated historical local state vector and the local state information of the current agent are input into a multilayer perceptron, and the routing forwarding strategy of the current agent is obtained as an output.

[0031] Furthermore, the current agent is a current network node determined according to the routing request and the routing forwarding strategy.

[0032] Furthermore, the method further comprises:

[0033] Using the multilayer perceptron to generate a Q value based on the local state information of the current agent and a reward value, wherein the Q value is used to evaluate the quality of the routing forwarding strategy, and the reward value is related to the single-hop routing forwarding distance and the queue utilization rate;

[0034] The policy gradient method is combined with the Q value to calculate the loss to optimize the parameters of the neural network model.

[0035] In a second aspect, an embodiment of the present invention further provides an unmanned cluster distributed intelligent routing device, comprising:

[0036] A nonlinear transformation unit, configured to perform nonlinear transformation on the local state information of the current agent to obtain the local state vector of the current agent, and perform nonlinear transformation on the local state information of the neighboring agent of the current agent to obtain the local state vector of the neighboring agent, wherein the local state information includes: location information, distance to the neighboring agent and queue load, and the local state vector includes: query vector, key vector and value vector;

[0037] A compression unit, configured to compress the key vector of the neighbor agent and the key vector of the current agent into an aggregate key vector, and compress the value vector of the neighbor agent and the value vector of the current agent into an aggregate value vector;

[0038] A weighted fusion unit, used for weighted fusion of the query vector of the current agent and the aggregate key vector to obtain an attention vector;

[0039] A relative reasoning operation unit, used for performing a relative reasoning operation on the aggregate value vector and the historical local state vector of the current agent to obtain a state change value of the current agent;

[0040] A conversion and fusion unit, used to convert the state change value into a new state representation, and fuse the new state representation with the attention vector to obtain an updated historical local state vector;

[0041] A determination unit is used to determine the routing forwarding strategy of the current agent according to the updated historical local state vector and the local state information of the current agent.

[0042] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0043] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to execute any method described in the first aspect above.

[0044] In an embodiment of the present invention, a method for unmanned cluster distributed intelligent routing is provided, including: performing nonlinear transformation on the local state information of the current intelligent agent to obtain the local state vector of the current intelligent agent, and performing nonlinear transformation on the local state information of the neighbor intelligent agent of the current intelligent agent to obtain the local state vector of the neighbor intelligent agent, wherein the local state information includes: location information, distance from the neighbor intelligent agent and queue load, and the local state vector includes: query vector, key vector and value vector; compressing the key vector of the neighbor intelligent agent and the key vector of the current intelligent agent into an aggregated key vector, and compressing the value vector of the neighbor intelligent agent and the value vector of the current intelligent agent into an aggregated value vector; weightedly fusing the query vector of the current intelligent agent with the aggregated key vector to obtain an attention vector; performing relative reasoning operation on the aggregated value vector and the historical local state vector of the current intelligent agent to obtain the state change value of the current intelligent agent; converting the state change value into a new state representation, and fusing the new state representation with the attention vector to obtain an updated historical local state vector; determining the routing forwarding strategy of the current intelligent agent according to the updated historical local state vector and the local state information of the current intelligent agent. It can be seen from the above description that in the method of distributed intelligent routing of unmanned clusters of the present invention, through information sharing and dynamic exchange (i.e., compression and weighted fusion process) between each intelligent agent (i.e., each network node) in the unmanned cluster network, the perception ability of the intelligent agent to the network state is comprehensively improved, the "information island" effect of a single node is reduced, thereby enhancing the global optimization ability of routing decisions, so that the network can still maintain efficient collaboration in a highly dynamic environment. In addition, each node can intelligently predict the future network state and formulate routing forwarding strategies based on its own historical local state vector and combined with real-time network changes (i.e., local state information). This mechanism fully taps the potential value of historical information and greatly improves the accuracy and foresight of nodes in making routing decisions in a complex network environment. In addition, the above process only requires the local state information of the current intelligent agent, the local state information of the neighboring intelligent agent, and the historical local state vector of the current intelligent agent, without the global information of the entire unmanned cluster network, and the above distributed synchronous calculation can be performed locally on the current intelligent agent, the network communication has good flexibility and high efficiency, and alleviates the technical problem that the prior art cannot achieve the flexibility and optimality of network communication when the network topology changes frequently and the resources of unmanned equipment are limited. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic diagram of an intelligent network system provided by an embodiment of the present invention;

[0047] Figure 2 A flowchart of a method for distributed intelligent routing of an unmanned cluster provided by an embodiment of the present invention;

[0048] Figure 3 A flow chart of another distributed intelligent routing method provided by an embodiment of the present invention;

[0049] Figure 4 A schematic diagram of an unmanned cluster distributed intelligent routing device provided by an embodiment of the present invention;

[0050] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Traditional technologies cannot achieve flexibility and optimality in network communications when network topology changes frequently and unmanned equipment resources are limited.

[0053] Based on this, in the method of distributed intelligent routing of unmanned clusters of the present invention, through information sharing and dynamic exchange (i.e., compression and weighted fusion process) between each intelligent agent (i.e., each network node) in the unmanned cluster network, the ability of the intelligent agent to perceive the network state is comprehensively improved, and the "information island" effect of a single node is reduced, thereby enhancing the global optimization ability of routing decisions, so that the network can still maintain efficient collaboration in a highly dynamic environment. In addition, each node can intelligently predict the future network state and formulate routing forwarding strategies based on its own historical local state vector and combined with real-time network changes (i.e., local state information). This mechanism fully taps the potential value of historical information and greatly improves the accuracy and foresight of nodes in making routing decisions in complex network environments. In addition, the above process only requires the local state information of the current intelligent agent, the local state information of the neighboring intelligent agent, and the historical local state vector of the current intelligent agent, without the global information of the entire unmanned cluster network, and the above distributed synchronous calculation can be performed locally on the current intelligent agent, and the network communication has good flexibility and high efficiency.

[0054] To facilitate understanding of this embodiment, a method for unmanned cluster distributed intelligent routing disclosed in an embodiment of the present invention is first introduced in detail.

[0055] Embodiment 1:

[0056] According to an embodiment of the present invention, an embodiment of a method for unmanned cluster distributed intelligent routing is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0057] Figure 2 is a flow chart of a method for distributed intelligent routing of an unmanned cluster according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0058] Step S202, performing a nonlinear transformation on the local state information of the current agent to obtain the local state vector of the current agent, and performing a nonlinear transformation on the local state information of the neighboring agents of the current agent to obtain the local state vector of the neighboring agents, wherein the local state information includes: location information, distance to the neighboring agents and queue load, and the local state vector includes: query vector, key vector and value vector;

[0059] Specifically, the local state information is the current local state information. The local state vectors correspond to characteristic representations of the network node states, and are used for subsequent compression and exchange operations.

[0060] The above-mentioned current intelligent agent is the current network node determined according to the routing request and the routing forwarding strategy. For example, when the algorithm is executed for the first time, the above-mentioned current intelligent agent can be specifically the source node. When the algorithm is executed for the second time, the above-mentioned current intelligent agent can be specifically the network node corresponding to the previous routing forwarding strategy. And so on, the current intelligent agent can be determined.

[0061] Each network node in the unmanned swarm network is modeled as an intelligent agent, and each node has local state information, reward value and action.

[0062] Step S204, compressing the key vector of the neighboring agent and the key vector of the current agent into an aggregate key vector, and compressing the value vector of the neighboring agent and the value vector of the current agent into an aggregate value vector;

[0063] Specifically, the above compression can be understood as finding the similarity between the two and obtaining the association relationship between the two. The aggregate key vector and the aggregate value vector are used to infer the importance of each relative state information to form a global collaborative view.

[0064] Step S206, weighted fusion of the query vector of the current agent and the aggregate key vector to obtain an attention vector;

[0065] Specifically, this process can be understood as “extracting knowledge from information”, that is, identifying the most valuable information for collaboration and providing support for routing decisions.

[0066] The process from step S202 to step S206 is a collaborative synchronization process (i.e., a process executed by a collaborative synchronization module). In order to generate an efficient routing strategy, each intelligent agent node in the unmanned cluster network needs to collaborate with other intelligent agents and improve its routing strategy. To this end, the distributed intelligent routing method of the present invention introduces a collaborative synchronization module to enhance the global optimality of the routing strategy, which includes the process from step S202 to step S206.

[0067] Each node can not only use its own state information, but also obtain key information related to neighboring nodes, thereby realizing a routing strategy with a more global perspective. On this basis, in addition to using the above steps to collaborate and share information with other agents, the current agent node also needs to capture the time correlation in its historical state information to enhance its routing decision. The present invention implements this process through the proposed time prediction module, and its steps are as follows.

[0068] Step S208, performing relative reasoning operation on the aggregated value vector and the historical local state vector of the current agent to obtain the state change value of the current agent;

[0069] Specifically, when performing relative reasoning operations, the time correlation contained in the historical local state vector is analyzed to obtain the state change value of the current agent, that is, the state change value of the current state of the current agent relative to each historical state.

[0070] Step S210, converting the state change value into a new state representation, and fusing the new state representation with the attention vector to obtain an updated historical local state vector;

[0071] Specifically, the process of converting the state change value into a new state representation can actually be regarded as a process of dimensionality transformation. The new state representation is essentially still the state change value. The dimension of the new state representation is the same as the dimension of the attention vector. Then, the new state representation is fused with the attention vector (specifically, multiplication) using linear transformation to obtain the updated historical local state vector.

[0072] Step S212, determining the routing forwarding strategy of the current agent according to the updated historical local state vector and the local state information of the current agent.

[0073] Specifically, the routing forwarding strategy clarifies the current next-hop forwarding target of the current agent, thereby achieving efficient data packet transmission. The routing forwarding strategy is an action, which means that the current node selects the next neighbor node as the next-hop forwarding node for the data packet.

[0074] It can be seen that the present invention introduces a collaborative synchronization module and a time prediction module. The collaborative synchronization module is used to enhance the information collaboration capability between nodes, and the time prediction module is used to enhance the utilization capability of historical states.

[0075] In an embodiment of the present invention, a method for unmanned cluster distributed intelligent routing is provided, including: performing nonlinear transformation on the local state information of the current intelligent agent to obtain the local state vector of the current intelligent agent, and performing nonlinear transformation on the local state information of the neighbor intelligent agent of the current intelligent agent to obtain the local state vector of the neighbor intelligent agent, wherein the local state information includes: location information, distance from the neighbor intelligent agent and queue load, and the local state vector includes: query vector, key vector and value vector; compressing the key vector of the neighbor intelligent agent and the key vector of the current intelligent agent into an aggregated key vector, and compressing the value vector of the neighbor intelligent agent and the value vector of the current intelligent agent into an aggregated value vector; weightedly fusing the query vector of the current intelligent agent with the aggregated key vector to obtain an attention vector; performing relative reasoning operation on the aggregated value vector and the historical local state vector of the current intelligent agent to obtain the state change value of the current intelligent agent; converting the state change value into a new state representation, and fusing the new state representation with the attention vector to obtain an updated historical local state vector; determining the routing forwarding strategy of the current intelligent agent according to the updated historical local state vector and the local state information of the current intelligent agent. It can be seen from the above description that in the method of distributed intelligent routing of unmanned clusters of the present invention, through information sharing and dynamic exchange (i.e., compression and weighted fusion process) between each intelligent agent (i.e., each network node) in the unmanned cluster network, the perception ability of the intelligent agent to the network state is comprehensively improved, the "information island" effect of a single node is reduced, thereby enhancing the global optimization ability of routing decisions, so that the network can still maintain efficient collaboration in a highly dynamic environment. In addition, each node can intelligently predict the future network state and formulate routing forwarding strategies based on its own historical local state vector and combined with real-time network changes (i.e., local state information). This mechanism fully taps the potential value of historical information and greatly improves the accuracy and foresight of nodes in making routing decisions in a complex network environment. In addition, the above process only requires the local state information of the current intelligent agent, the local state information of the neighboring intelligent agent, and the historical local state vector of the current intelligent agent, without the global information of the entire unmanned cluster network, and the above distributed synchronous calculation can be performed locally on the current intelligent agent, the network communication has good flexibility and high efficiency, and alleviates the technical problem that the prior art cannot achieve the flexibility and optimality of network communication when the network topology changes frequently and the resources of unmanned equipment are limited.

[0076] The above content briefly introduces the unmanned cluster distributed intelligent routing method of the present invention, and the specific contents involved are described in detail below.

[0077] In an alternative embodiment of the present invention, reference Figure 3 , the method further comprises the following steps:

[0078] (1) Determine whether the routing forwarding strategy points to the destination node, where the destination node is carried in the routing request, and the routing request also carries the source node;

[0079] (2) If it does not point to the destination node, execute the step of performing nonlinear transformation on the local state information of the current agent;

[0080] (3) If it points to the destination node, the routing forwarding strategy is executed to form a multi-hop routing path.

[0081] In an optional embodiment of the present invention, weighted fusion of the query vector of the current agent and the aggregate key vector specifically includes the following steps:

[0082] The attention mechanism is used to weight the query vector of the current agent and the aggregation key vector to obtain the attention vector.

[0083] In an optional embodiment of the present invention, converting the state change value into a new state representation specifically includes the following steps:

[0084] The state change value is converted into a new state representation through a feedforward neural network.

[0085] In an optional embodiment of the present invention, determining the routing forwarding strategy of the current agent according to the updated historical local state vector and the local state information of the current agent specifically includes the following steps:

[0086] The updated historical local state vector and the local state information of the current agent are input into the multi-layer perceptron, and the routing forwarding strategy of the current agent is output.

[0087] In an optional embodiment of the present invention, the current agent is a current network node determined according to the routing request and the routing forwarding strategy.

[0088] In an optional embodiment of the present invention, the method further comprises:

[0089] (1) Use a multi-layer perceptron to generate a Q value based on the local state information and reward value of the current agent. The Q value is used to evaluate the quality of the routing forwarding strategy, and the reward value is related to the single-hop routing forwarding distance and queue utilization.

[0090] Specifically, the reward value is an indicator used to evaluate the effectiveness of routing decisions, which mainly includes factors such as single-hop routing forwarding distance and queue utilization.

[0091] (2) Use the policy gradient method combined with the Q value to calculate the loss to optimize the parameters of the neural network model.

[0092] Specifically, through multiple rounds of training, the routing and forwarding strategy gradually approaches the optimal state and adapts to the highly dynamic environment of the network.

[0093] The present invention proposes a distributed intelligent routing method that combines a collaborative synchronization module with a time prediction module, and utilizes the distributed exchange and sharing of information between unmanned cluster device nodes to enhance the observation field of each unmanned device and enhance its understanding of the network status. At the same time, through the historical local status of each device, the hop-by-hop routing forwarding strategy of the device (i.e., the network node) is jointly output to improve the flexibility, stability and reliability of the routing method.

[0094] Through the organic combination of the collaborative synchronization module and the time prediction module, this method achieves the following in unmanned cluster networks:

[0095] 1. Global collaborative optimization: Enhance the global optimality of routing decisions through information sharing between nodes and weighted calculation based on the attention mechanism;

[0096] 2. Enhanced dynamic adaptability: Use historical status information to improve the time relevance and dynamic environment adaptability of routing strategies;

[0097] 3. Efficient training and deployment: Through a distributed multi-agent training framework, efficient routing strategy optimization is achieved, which is suitable for unmanned equipment deployment scenarios with limited resources (only local information can be obtained, and global information of the entire unmanned cluster network cannot be obtained).

[0098] The present invention proposes a distributed intelligent routing method that combines collaborative synchronization and time prediction to achieve efficient, reliable and stable unmanned cluster network routing forwarding, thereby improving network transmission quality. Specifically:

[0099] 1. This method uses a collaborative synchronization mechanism to promote information sharing and dynamic exchange among nodes in the unmanned cluster network, comprehensively improve the nodes' ability to perceive the network status, reduce the "information island" effect of a single node, thereby enhancing the global optimization capability of routing decisions and enabling the network to maintain efficient collaboration in a highly dynamic environment.

[0100] 2. This method further introduces a time prediction mechanism, which enables each node to intelligently predict the future network status and formulate routing forwarding strategies based on its own historical status data and real-time network changes. This mechanism fully taps the potential value of historical information and greatly improves the accuracy and foresight of node routing decisions in complex network environments.

[0101] 3. The organic combination of the above-mentioned collaborative synchronization mechanism and time prediction mechanism can effectively reduce problems such as routing decision lag and communication interruption, and achieve a balance between performance and energy efficiency in resource-constrained unmanned equipment, thereby meeting the application requirements of unmanned cluster networks in multi-task collaboration, highly dynamic topology and harsh environments.

[0102] The key technical points of the present invention are:

[0103] 1. Distributed intelligent routing method combining collaborative synchronization and time prediction: The present invention proposes a distributed intelligent routing method combining a collaborative synchronization module and a time prediction module. The collaborative synchronization mechanism is used to realize information sharing and dynamic coordination among nodes in an unmanned cluster network, and enhance the node's ability to understand the network status globally. At the same time, the time prediction mechanism is used to dynamically optimize the current routing forwarding strategy in combination with the node's historical status information. This method effectively improves the routing flexibility and global optimality of unmanned cluster networks in highly dynamic topology scenarios, and can cope with communication interruptions and network performance degradation in complex environments.

[0104] 2. Routing decision optimization based on attention mechanism: The present invention adopts a collaborative synchronization module based on the attention mechanism, which generates an aggregated routing decision vector by weighted fusion of node local state information and neighbor node information, and extracts key collaborative information; at the same time, it combines the time correlation analysis module to update the node's historical state information and generate an efficient hop-by-hop routing forwarding strategy. This method reduces communication overhead when unmanned equipment resources are limited, while improving the real-time and accuracy of routing decisions, and is suitable for highly dynamic, resource-constrained unmanned cluster network environments.

[0105] Embodiment 2:

[0106] An embodiment of the present invention also provides an unmanned cluster distributed intelligent routing device, which is mainly used to execute the unmanned cluster distributed intelligent routing method provided in the first embodiment of the present invention. The unmanned cluster distributed intelligent routing device provided in the embodiment of the present invention is specifically introduced below.

[0107] Figure 4 is a schematic diagram of an unmanned cluster distributed intelligent routing device according to an embodiment of the present invention, such as Figure 4 As shown, the device mainly includes: a nonlinear transformation unit 10, a compression unit 20, a weighted fusion unit 30, a relative reasoning operation unit 40, a conversion and fusion unit 50 and a determination unit 60, wherein:

[0108] A nonlinear transformation unit is used to perform nonlinear transformation on the local state information of the current agent to obtain the local state vector of the current agent, and to perform nonlinear transformation on the local state information of the neighboring agents of the current agent to obtain the local state vector of the neighboring agents, wherein the local state information includes: location information, distance to the neighboring agents and queue load, and the local state vector includes: query vector, key vector and value vector;

[0109] A compression unit, used to compress the key vectors of neighboring agents and the key vector of the current agent into an aggregate key vector, and compress the value vectors of neighboring agents and the value vector of the current agent into an aggregate value vector;

[0110] The weighted fusion unit is used to perform weighted fusion of the query vector of the current agent and the aggregation key vector to obtain the attention vector;

[0111] The relative reasoning operation unit is used to perform relative reasoning operation on the aggregate value vector and the historical local state vector of the current agent to obtain the state change value of the current agent;

[0112] The conversion and fusion unit is used to convert the state change value into a new state representation and fuse the new state representation with the attention vector to obtain an updated historical local state vector;

[0113] The determination unit is used to determine the routing forwarding strategy of the current intelligent agent according to the updated historical local state vector and the local state information of the current intelligent agent.

[0114] In an embodiment of the present invention, a device for unmanned cluster distributed intelligent routing is provided, including: performing nonlinear transformation on the local state information of the current intelligent agent to obtain the local state vector of the current intelligent agent, and performing nonlinear transformation on the local state information of the neighbor intelligent agent of the current intelligent agent to obtain the local state vector of the neighbor intelligent agent, wherein the local state information includes: location information, distance from the neighbor intelligent agent and queue load, and the local state vector includes: query vector, key vector and value vector; compressing the key vector of the neighbor intelligent agent and the key vector of the current intelligent agent into an aggregated key vector, and compressing the value vector of the neighbor intelligent agent and the value vector of the current intelligent agent into an aggregated value vector; weightedly fusing the query vector of the current intelligent agent with the aggregated key vector to obtain an attention vector; performing relative reasoning operation on the aggregated value vector and the historical local state vector of the current intelligent agent to obtain the state change value of the current intelligent agent; converting the state change value into a new state representation, and fusing the new state representation with the attention vector to obtain an updated historical local state vector; determining the routing forwarding strategy of the current intelligent agent according to the updated historical local state vector and the local state information of the current intelligent agent. It can be seen from the above description that in the unmanned cluster distributed intelligent routing device of the present invention, through the information sharing and dynamic exchange (i.e., the process of compression and weighted fusion) between each intelligent agent (i.e., each network node) in the unmanned cluster network, the intelligent agent's perception ability of the network state is comprehensively improved, and the "information island" effect of a single node is reduced, thereby enhancing the global optimization ability of routing decisions, so that the network can still maintain efficient collaboration in a highly dynamic environment. In addition, each node can intelligently predict the future network state and formulate a routing forwarding strategy based on its own historical local state vector and combined with real-time network changes (i.e., local state information). This mechanism fully taps the potential value of historical information and greatly improves the accuracy and foresight of nodes in making routing decisions in a complex network environment. In addition, the above process only requires the local state information of the current intelligent agent, the local state information of the neighboring intelligent agent, and the historical local state vector of the current intelligent agent, without the global information of the entire unmanned cluster network, and the above distributed synchronous calculation can be performed locally on the current intelligent agent, the network communication has good flexibility and high efficiency, and alleviates the technical problem that the prior art cannot achieve the flexibility and optimality of network communication when the network topology changes frequently and the resources of unmanned equipment are limited.

[0115] Optionally, the device is also used to: determine whether the routing forwarding strategy points to the destination node, wherein the destination node is carried in the routing request, and the routing request also carries the source node; if it does not point to the destination node, execute the step of performing a nonlinear transformation on the local state information of the current intelligent agent; if it points to the destination node, execute the routing forwarding strategy to form a multi-hop routing path.

[0116] Optionally, the weighted fusion unit is also used to: perform weighted fusion of the query vector of the current agent and the aggregation key vector through an attention mechanism to obtain an attention vector.

[0117] Optionally, the conversion and fusion unit is further used to: convert the state change value into a new state representation through a feedforward neural network.

[0118] Optionally, the determination unit is further used to: input the updated historical local state vector and the local state information of the current agent into the multi-layer perceptron, and output a routing forwarding strategy of the current agent.

[0119] Optionally, the current agent is a current network node determined according to a routing request and a routing forwarding strategy.

[0120] Optionally, the device is also used to: use a multilayer perceptron to generate a Q value based on the local state information and reward value of the current intelligent agent, wherein the Q value is used to evaluate the quality of the routing forwarding strategy, and the reward value is related to the single-hop routing forwarding distance and queue utilization; use a policy gradient method combined with the Q value to calculate the loss to optimize the parameters of the neural network model.

[0121] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0122] like Figure 5 As shown, an electronic device 600 provided in an embodiment of the present application includes: a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the unmanned cluster distributed intelligent routing method as described above.

[0123] Specifically, the above-mentioned memory 602 and processor 601 can be general-purpose memory and processor, which are not specifically limited here. When the processor 601 runs the computer program stored in the memory 602, the above-mentioned unmanned cluster distributed intelligent routing method can be executed.

[0124] The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 601. The above processor 601 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0125] Corresponding to the above-mentioned method of unmanned cluster distributed intelligent routing, an embodiment of the present application also provides a computer-readable storage medium, which stores machine-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned method of unmanned cluster distributed intelligent routing.

[0126] The device for unmanned cluster distributed intelligent routing provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application, its implementation principle and the technical effect produced are the same as those in the aforementioned method embodiment. For the sake of brief description, where the device embodiment is not mentioned, reference can be made to the corresponding content in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0127] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0128] For another example, the flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0131] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the unmanned cluster distributed intelligent routing method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0132] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0133] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solution of the present application, rather than to limit it. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solution recorded in the aforementioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiment of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for distributed intelligent routing of unmanned clusters, characterized in that: include: Performing a nonlinear transformation on the local state information of the current agent to obtain a local state vector of the current agent, and performing a nonlinear transformation on the local state information of the neighboring agents of the current agent to obtain a local state vector of the neighboring agents, wherein the local state information includes: location information, distance to the neighboring agents and queue load, and the local state vector includes: a query vector, a key vector and a value vector; Compressing the key vector of the neighbor agent and the key vector of the current agent into an aggregate key vector, and compressing the value vector of the neighbor agent and the value vector of the current agent into an aggregate value vector; Performing weighted fusion on the query vector of the current agent and the aggregate key vector to obtain an attention vector; Performing a relative reasoning operation on the aggregate value vector and the historical local state vector of the current agent to obtain a state change value of the current agent; Converting the state change value into a new state representation, and fusing the new state representation with the attention vector to obtain an updated historical local state vector; The routing forwarding strategy of the current intelligent agent is determined according to the updated historical local state vector and the local state information of the current intelligent agent.

2. The method according to claim 1, characterized in that The method further comprises: Determine whether the routing forwarding strategy points to a destination node, wherein the destination node is carried in the routing request, and the routing request also carries a source node; If it does not point to the destination node, then executing the step of performing nonlinear transformation on the local state information of the current agent; If it points to the destination node, the routing forwarding strategy is executed to form a multi-hop routing path.

3. The method according to claim 1, characterized in that The query vector of the current agent is weightedly fused with the aggregate key vector, including: The attention vector is obtained by weightedly merging the query vector of the current agent and the aggregate key vector through an attention mechanism.

4. The method according to claim 1, characterized in that: Converting the state change value into a new state representation includes: The state change value is converted into the new state representation through a feedforward neural network.

5. The method according to claim 1, characterized in that Determining the routing forwarding strategy of the current agent according to the updated historical local state vector and the local state information of the current agent includes: The updated historical local state vector and the local state information of the current agent are input into a multilayer perceptron, and the routing forwarding strategy of the current agent is obtained as an output.

6. The method according to claim 2, characterized in that The current agent is a current network node determined according to the routing request and the routing forwarding strategy.

7. The method according to claim 5, characterized in that The method further comprises: Using the multilayer perceptron to generate a Q value based on the local state information of the current agent and a reward value, wherein the Q value is used to evaluate the quality of the routing forwarding strategy, and the reward value is related to the single-hop routing forwarding distance and the queue utilization rate; The policy gradient method is combined with the Q value to calculate the loss to optimize the parameters of the neural network model.

8. An unmanned cluster distributed intelligent routing device, characterized in that: include: A nonlinear transformation unit, configured to perform nonlinear transformation on the local state information of the current agent to obtain the local state vector of the current agent, and perform nonlinear transformation on the local state information of the neighboring agent of the current agent to obtain the local state vector of the neighboring agent, wherein the local state information includes: location information, distance to the neighboring agent and queue load, and the local state vector includes: query vector, key vector and value vector; A compression unit, configured to compress the key vector of the neighbor agent and the key vector of the current agent into an aggregate key vector, and compress the value vector of the neighbor agent and the value vector of the current agent into an aggregate value vector; A weighted fusion unit, used for weighted fusion of the query vector of the current agent and the aggregate key vector to obtain an attention vector; A relative reasoning operation unit, used for performing a relative reasoning operation on the aggregate value vector and the historical local state vector of the current agent to obtain a state change value of the current agent; A conversion and fusion unit, used to convert the state change value into a new state representation, and fuse the new state representation with the attention vector to obtain an updated historical local state vector; A determination unit is used to determine the routing forwarding strategy of the current agent according to the updated historical local state vector and the local state information of the current agent.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to execute the method according to any one of claims 1 to 7.

Images