A multi-robot cluster-oriented integrated anti-interference ad hoc network method

Abstract

The application belongs to the technical field of wireless communication and robot cluster cooperative control, and discloses a multi-robot cluster-oriented integrated anti-interference ad hoc network method of sensing and computing, which comprises the following steps: each robot node broadcasts a beacon signal through a multi-mode communication terminal to construct an initial network topology; a cluster management node collects state information of each node, constructs a state matrix, inputs a competitive double-deep Q network, and establishes a dynamic multi-hop communication network; an upper-layer resource scheduling intelligent agent outputs a joint allocation scheme of communication bandwidth, computing power quota and sensing tasks through a pointer network; a lower-layer routing planning intelligent agent plans a communication route, and avoids relay node competition conflict through a conflict detection and dynamic arbitration mechanism; an electromagnetic situation is sensed in real time, an anti-interference strategy is selected through a competitive double-deep Q network intelligent agent; and link quality and energy state are continuously monitored, and topology reconstruction is automatically triggered. The application effectively improves the communication reliability, real-time performance and anti-interference capability of the multi-robot cluster in a complex dynamic environment.

Description

Technical Field

[0001] This invention belongs to the field of wireless communication and robot swarm collaborative control technology, specifically involving an integrated communication, sensing, and computing anti-interference self-organizing network method for multi-robot swarms, applicable to dynamic networking, anti-interference communication, and collaborative scheduling of communication-sensing-computing resources in multi-robot collaborative operation scenarios. Background Technology

[0002] With the rapid development of communication technology, embodied intelligence, and bionic robot technology, collaborative operations of multi-robot swarms have been widely applied in complex scenarios such as industrial inspection, emergency rescue, security patrol, and petrochemicals. Existing bionic quadruped robots are typically equipped with dual-light gimbals, lidar, depth cameras, and 4G / 5G / BeiDou communication modules, possessing basic capabilities such as environmental perception, autonomous navigation, and remote control.

[0003] In collaborative operations among multiple robots, robots need to share environmental perception data, location information, task status, and control commands in real time, placing extremely high demands on the reliability, real-time performance, and anti-interference capabilities of the communication network. However, existing communication solutions face the following prominent problems: (1) Insufficient communication reliability: Traditional Wi-Fi, 4G and other communication methods are prone to interruption in complex terrain, building obstruction and severe electromagnetic interference industrial environments, making it difficult to guarantee real-time data interaction between multiple robots.

[0004] (2) Spectrum resource competition: Communication, sensing and computing tasks occupy spectrum and power resources independently, resulting in resource competition. Sensing tasks (such as lidar mapping and image recognition) may affect communication performance.

[0005] (3) Lack of self-organizing network capability: The existing cluster system lacks dynamic topology management capability and is difficult to adapt to dynamic scenarios such as changes in the number of robots, location movement, and node failure.

[0006] (4) Low efficiency of communication-sensing-computing collaboration: Communication, sensing and computing functions are separated, and data needs to be transmitted multiple times between different nodes, which increases latency and bandwidth consumption.

[0007] Currently, multi-robot swarm communication mainly adopts the following technical solutions, but these solutions have the following shortcomings: (1) Communication scheme based on fixed infrastructure: It relies on pre-deployed fixed nodes such as base stations and APs. In complex terrain or disaster scenarios, the fixed infrastructure may be damaged or unavailable, resulting in communication interruption.

[0008] (2) Based on traditional Ad-Hoc self-organizing network scheme: Traditional routing protocols such as AODV and DSR are adopted, which lack the ability to adapt to dynamic topology changes and channel quality, and the routing convergence speed is slow, making it difficult to meet real-time requirements.

[0009] (3) Communication scheduling scheme based on single deep reinforcement learning: A few studies have attempted to apply deep reinforcement learning to communication scheduling, but existing schemes usually only target a single decision dimension (such as optimizing only the route or only the spectrum allocation), lacking a systematic consideration of the coordination of communication-sensing-computing. Summary of the Invention

[0010] This invention aims to solve the technical problems faced by multi-robot swarms when working collaboratively in complex dynamic environments, such as insufficient communication reliability, spectrum resource competition, lack of self-organizing network capability, and low efficiency of communication-sensing-computing collaboration. It provides a communication-sensing-computing integrated anti-interference self-organizing network method for multi-robot swarms.

[0011] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a method for integrated sensing and computing anti-interference self-organizing networks for multi-robot clusters, comprising the following steps: S1: Each robot node broadcasts beacon signals through a multi-mode communication terminal to discover neighboring nodes and build the initial network topology; S2: The cluster management node collects the status information of each node, constructs a status matrix, inputs it into a competitive dual-depth Q network, outputs a networking strategy, and establishes a dynamic multi-hop communication network. S3: The upper-layer resource scheduling agent outputs a joint allocation scheme of communication bandwidth, computing power quota and perception tasks through the pointer network based on the task pool status and node resource status; the perception tasks include environmental perception tasks and communication perception tasks. S4: The lower-level routing planning agent plans the optimal communication route for each node and avoids relay node contention through conflict detection and dynamic arbitration mechanisms. S5: Real-time electromagnetic situation perception, which includes environmental space situation and electromagnetic spectrum situation. The optimal anti-interference strategy is selected through a competitive dual-deep Q network agent, and communication parameters are dynamically adjusted to deal with interference. S6: Continuously monitors the link quality and energy status of each node. When a node failure, link interruption, or new node addition is detected, topology reconfiguration is automatically triggered.

[0012] Secondly, embodiments of the present invention also provide an integrated sensing and computing anti-interference self-organizing network system for multi-robot clusters, comprising: Cluster management nodes, deployed on edge servers or in the cloud, are responsible for global topology management, spectrum scheduling, and task orchestration. Multiple robot communication nodes are provided, with each robot equipped with a multi-mode communication terminal, a perception computing unit, and a dynamic self-organizing network protocol stack. The multi-mode communication terminal includes a 5G / B5G cellular communication module, a Mesh self-organizing network module, and a directional antenna array, supporting multi-hop relay communication. The perception computing unit is integrated into the embodied intelligent brainboard platform of each robot, adopts an edge AI chip, and is interconnected with the communication module via PCIe or Ethernet, responsible for environmental perception, data preprocessing, and local inference. The dynamic self-organizing network protocol stack is based on a distributed "communication-computation-execution" protocol, enabling nodes to autonomously discover, dynamically join the network, and reconstruct the topology. The system performs the method described in the first aspect.

[0013] Thirdly, embodiments of the present invention also provide a storage medium storing instructions, wherein when a computer reads the instructions, the computer executes any of the above-described methods for integrated sensory computing anti-interference self-organizing networks for multi-robot clusters.

[0014] Fourthly, embodiments of the present invention also provide an electronic device, including a processor and the aforementioned storage medium, wherein the processor executes instructions in the storage medium.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. High reliability: Through D3QN dynamic networking and multi-mode communication redundancy, the connectivity of cluster communication can still be maintained when some nodes fail or links are interrupted; 2. Low latency: By leveraging edge computing and collaborative scheduling of communication-sensing-computing, the latency of sensing data transmission and processing is reduced; 3. Strong anti-interference: It integrates multi-source sensing (including wireless link sensing such as CSI), D3QN intelligent decision-making and dynamic spectrum access to achieve proactive intelligent anti-interference; 4. High spectrum efficiency: Communication and sensing functions share spectrum resources, and efficient reuse of resources is achieved through joint scheduling. Attached Figure Description

[0016] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the overall architecture of a communication system for multi-robot clusters according to an embodiment of the present invention; Figure 2 This is a flowchart of a sensory-computing integrated anti-interference self-organizing network method for multi-robot clusters according to an embodiment of the present invention; Figure 3 This is a network architecture diagram of the D3QN dynamic networking intelligent agent according to an embodiment of the present invention; Figure 4This is a diagram illustrating the communication-sensing-computing resource collaborative scheduling framework of an embodiment of the present invention. Figure 5 This is a flowchart of the spatial decomposition and parallel search mechanism in an embodiment of the present invention; Figure 6 This is a flowchart of intelligent anti-interference communication decision-making. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] To address the technical problems mentioned in the background section, this invention proposes a sensor-computing integrated anti-interference self-organizing network method for multi-robot swarms. The core ideas include: constructing a multi-layered collaborative agent architecture, decomposing dynamic networking, anti-interference communication, and communication-sensing-computing resource coordination into multiple coupled sub-problems, with multiple collaborative agents making decisions separately and jointly optimizing; introducing a competitive dual-deep Q-network architecture to alleviate the Q-value overestimation problem and achieve more stable communication strategy learning; designing a communication-sensing-computing resource collaborative scheduler to achieve dynamic joint optimization allocation of communication, sensing, and computing resources through deep reinforcement learning; and employing spatial decomposition and parallel search mechanisms to decompose the high-dimensional decision space into multiple differentiated subspaces for parallel optimization, improving the algorithm's solution efficiency in large-scale swarm scenarios.

[0019] Furthermore, such as Figure 1 As shown, this invention proposes a four-layer collaborative communication architecture, including a physical layer, a network layer, a collaboration layer, and an application layer, which achieves collaborative optimization of communication, perception, and computing capabilities through a multi-layered deep reinforcement learning agent.

[0020] Based on the above four-layer collaborative communication architecture, a sensor-computer integrated anti-interference self-organizing network system for multi-robot clusters is constructed, including the following core modules: (a) Cluster management node: Deployed on edge servers or in the cloud, responsible for global topology management, spectrum scheduling and task orchestration; (ii) Multiple robot communication nodes: Each robot is equipped with a multi-mode communication terminal (including a 5G / B5G cellular communication module with reserved upgrade capability for 6G, a Mesh self-organizing network module and a directional antenna array), supporting multi-hop relay communication; Perception computing unit: Integrated into the embodied intelligent brainboard platform of each robot, using an edge AI chip with an INT8 computing power of no less than 30 TOPS, interconnected with the communication module through PCIe or Ethernet, responsible for environmental perception, data preprocessing and local inference; Dynamic self-organizing network protocol stack: Based on the distributed "communication-computation-execution" protocol, it realizes the autonomous discovery, dynamic network entry and topology reconstruction of nodes.

[0021] In this invention, "perception" uniformly refers to the integration of environmental spatial perception information and wireless link perception information: (1) environmental spatial perception information obtained by sensors such as lidar, depth camera, and infrared thermal imaging mounted on the robot; (2) wireless link-side perception information, including communication-side perception parameters such as channel state information (CSI), spectrum occupancy status, and beam quality. In state space construction, resource scheduling decision-making, and anti-interference decision-making, "perception" refers to the integrated input of the above two types of information. The conceptual levels of "environmental perception" and "communication perception" are no longer distinguished. The source category is only indicated in the specific feature description.

[0022] The integrated sensory-computing anti-interference self-organizing network system for multi-robot clusters is used to execute an integrated sensory-computing anti-interference self-organizing network method for multi-robot clusters as described in the following embodiment.

[0023] like Figure 2 As shown, a sensor-computer integrated anti-interference self-organizing network method for multi-robot swarms is applied to multi-robot swarm communication scenarios in industrial inspection. For example, in industrial inspection scenarios such as large chemical plants and substations, multiple intelligent bionic quadruped robots collaboratively perform inspection tasks. Each robot is equipped with a dual-light gimbal (visible light + infrared), multi-line LiDAR, depth camera, and 5G / B5G communication module (with a reserved 6G upgrade interface). The method includes the following steps: S1: Each robot node broadcasts beacon signals through a multi-mode communication terminal to discover neighboring nodes and build the initial network topology; Step S1 is used to perform system initialization and node discovery. Each robot node broadcasts beacon signals through the 5G / B5G communication module to discover neighboring nodes and build the initial network topology.

[0024] S2: The cluster management node collects the status information of each node, constructs a status matrix, inputs it into a competitive dual-depth Q network, outputs a networking strategy, and establishes a dynamic multi-hop communication network. Step S2 is used to construct a dynamic self-organizing network based on a contention-based dual-deep Q network. The cluster management node collects the state information of each node, constructs a state matrix, inputs it into the D3QN network, and outputs the optimal networking strategy (parent node selection, power level, modulation scheme) to establish a dynamic multi-hop communication network. Each action in the action space of the contention-based dual-deep Q network corresponds to a combination of networking strategies, including parent node selection strategy, transmit power level, modulation and coding scheme selection, and frequency band switching decision. Each action consists of relay selection decision, power level decision, modulation and coding scheme decision, and frequency band switching decision. The relay selection decision serves as the primary decision level for the action, while the power level decision, modulation and coding scheme decision, and frequency band switching decision together constitute the auxiliary decision level for the action. Specifically: This invention employs a competing Double Deep Q-Network (D3QN) architecture to achieve dynamic self-organizing network construction for multi-robot swarms. D3QN decouples action selection and value assessment through a dual-network structure (online network and target network), effectively alleviating the Q-value overestimation problem of traditional DQN; simultaneously, the competing network architecture decouples state value and advantage function, improving learning stability. Figure 3 The diagram shown is a network architecture diagram of the D3QN dynamic networking intelligent agent according to an embodiment of the present invention.

[0025] Compared with the standard D3QN algorithm, this invention makes the following targeted improvements for multi-robot swarm communication scenarios: (1) The state space is designed with a ten-dimensional feature vector that includes signal-to-noise ratio, packet loss rate, hop count, remaining energy, communication load, moving speed, task priority and representative CSI indicators (mean subcarrier gain, delay spread, and Doppler shift), which fully covers four dimensions: communication link quality, node resource status, task urgency and wireless channel physical layer status. Among them, the three CSI indicators are obtained by statistical feature extraction of the original CSI matrix reported by the node (mean subcarrier gain is the arithmetic mean of the gains of each subcarrier, delay spread is the square root of the second central moment of the power delay spectrum, and Doppler shift is estimated according to the channel time correlation function). (2) The attention pooling layer is used to replace the fully connected pooling in the standard D3QN, so that the network can adaptively focus on the most critical node information in the current state, which solves the problem that fixed-dimensional input cannot be processed when the cluster size changes dynamically.

[0026] The specific implementation of the above improvements is as follows: (1) State space design: Model the network topology of the robot cluster as a state matrix. ,in This represents the number of robot nodes in the cluster. This represents the state feature dimension. (Node) State feature vector ( The node number is defined as follows:

[0027] in, , Seven basic link state characteristics ( For nodes The received signal-to-noise ratio (in dB). For packet loss rate, The number of hops to the management node. For the remaining energy, For the current communication load, For node movement speed, (task priority) The average subcarrier gain. For delay extension, For Doppler frequency shift, all three terms are representative scalar indices of CSI obtained through statistical feature extraction. Therefore, the total dimension of the state vector is... .

[0028] (2) Motion space design: Define motion space Each action corresponds to a combination of networking strategies, including: parent node selection strategy (which neighbor node to select as the uplink relay), transmit power level (multi-level adjustable), modulation and coding scheme selection, and frequency band switching decision.

[0029] Each action adopts a two-layer priority structure of "main strategy + auxiliary strategy": the main strategy determines the relay selection direction, and the auxiliary strategy determines the power and modulation parameters; the frequency band switching decision is included in the auxiliary strategy, and together with the power level and modulation coding scheme, it constitutes the auxiliary strategy output vector.

[0030] (3) Reward Function Design: The reward function of the competitive dual-deep Q network is constructed based on the overall network performance. The reward function includes terms for throughput variation, end-to-end latency variation, link reliability variation, and spectrum and energy consumption costs. Specifically, the overall network performance is transformed into an instantaneous reward signal perceptible to the agent:

[0031] in: This represents the change in throughput. This represents the change in end-to-end latency (a negative value indicates improvement; since the coefficient before this item is negative, this item is a positive reward when latency is improved). This represents the change in link reliability. Costs related to spectrum and energy consumption. This is the throughput weighting coefficient. This is the time delay weighting coefficient. For reliability weighting coefficients, As a weighting coefficient for energy consumption costs, in a preferred embodiment of the present invention, =0.4, =0.3, =0.2, =0.1.

[0032] (4) Network architecture: The competitive dual-deep Q-network employs a multilayer perceptron-attention-pooling-competitive network architecture (MLP-Attention-Dueling Network). The input layer receives a variable number of node state vectors. The attention-pooling layer compresses the variable-length input into a fixed-dimensional global state representation. This representation then passes through a shared fully connected layer (set to 256 dimensions) before entering the competitive network branch to output the actions. The Q value is calculated from the state value function and the advantage function, where the value of the advantage function is subtracted from its average value across all actions, thus decoupling the state value and the advantage function along the action dimension. The specific formula is as follows:

[0033] in, State 1 Action Pair Action value function; :state State value function; :state Next action The dominant function; Action space The total number of actions in the middle; :state Next, execute candidate actions The dominant function; Action space Candidate actions in the process.

[0034] (5) Training Mechanism: Prioritized Experience Replay and soft update mechanisms are adopted. The target Q-value is calculated using a Double Deep Q-Network (DQN) mechanism. The online network selects an action based on the state at the next time step, and the target network evaluates the value of the action to calculate the target Q-value, as shown in the following formula:

[0035] in, : No. The target Q value (TD target) for the step; : No. Instant rewards earned with each step; Discount factor (range of values) In this embodiment, a value of 0.95 is used to balance the weights of immediate rewards and future rewards; The Q-value estimation function of the TargetNetwork for state-action pairs; : No. The state of the next step; In action space Select the action that maximizes the Q value. Operators; Online network (Network) parameters Next, for the next state and candidate actions The Q-value estimation function; Learnable parameters of an online network; Learnable parameters of the target network.

[0036] S3: The upper-layer resource scheduling agent outputs a joint allocation scheme of communication bandwidth, computing power quota and perception tasks through the pointer network based on the task pool status and node resource status; the perception tasks include environmental perception tasks and communication perception tasks. Step S3 is used to implement joint scheduling of communication, sensing, and computing resources. The upper-layer resource scheduling agent, based on the task pool state and node resource state, outputs a joint allocation scheme for communication bandwidth, computing power quota, and sensing tasks through a pointer network. The upper-layer resource scheduling agent in step S3 and the lower-layer routing planning agent in step S4 constitute a dual-agent collaborative scheduling framework with a bidirectional coupling relationship: the bandwidth allocation result output by the upper-layer resource scheduling agent serves as the constraint feature input to the state space of the lower-layer routing planning agent; the conflict rate and average end-to-end latency during the routing planning process of the lower-layer routing planning agent serve as negative feedback terms for the Critic network reward of the upper-layer resource scheduling agent.

[0037] S4: The lower-level routing planning agent plans the optimal communication route for each node and avoids relay node contention through conflict detection and dynamic arbitration mechanisms. Step S4 is used to implement communication route planning. The lower-level routing planning agent plans the optimal communication route for each node and avoids relay node contention through conflict detection and dynamic arbitration mechanisms.

[0038] like Figure 4 The diagram shown illustrates the communication-sensing-computing resource collaborative scheduling framework of an embodiment of the present invention. The communication-sensing-computing resource collaborative scheduler is designed as follows: This invention designs a dual-agent cooperative scheduling framework, comprising an upper-layer resource scheduling agent and a lower-layer communication routing planning agent. Their coupling relationship is as follows: the bandwidth allocation result output from the upper layer serves as an additional constraint feature input to the lower-layer D3QN state space; the conflict rate and average end-to-end latency during the lower-layer communication routing planning execution process serve as negative feedback terms for the upper-layer Critic network reward, forming a complete cooperative training closed loop.

[0039] Furthermore, the upper-layer resource scheduling agent adopts a pointer network architecture, which includes an encoder and a decoder. The encoder uses a long short-term memory network to perform context encoding on the input sequence, and the decoder generates resource allocation decisions bit by bit through an attention mechanism, outputting the resource combination obtained by each node. The resource combination includes spectrum bandwidth, computing power quota, and perception task allocation. The upper-layer resource scheduling agent is trained using the Actor-Critic algorithm, where the Critic network is used to estimate the state baseline value to reduce the policy gradient variance. The lower-layer routing planning agent adopts a competitive dual-depth Q-network, whose state representation uses a four-channel matrix. The four-channel matrix includes: a link availability matrix for representing the link connectivity status between nodes, a link quality matrix for representing the estimated signal-to-noise ratio and bit error rate of each link, a node load matrix for representing the current communication queue depth and computational load of each node, and a target node matrix for representing the target node's location identifier. The reward function of the lower-layer routing planning agent is constructed based on a combination of route hop count, link quality, and end-to-end latency. Specifically: 1. Upper layer: Communication-sensing-computing resource scheduling agent, using a pointer network architecture to jointly allocate communication, sensing, and computing resources: (1) State space: contains the task pool state (Communication requirements, sensing tasks, and computing tasks of each node) and node resource status (Available spectrum, computing power margin, energy status).

[0040] (2) Action space: Output resource allocation vector ,in, Represents a node The acquired resource combination (spectrum bandwidth, computing power quota, and sensing task allocation).

[0041] (3) Encoder-decoder architecture: The encoder uses a Long Short-Term Memory (LSTM) network to perform context encoding on the input sequence. Among them, For the first inputs (i.e., nodes) Embedded vectors of ) For encoder number Hidden state of the step:

[0042] The decoder generates allocation decisions bit by bit through an attention mechanism:

[0043] in, : Encoder (LSTM) The hidden state of the step; Long Short-Term Memory (LSTM) network units are embedded with the current input. Hidden state from the previous moment Given the input, output the hidden state of the current step. ; Decoder No. The hidden state of the step; Learnable weight matrix parameters used to linearly transform the encoder and decoder hidden states to an attention space of the same dimension; Learnable weight vector parameters used to compress attention scores into scalars; : No. In the decoding step, the first The unnormalized attention score (alignment score) corresponding to each input; Learnable weight vector Transpose of; The hyperbolic tangent activation function maps the input to... Intervals are used to introduce nonlinearity; : No. Decoding step selection number inputs (i.e., nodes) The probability of the current allocation target. Indicates the decoder's first The output index of the step; The normalized exponential function transforms the unnormalized score vector into a probability distribution.

[0044] (4) Training method: The Actor-Critic algorithm is used. The Critic network estimates the state baseline value to reduce the policy gradient variance.

[0045]

[0046] in, Learnable parameters of the Actor network (policy network) used to output the probability distribution of actions; Learnable parameters of the Critic network (value network) used to estimate state value; : The cumulative reward (total return) obtained under the current strategy; Critic network in parameters Next state The value estimate (as a baseline for the policy gradient); Objective function Regarding strategy parameters The gradient; The mathematical expectation operator takes the expectation of the random variable within the parentheses. In state Take action below The strategy log probability with respect to parameters The gradient (policy score function); : The Actor network defines the state Select action The strategy probability distribution.

[0047] 2. Lower Layer: Communication routing planning agent, employing an improved competitive dual-depth Q network (D3QN) to plan the optimal communication route for each robot: (1) State representation: A four-channel matrix is ​​used to represent the network layer state—the link availability matrix. (Link connectivity status between nodes), Link quality matrix (SNR and bit error rate estimation for each link), node load matrix (Current communication queue depth and computational load of each node), target node matrix (Target node location identifier).

[0048] (2) Reward function: The reward function combines three factors: route hop count, link quality, and end-to-end latency, as detailed below:

[0049] in, This is the hop count weighting coefficient. This is the link quality weighting coefficient. The latency compliance reward coefficient is denoted by path, which represents the current communication path from the source node to the target node and consists of multiple links connected sequentially. This represents any link in the path. For the links in this path signal-to-noise ratio, is a numerical stability constant (to prevent the denominator from approaching zero), delay is the end-to-end delay of the current path, and delay_threshold is the delay threshold value; : The number of hops traversed on the current path; : Indicator function, which takes a value of 1 when the time delay is met, and 0 otherwise; : Reward function for the communication routing planning agent.

[0050] (3) Routing conflict detection and resolution: Define resource contention conflicts when multiple nodes simultaneously select the same relay node.

[0051]

[0052] :time node With nodes The routing conflict detection function between them takes a value of 1 to indicate that a conflict exists and 0 to indicate that there is no conflict; :node At any moment The planned relay node selection; :node At any moment The planned relay node selection; :node At any moment Currently occupied relay node; :node At any moment Currently occupied relay node; : Logical AND operator.

[0053] Conflict detection includes relay node contention conflict detection and relay switching conflict detection. When a conflict is detected, dynamic arbitration and route adjustment are performed based on task priority and urgency. Nodes with higher priority retain the original relay, while nodes with lower priority perform alternative route selection. The alternative route is generated by temporarily marking the link corresponding to the conflicting relay node as unavailable and then re-performing route planning reasoning.

[0054] Upon detecting a conflict, dynamic arbitration and routing adjustments are performed based on task priority and urgency. The arbitration rules are as follows: For task static priority, Due to the urgency of the deadline, This represents the percentage of remaining time. Weighting coefficients:

[0055] Represents a node The comprehensive priority score is obtained by weighting three factors: task static priority, deadline urgency, and remaining time percentage. It is used to compare the priority of different nodes during conflict arbitration.

[0056] Nodes with higher priority retain the original relay, while nodes with lower priority perform alternative route selection. Local detour paths are determined by the D3QN agent of that node after detecting a collision, by adding the conflicting relay node to the link availability matrix. The corresponding link is temporarily marked as unavailable, and the route planning inference is re-executed to generate alternative paths.

[0057] S5: Real-time electromagnetic situation perception, which includes environmental space situation and electromagnetic spectrum situation. The optimal anti-interference strategy is selected through a competitive dual-deep Q network agent, and communication parameters are dynamically adjusted to deal with interference. Step S5 is used to achieve intelligent anti-interference. It involves real-time sensing of the electromagnetic situation (including wireless link sensing information such as CSI and spectrum occupancy), selecting the optimal anti-interference strategy through the D3QN agent, and dynamically adjusting communication parameters to cope with interference.

[0058] In step S5, electromagnetic situation perception includes: fusing environmental perception sensor data from each robot node, along with channel state information and spectrum occupancy detection results reported by the nodes, to construct a comprehensive situation map in real time that includes both the environmental spatial situation and the electromagnetic spectrum situation; modeling anti-interference decision-making as a Markov decision process, with competing dual-deep Q network agents selecting anti-interference strategies based on the current situation; employing a centralized-distributed hybrid architecture for multi-agent collaborative anti-interference, with each node periodically reporting its local situation feature vector to the cluster management node. The management node determines the fusion weight of each node after normalization based on the product of the received signal-to-noise ratio and the confidence score of the perception data, and uses the fusion weight to weight and fuse the local situation feature vectors of each node to generate a global situation before issuing anti-interference action suggestions, which are global suggested actions; each node compares the optimal action output by its local competing dual-deep Q network with the global suggested action, and selects either the local action or the global suggested action as the final decision based on the comparison result. Figure 6 The diagram shown is a flowchart of the intelligent anti-interference communication decision-making process. The intelligent anti-interference communication mechanism constructed in step S5 is as follows: (1) Multi-source perception and situation construction: Integrate multi-source environmental perception data such as lidar, depth camera, and infrared thermal imaging carried by each robot, as well as wireless link perception data such as channel state information (CSI) and spectrum occupancy detection results reported by nodes, and construct a comprehensive situation map in real time, which includes two dimensions: environmental space situation and electromagnetic spectrum situation.

[0059] (2) Anti-interference strategy learning based on D3QN: Anti-interference decision-making is modeled as a Markov decision process. The agent selects the optimal anti-interference strategy (frequency hopping pattern, spreading gain, transmit power, beam direction, etc.) based on the current situation (including CSI features, spectrum occupancy, interference power and other communication-side perception information). Channel state information (CSI) participates in the state space construction as part of environmental perception, realizing the integration of communication-side perception and robot environmental perception at the decision-making level.

[0060] (3) Multi-agent collaborative anti-interference: A centralized-distributed hybrid architecture is adopted. Each node periodically reports its local situation feature vector to the cluster management node. The management node generates a global situation by using attention-weighted fusion and then issues anti-interference action suggestions to each node. The nodes make the final decision based on the local D3QN output.

[0061] Specifically, the management node fusion weight is used to represent the node. The contribution weight of local situation features in global situation fusion: ; in, : Management node fusion weight; Learnable parameters (obtained through joint training with the management node fusion network and updated by the Adam optimizer before each fusion cycle) are used to adjust the scaling of the signal-to-noise ratio and confidence product in the fusion weight calculation. :node The received signal-to-noise ratio, For nodes The confidence score of the perceived data (calculated by the node based on the sensor self-test status and the historical stability of the link).

[0062] Global situation generated by management node fusion . For nodes The local situation feature vector.

[0063] Integration cycle With cluster size satisfy .in, The fusion period is the time interval at which each node reports its local situational feature vector to the management node. : The total number of robot nodes in the cluster.

[0064] The fusion rule for node local decisions and management node global recommendations is as follows: Let the global anti-interference action recommendation issued by the management node be... The optimal action output by the local D3QN is .in, In action space Select the operator that maximizes the function value; Local D3QN network status Next action Q-value estimation.

[0065] Management node's suggested actions The global Q-value estimate is denoted as .

[0066] To eliminate the influence of the Q-value sign on the decision logic, a difference normalization form is adopted: like Then adopt local actions. ; Otherwise, adopt the globally recommended action. .

[0067] in, The local D3QN network is in status. The maximum Q value among all actions, i.e. ; : Confidence threshold, used to control the trade-off between local decision-making and global recommendations, is set to 0.85 in this embodiment; The difference-normalized decision threshold is used to eliminate the influence of the Q-value sign on the decision logic; Maximum value operator.

[0068] This determination method is in The judgment logic remains consistent whether the value is positive, negative, or zero.

[0069] (4) Spectrum sensing and dynamic access: Based on deep reinforcement learning, the spectrum occupancy is sensed in real time, and idle frequency bands are dynamically selected for access, so as to realize the availability utilization of spectrum in the interference environment.

[0070] (5) Autonomous Deployment of Communication Relay Nodes: When a node detects that the SNR with the previous hop node is lower than the threshold of -15dB and there are no available neighbor nodes for more than 10 seconds, it triggers the deployment of the relay module by launching it. The node launches its own small relay module (built-in battery, self-powered for at least 4 hours) through the launch mechanism mounted on the robot's back. The launch distance is adjustable from 5 to 15 meters. Before launching, the target landing point is determined by the laser ranging module, and the landing point deviation is controlled within ±1 meter. The module automatically completes the handshake with neighboring nodes to join the network and expands the network coverage. After the task is completed, the robot retrieves the relay module according to the location marker broadcast periodically. If it cannot be retrieved, it will be used as a one-time communication node until the battery is exhausted.

[0071] S6: Continuously monitors the link quality and energy status of each node. When a node failure, link interruption, or new node addition is detected, topology reconfiguration is automatically triggered.

[0072] Step S6 is used to implement dynamic topology maintenance and reconfiguration. The link quality and energy status of each node are continuously monitored. When a node failure, link interruption, or new node addition is detected, topology reconfiguration is automatically triggered without manual intervention.

[0073] Preferably, the method of this embodiment further includes: S7, uniformly encoding the joint decision variables corresponding to the decision process of establishing a dynamic multi-hop communication network, outputting a joint allocation scheme, and planning a communication route to generate coded individuals; applying random perturbations to each decision dimension with the joint decision variables at the current moment as the center to generate a population containing multiple candidate coded individuals; using a unified manifold approximation and projection method to project the candidate coded individuals from the high-dimensional decision space to a low-dimensional space, constructing a nearest neighbor graph based on the Manhattan distance between coded individuals in the low-dimensional space and optimizing the low-dimensional embedding coordinates; using an adaptive K-means clustering method to automatically divide the projected low-dimensional space into multiple differential subspaces; and performing neighborhood search in parallel in each subspace. Specifically: like Figure 5 The diagram shown is a flowchart of the spatial decomposition and parallel search mechanism according to an embodiment of the present invention. The large-scale cluster optimization mechanism based on spatial decomposition and parallel search is designed as follows: To address the decision space explosion problem caused by large-scale robot swarms, this invention introduces a spatial decomposition and parallel search mechanism. This mechanism is applied to the joint decision space of the three modules: network formation decision, resource scheduling, and route planning. In this invention, the joint decision variables of the D3QN dynamic networking, resource scheduler, and route planning agent are uniformly encoded into a length of [length missing]. Encoded individuals , where each element It corresponds to a discrete or continuous decision variable (such as parent node index, power level, frequency band selection, bandwidth quota, etc.), where l=1,2,…,L, and l represents the position index of the decision variable in the coded individual.

[0074] The specific method for generating the candidate encoded individual population is as follows: Centering on the joint decision variables at the current moment, random perturbations are applied to each decision dimension within a preset proportion range, and samples are generated to form a population containing... A population of candidate encoded individuals is used as input for UMAP dimensionality reduction and K-means subspace partitioning. The upper limit of the perturbation magnitude for each decision dimension is a fraction of its range. Discrete variables are perturbed by integer multiples of the interval between adjacent values ​​to ensure that candidate individuals maintain sufficient coverage diversity while retaining structural similarity to the current decision scheme. UMAP dimensionality reduction and K-means clustering both operate on the encoded individual space, rather than the original high-dimensional state space. Parallel optimization is achieved through dimensionality reduction projection and subspace decomposition. The two-dimensional projection is only used to preserve the topological neighborhood relationships of the solution space to assist in subspace partitioning and does not directly participate in the update calculation of the final optimization variables. The final optimization is still based on neighborhood search of the original encoded individuals in the joint decision space.

[0075] Dimensionality Reduction Projection: The Uniform Manifold Approximation and Projection (UMAP) method is employed. It should be noted that the Manhattan distance formula presented in this invention... It is UMAP that is building it. The nearest neighbor graph stage uses the input space distance metric, not the output space distance after dimensionality reduction.

[0076] in, and Two coded individuals and In the The element value at each position, This is for encoding the length of an individual.

[0077] The core steps of UMAP are as follows: First, construct a model for each individual based on the input distance. Nearest Neighbor Graph (Nearest Neighbor Count) Pick , The candidate encoded individuals population size is used as a reference. Then, the low-dimensional embedding coordinates are optimized by minimizing the cross-entropy loss function, so that the fuzzy topological structure in the low-dimensional space is as consistent as possible with the topological structure of the high-dimensional input space, and finally, a two-dimensional projection result is obtained. The two-dimensional coordinates after UMAP dimensionality reduction are only used to maintain the topological neighborhood relationship of the encoded individuals in the solution space to assist in subspace partitioning. They do not directly participate in the update calculation of the final optimization variables. The final optimization is still based on the original encoded individuals performing neighborhood search in the joint decision space.

[0078] (2) Subspace decomposition: Based on the K-means clustering method with adaptive termination conditions, the projected low-dimensional space is automatically divided into subspaces. Each sample is assigned to a distinct subspace. During the clustering assignment and iterative update phase, this method still uses the geometric centroid distance based on low-dimensional embedding coordinates for sample classification. Specifically, each sample is assigned to the nearest cluster based on the Euclidean distance between its two-dimensional projected coordinates and the geometric centroids (coordinate mean) of each cluster. The cluster center is updated in each iteration step to the geometric mean of the coordinates of the samples within the current cluster, iterating until the cluster center no longer changes significantly. In the termination condition determination phase, a separate fitness evaluation is introduced as a benchmark for measuring the consistency of solution quality within clusters.

[0079] Unlike traditional K-means, which uses a fixed number of iterations or a fixed K value, the adaptive termination condition of this invention is: for any clustering subspace Its depth None of them exceed the overall fitness range ,Right now:

[0080] in, : No. Each cluster subspace; Clustering subspace The first in One coded individual; Fitness function, used to evaluate the quality of encoded individuals; Clustering subspace Middle The fitness value of each encoded individual; Clustering subspace The fitness value corresponding to the centroid, where the centroid is defined as the individual with the best fitness within the cluster; : The set of all encoded individuals The first in Individual; : The set of all encoded individuals Middle The fitness value of each individual; : The set of all encoded individuals; : The set of all encoded individuals The maximum fitness value in the system; : The set of all encoded individuals The minimum fitness value in; Clustering subspace The depth is the difference between the maximum fitness value within the cluster and the fitness value at the cluster center. : Decomposition granularity parameter, used to control the fineness of subspace decomposition; in this embodiment, it is set to 1.4, meaning that the desired subspace depth does not exceed the overall fitness range. (Approximately 71%); This value is in Within the range, a grid search is used for comparative calibration to balance the number of subspaces and intra-cluster consistency; : An operator that takes the maximum value across all clustered subspaces.

[0081] This termination condition ensures that the fitness fluctuation range within each subspace is controlled within the overall fitness range. Within this range, the number of subspaces is determined to ensure good consistency of the "valleys" within each subspace, while preventing excessive subdivision that could lead to an excessively large number of subspaces. Compared to traditional methods with a fixed K value, the adaptive termination condition of this invention can automatically determine a reasonable number of subspaces based on the data distribution, avoiding the subjectivity of pre-specifying the K value.

[0082] (3) Parallel search: Fast exploration of small neighborhoods and deep mining of large neighborhoods are performed in parallel in each subspace: Rapid exploration of small neighborhoods: Three lightweight neighborhood operators, namely, exchange, transfer, and replacement, are designed to quickly scan the effective regions in each subspace; Large neighborhood deep mining: Dedicated neighborhood search operators are designed for communication topology optimization (discrete decision-making) and spectrum power allocation (continuous decision-making) respectively, to achieve deep optimization of the hybrid decision space.

[0083] In one embodiment of the present invention, for a typical industrial inspection cluster scenario comprising 30 robot nodes, the key hyperparameter configurations of the aforementioned agents and algorithms are as follows: the learning rate of the competing dual-deep Q-network. Experience replay capacity M=50000, batch size B=128, soft update coefficient. , decay period =3500 steps. The number of subspaces K is not a fixed value, but is automatically determined according to the adaptive termination condition: when the decomposition granularity parameter λ is 1.4, the corresponding measured number of subspaces ranges from 3 to 8. The random perturbation amplitude of the decision variables is 5% to 15% of their range (discrete variables are perturbed by integer multiples of the interval between adjacent values). The number of nearest neighbors in the UMAP dimensionality reduction process. The population size of candidate encoded individuals The above parameters can be adjusted according to the actual cluster size and scenario requirements through grid search or experimental calibration, and can be determined by those skilled in the art without creative effort.

[0084] In summary, this invention proposes an integrated sensing and computing anti-interference self-organizing network method for multi-robot swarms. It achieves dynamic self-organizing networking by constructing a competitive dual-depth Q-network, optimizes the allocation of communication, sensing, and computing resources through a dual-agent cooperative scheduling framework, addresses the decision space explosion problem in large-scale swarms through spatial decomposition and parallel search mechanisms, and achieves intelligent anti-interference through a centralized-distributed hybrid architecture. This solves the technical problems faced by multi-robot swarms in collaborative operations under complex dynamic environments, such as insufficient communication reliability, spectrum resource competition, lack of self-organizing network capabilities, and low efficiency of communication-sensing-computing collaboration.

[0085] This invention also provides a storage medium storing instructions, which, when read by a computer, cause the computer to execute any of the described methods for integrated sensory computing anti-interference self-organizing networks for multi-robot clusters.

[0086] This invention also provides an electronic device, including a processor and the aforementioned storage medium, wherein the processor executes instructions in the storage medium.

[0087] The examples described herein are merely preferred embodiments of the invention and are not intended to limit the concept and scope of the invention. Any modifications and improvements made by those skilled in the art to the technical solutions of the invention without departing from the design concept of the invention should fall within the protection scope of the invention.

Claims

1. A method for integrated sensing and computing anti-interference self-organizing networks for multi-robot swarms, characterized in that, Includes the following steps: S1: Each robot node broadcasts beacon signals through a multi-mode communication terminal to discover neighboring nodes and build the initial network topology; S2: The cluster management node collects the status information of each node, constructs a status matrix, inputs it into a competitive dual-depth Q network, outputs a networking strategy, and establishes a dynamic multi-hop communication network. The state space of the competitive dual-deep Q network is characterized by a ten-dimensional feature vector, which includes: received signal-to-noise ratio, packet loss rate, hop count to the management node, remaining energy, current communication load, node movement speed, task priority, mean subcarrier gain, delay spread, and Doppler shift. The mean subcarrier gain is the arithmetic mean of the gains of each subcarrier, the delay spread is the square root of the second-order central moment of the power delay spectrum, and the Doppler shift is estimated based on the channel time correlation function. In the competitive dual-deep Q network, each action in the action space corresponds to a combination of networking strategies, which includes parent node selection strategy, transmit power level, modulation and coding scheme selection, and frequency band switching decision. Each action consists of relay selection decision, power level decision, modulation and coding scheme decision, and frequency band switching decision. The relay selection decision serves as the main decision level of the action, while the power level decision, modulation and coding scheme decision, and frequency band switching decision together constitute the auxiliary decision level of the action. The competitive dual-depth Q-network adopts a multilayer perceptron-attention pooling-competitive network architecture. Its input layer receives a variable number of node state vectors. The attention pooling layer compresses the variable-length input into a fixed-dimensional global state representation. After passing through a shared fully connected layer, it enters the competitive network branch to output the Q-value of each action. The Q-value is calculated by the state value function and the advantage function. The value of the advantage function is reduced by its average value over all actions to decouple the state value and the advantage function in the action dimension. The reward function of the competitive dual-deep Q network is constructed based on the overall network performance. The reward function includes a throughput change term, an end-to-end latency change term, a link reliability change term, and a spectrum and energy consumption cost term. The competitive dual-deep Q-network is trained using a priority experience replay mechanism and a soft update mechanism. The target Q-value is calculated using the dual-deep Q-network mechanism, where the online network selects an action based on the state at the next time step, and the target network evaluates the value of the action to calculate the target Q-value. S3: The upper-layer resource scheduling agent outputs a joint allocation scheme of communication bandwidth, computing power quota and perception tasks through the pointer network based on the task pool status and node resource status; the perception tasks include environmental perception tasks and communication perception tasks. S4: The lower-level routing planning agent plans the optimal communication route for each node and avoids relay node contention through conflict detection and dynamic arbitration mechanisms. S5: Real-time electromagnetic situation perception, which includes environmental space situation and electromagnetic spectrum situation. The optimal anti-interference strategy is selected through a competitive dual-deep Q network agent, and communication parameters are dynamically adjusted to deal with interference. S6: Continuously monitors the link quality and energy status of each node. When a node failure, link interruption, or new node addition is detected, topology reconfiguration is automatically triggered.

2. The integrated sensory-computing anti-interference self-organizing network method for multi-robot swarms according to claim 1, characterized in that, The upper-layer resource scheduling agent and the lower-layer routing planning agent constitute a dual-agent collaborative scheduling framework and have a bidirectional coupling relationship; the bandwidth allocation result output by the upper-layer resource scheduling agent serves as the constraint feature input to the state space of the lower-layer routing planning agent. The conflict rate and average end-to-end latency during the routing planning process performed by the lower-layer routing planning agent serve as negative feedback terms for the Critic network reward of the upper-layer resource scheduling agent.

3. The integrated sensory-computing anti-interference self-organizing network method for multi-robot swarms according to claim 1, characterized in that, In step S3, the upper-layer resource scheduling agent adopts a pointer network architecture, which includes an encoder and a decoder. The encoder uses a long short-term memory network to perform context encoding on the input sequence, and the decoder generates resource allocation decisions bit by bit through an attention mechanism, outputting the resource combination obtained by each node. The resource combination includes spectrum bandwidth, computing power quota, and perception task allocation. The upper-layer resource scheduling agent is trained using the Actor-Critic algorithm, where the Critic network is used to estimate the state baseline value to reduce the policy gradient variance.

4. The integrated sensory-computation anti-interference self-organizing network method for multi-robot clusters according to claim 1, characterized in that, In step S4, the lower-layer routing planning agent adopts a competitive dual-depth Q network, and its state representation uses a four-channel matrix. The four-channel matrix includes: a link availability matrix for representing the on / off status of links between nodes, a link quality matrix for representing the estimated signal-to-noise ratio and bit error rate of each link, a node load matrix for representing the current communication queue depth and computational load of each node, and a target node matrix for representing the target node location identifier. The reward function of the lower-layer routing planning agent is constructed based on the routing hop count, link quality, and end-to-end delay.

5. The integrated sensory-computation anti-interference self-organizing network method for multi-robot swarms according to claim 1, characterized in that, In step S4, the conflict detection includes relay node contention conflict detection and relay switching conflict detection. When a conflict is detected, dynamic arbitration and route adjustment are performed based on task priority and urgency. Nodes with higher priority retain the original relay, while nodes with lower priority perform alternative route selection. The alternative route is generated by temporarily marking the link corresponding to the conflicting relay node as unavailable and then re-performing route planning reasoning.

6. The integrated sensory-computation anti-interference self-organizing network method for multi-robot swarms according to claim 1, characterized in that, Also includes: The joint decision variables corresponding to the decision-making process of establishing a dynamic multi-hop communication network, outputting a joint allocation scheme, and planning communication routes are uniformly encoded to generate coded individuals; Centered on the joint decision variables at the current moment, random perturbations are applied to each decision dimension to generate a population containing multiple candidate coded individuals. The candidate coded individuals are then projected from the high-dimensional decision space to a low-dimensional space using a unified manifold approximation and projection method. In the low-dimensional space, a nearest neighbor graph is constructed based on the Manhattan distance between coded individuals, and the low-dimensional embedding coordinates are optimized. An adaptive K-means clustering method is used to automatically divide the projected low-dimensional space into multiple differential subspaces. Neighborhood search is then performed in parallel in each subspace.

7. The integrated sensory-computation anti-interference self-organizing network method for multi-robot clusters according to claim 1, characterized in that, In step S5, the electromagnetic situation perception includes: By integrating environmental perception sensor data from each robot node, as well as channel status information and spectrum occupancy detection results reported by the nodes, a comprehensive situation map including environmental spatial situation and electromagnetic spectrum situation is constructed in real time. Anti-interference decision-making is modeled as a Markov decision process, and competitive dual-deep Q network agents select anti-interference strategies based on the current situation. A centralized-distributed hybrid architecture is adopted for multi-agent collaborative anti-interference. Each node periodically reports its local situation feature vector to the cluster management node. The management node determines the fusion weight of each node after normalization based on the product of the received signal-to-noise ratio and the confidence score of the perceived data. The local situation feature vector of each node is weighted and fused using the fusion weight to generate a global situation and then anti-interference action suggestions are issued. These anti-interference action suggestions are referred to as global suggested actions. Each node compares the optimal action output by the local competitive dual-depth Q-network with the globally suggested action. If the difference between the maximum Q-value of the local competitive dual-depth Q-network and the Q-value corresponding to the globally suggested action is greater than or equal to the decision threshold determined by the confidence threshold and the absolute value of the Q-value corresponding to the globally suggested action, the local action is adopted as the final decision; otherwise, the globally suggested action issued by the management node is adopted as the final decision.

Images