A reinforcement learning-based collaborative autonomous perception and navigation method in a denial environment

Abstract

The application relates to a cooperative autonomous sensing and navigation method in a denial environment, and the method realizes dynamic networking and data sharing among multiple unmanned aerial vehicles (UAVs) in the denial environment through construction of a self-organizing robust communication system, realizes automatic configuration of a communication network under network topology change or communication interference by adopting an adaptive network protocol, processes environmental information and motion information of the UAVs by a data fusion algorithm based on a multi-sensor fusion sensing system, constructs a reward function based on energy consumption of the UAV, exploration area, path overlap degree and path danger degree of the UAV, and executes autonomous sensing and exploration and completes mapping in an unknown environment, and the distributed multiple UAVs exchange and game among individuals in the UAV cluster according to map information, respective positions and working states, realize task allocation, and complete path planning and action decision. The application improves the autonomy and cooperation of the UAV in the denial environment, and provides an effective solution for navigation in a complex environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot autonomous perception and navigation, in particular to a collaborative autonomous perception and navigation method in denial environment based on reinforcement learning. BACKGROUND

[0002] Autonomous navigation technology plays a crucial role in the field of robots and unmanned systems. In recent years, autonomous navigation methods based on deep learning and reinforcement learning have gradually become a research hotspot. Reinforcement learning adjusts the strategy through interaction with the environment to achieve optimal decision-making, which shows great potential in autonomous navigation. Reinforcement learning is a machine learning method that learns the optimal strategy through interaction with the environment. Its core idea is that the agent adjusts the strategy according to the reward feedback from the environment by performing actions in the environment to maximize cumulative rewards. Compared with traditional path planning methods, reinforcement learning does not rely on accurate environment models and can adapt to dynamic environments. In recent years, deep reinforcement learning combines the powerful representation ability of deep learning, making reinforcement learning make breakthrough progress in high-dimensional perception and complex decision-making tasks.

[0003] Denial environment refers to areas that have strong interference or complete shielding to external signals such as GPS. Such environments are common in military applications, post-disaster search and rescue, underground mines, and space exploration. In denial environments, traditional navigation methods such as GPS navigation cannot be used, and map-based navigation methods may fail due to dynamic changes in the environment. Therefore, it is of great practical significance and challenge to study autonomous navigation methods in denial environments.

[0004] Currently, autonomous navigation technology based on reinforcement learning has achieved many results in laboratory environments, but still faces many challenges in practical applications. First, the reinforcement learning training process requires a large amount of interaction data, which may be difficult to obtain in practical applications. Second, the navigation task in complex environments requires higher robustness and real-time performance of the algorithm. In addition, the communication and coordination problem in multi-agent collaboration is also a key difficulty that needs to be solved. SUMMARY

[0005] The purpose of the present application is to overcome the defects of the prior art and provide a collaborative autonomous perception and navigation method in denial environment based on reinforcement learning.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] A collaborative autonomous perception and navigation method in denial environment, the method steps include:

[0008] Construct a self-organizing robust communication system to achieve dynamic networking and data sharing among multiple UAVs in a denied environment, and adopt an adaptive network protocol to achieve automatic configuration of the communication network under network topology changes or communication interference;

[0009] A multi-sensor fusion perception system integrating lidar, vision sensors, and inertial measurement units is adopted to capture and process environmental information and the UAV's own motion information in real time through data fusion algorithms;

[0010] Based on its equipped sensors, a single UAV constructs a reward function using reinforcement learning algorithms, taking into account the UAV's energy consumption, exploration area, path overlap, and path hazard level. This enables autonomous perception and exploration in unknown environments, and the completion of mapping.

[0011] Distributed multi-UAV systems use reinforcement learning algorithms to enable communication and game-playing among UAVs of a specified size based on map information, their respective locations, and operational status, thereby achieving task allocation, path planning, and action decision-making.

[0012] As a preferred technical solution, the self-organizing robust communication system treats the UAV as an independent communication node, constructing a self-organizing UAV network:

[0013] When the drone U i U drones adjacent to it j The signal strength between them is Q ij When the strength threshold is met, the two drones establish stable communication; the edges of the network topology contain the corresponding signal strength Q between the two drones. ij and the gradient G of signal strength over time ij ;

[0014] When the signal strength meets the strength threshold, a strong communication edge is formed between the two UAV nodes. ij When the strength is less than the strength threshold but greater than the connectable strength, a weak communication edge is formed between the two drones;

[0015] In a topological network, drones in subgraphs connected by strong edges exchange information about the current subgraph's topology, clarifying their respective identities and topological connections.

[0016] Each UAV predicts future changes in the network topology based on the signal strength gradient, monitors neighboring nodes connected by weak edges, and deletes or transforms weak edges into strong edges when the communication strength changes.

[0017] As a preferred technical solution, the adaptive network protocol specifically involves the UAV sensing the surrounding wireless spectrum usage and automatically adjusting its communication frequency based on the spectrum occupancy.

[0018] The Q-learning algorithm is used to realize route selection and route establishment, and the minimum hop factor J of the route is:

[0019]

[0020] wherein hop min is the minimum hop of the breadth-first search, and hip max is the maximum hop allowed by the route.

[0021] The queuing delay of the data packet in the network node is used to evaluate the node load, and the M / M / 1 queuing model is used for modeling, and the expected value of the number of data packets in the network layer queue is:

[0022]

[0023] wherein p i represents the probability of the number of data packets in the queue being i, λ represents the rate of receiving data packets of the node, μ represents the rate of processing data packets of the node, and the average queuing delay of the data packet is:

[0024]

[0025] In the case that the rate μ of processing data packets of each node is constant, the greater the rate λ of receiving data packets of the node, the longer the queuing delay of the data packet, and the higher the load of the node.

[0026] Therefore, the rate λ of receiving data packets of the node can be used to measure the load of the network node, and the estimated value of the free degree factor F of a certain node in the network relative to the neighbor node i is as follows:

[0027]

[0028] wherein Z i , Z max is the current load and the maximum load of the network node i.

[0029] The instantaneous reward R obtained by the current node i from the node j is set according to the signal strength Q, the hop factor J and the free degree factor F:

[0030] R(i,j)=ω Q ·Q(i,j)+ω J ·J(i,j)+ω F ·F(i,j)

[0031] wherein ω Q , ω J and ω F are the weights of the signal strength, the hop factor and the free degree factor.

[0032] As a preferred technical scheme, in the self-organizing robust communication system,

[0033] Each UAV stores a copy of the key data and can synchronize information with other UAVs;

[0034] The UAVs that establish communication with each other have a relative motion range constraint before the end of data sharing to avoid communication interruption;

[0035] All UAVs must be authenticated when joining the network to prevent unauthorized access; the system uses data encryption technology, and the communication between UAVs uses digital signature technology to verify the integrity and source of the data.

[0036] As a preferred technical solution, the multi-sensor fusion perception system synchronizes and calibrates the multi-sensor data as follows:

[0037] Joint calibration of laser radar, vision sensor and IMU to obtain the pose transformation relationship between sensors, making the spatial consistency of sensor data coordinates consistent;

[0038] Pre-integrate the IMU to obtain the relative pose transformation relationship of the UAV at the previous and subsequent time, providing motion parameters of the UAV at a short time for the laser radar and vision sensor;

[0039] Using the pose change obtained by the IMU, an interpolation method is used to generate a corresponding pose for each point in the laser radar point cloud, and motion distortion is corrected;

[0040] Capture the environment image through the vision sensor, and use the feature extraction algorithm to obtain the feature points in the image;

[0041] Using the short-time motion parameters provided by the IMU, for the image feature points F captured by the camera, a vector is obtained from the origin of the camera coordinate system to the corresponding spatial position of the feature point, and the strip area through which the line segment is projected in the camera image space is the area where the feature point F may exist, and the matching of the image feature points in the front and rear frames of the vision sensor is performed.

[0042] As a preferred technical solution, the single-UAV mapping is as follows:

[0043] A laser inertial odometer is constructed using the fusion system of laser and IMU, and a vision odometer is constructed using the fusion system of vision sensor and IMU, respectively estimating the motion trajectory of the UAV; the two kinds of pose data are combined to construct a pose optimization problem, and the optimal pose estimation is obtained;

[0044] According to the optimal pose estimation, a point cloud map is created using laser point cloud and image feature points;

[0045] The line surface features captured by the laser point cloud and the image feature points of the visual sensor are used as the feature information of the local map to perform loop detection;

[0046] The map is data compressed, and the dense point cloud structure data is represented by a plane patch;

[0047] The shared loop detection key data is reserved, including the line surface features of the laser point cloud and the visual image features capable of representing the difference between places, and the path loop is formed in the multi-machine map, so that the loop detection is performed across the unmanned aerial vehicle, and the constructed map is shared and fused among the multiple unmanned aerial vehicles.

[0048] As a preferred technical solution, the single unmanned aerial vehicle adopts DDPG deep reinforcement learning, so that the unmanned aerial vehicle learns the local exploration capability in the unknown space, and the reward function includes an energy factor E, an area factor S, an overlap factor C, and a danger factor D, and is constructed as follows:

[0049] R(t)=ω E ·E+ω S ·S+ω C ·C+ω D ·D

[0050] Wherein, ω E , ω S , ω C , ω D are weight parameters of each factor;

[0051]

[0052] S=S t -S t-1

[0053]

[0054] D=||p t -o t || 2 ·I(||p t -o t || 2 <TH D )

[0055] Wherein, E max is the total energy storage of the unmanned aerial vehicle, E t is the current remaining energy of the unmanned aerial vehicle; S t-1 is the mapping area at the last time, S t is the mapping area at the current time; p t is the current position of the unmanned aerial vehicle, p i is the historical exploration trajectory position point, and TH C is the stable perception radius of the unmanned aerial vehicle; ot The TH is the closest obstacle point to the UAV at this time D The UAV safe working radius

[0056] As a preferred technical solution, the distributed multi-UAV converts a natural language task into a specific exploration and navigation task through a large model, and according to the capabilities of the UAV, the priority of the task, and the environmental conditions, specifies the number and capability indicators of the UAVs participating in the task, realizes intelligent decomposition of the task, and transmits it to the distributed multi-UAV through a communication system.

[0057] The large model refines the base model by selecting the base model, a certain number of UAV capabilities, task priority, and environmental condition data, and corresponding task decomposition example data sets to obtain related capabilities.

[0058] As a preferred technical solution, after the distributed multi-UAV receives the task, it uses a reinforcement learning algorithm to communicate and game between individuals according to map information, their own positions, and working states in a specified specification of a UAV cluster, and after the task planning converges, a consensus is reached between the UAVs to realize distributed task planning.

[0059] As a preferred technical solution, the reinforcement learning algorithm is used to realize distributed task planning, which is as follows:

[0060] Combining each UAV U and each exploration direction target V can form a decision space for task allocation {U i ,V j};

[0061] A strategy value function W is constructed to evaluate the expected return brought by the execution of each UAV's own strategy in a state, so as to construct an m x n strategy value matrix; the strategy value function W includes a movement cost and a state cost. In the case of unknown map, the actual distance of the UAV to the target exploration area cannot be accurately calculated, only the planning path length H based on search between the spatial position of the UAV and the boundary of the unknown area is considered as the movement cost; the UAV operation is constrained by energy P, and only the UAV with sufficient energy can complete the complete exploration task;

[0062] W=ω H ·H+ω P ·f(P)

[0063]

[0064] Where, ω H , ω P is the weight of the path length and energy constraint, P TH is the upper threshold of the energy required by the UAV to perform a single task.

[0065] The unmanned aerial vehicles are endowed with sequential codes, each unmanned aerial vehicle sends its own information and the information received from the unmanned aerial vehicles with low order to the unmanned aerial vehicle with the highest order that can be communicated by itself, and finally the information is collected to the unmanned aerial vehicle with the highest order in the unmanned aerial vehicle group; after the information collection is completed, the optimal allocation strategy P is solved by using the strategy value matrix * :

[0066]

[0067]

[0068] x ij = 1 or 0

[0069] Wherein, when the i-th aircraft is assigned the j-th target, x ij = 1, otherwise, x ij = 0;

[0070] After obtaining the optimal allocation strategy, the highest order unmanned aerial vehicle forwards to the low order unmanned aerial vehicle, and the task can be executed after the task forwarding is completed.

[0071] Compared with the prior art, the present application has the following beneficial effects:

[0072] 1) The self-organizing robust communication system constructed by the present application can flexibly and efficiently complete self-organizing and self-adapting networking in denial environment, thereby supporting multiple unmanned aerial vehicles to realize environment exploration more freely and flexibly. The self-organizing network and multi-hop communication technology enable the unmanned aerial vehicles to maintain stable communication connection in complex environment, ensuring efficient transmission and sharing of data. Dynamic routing protocol and spectrum management technology enable the system to adaptively adjust communication path and frequency, adapt to environmental changes, and maintain continuity and stability of communication. Redundant design and fault-tolerant technology improve the stability and reliability of the system when facing node failure or environmental interference, ensuring smooth task performance. Distributed computing and edge computing technology enables computing tasks to be completed near the data source, reducing data transmission delay and improving system response speed. Encryption communication and identity verification technology ensures the security of data transmission in the system, preventing information leakage and unauthorized access.

[0073] 2) The multi-sensor fusion perception system constructed by the present application fuses the perception data of laser radar, visual sensor and IMU three sensors, different modal perception data complement each other, enhances the perception ability and accuracy of the unmanned aerial vehicle to the environment, and the fusion of multiple modal data also improves the positioning and navigation accuracy. Through multi-sensor data fusion, the system can maintain normal operation by relying on the data of other sensors when a single sensor fails or the data quality decreases, improving the robustness of perception. Multi-sensor fusion enables the system to adapt to various complex environments, including indoor and outdoor, light changes, dynamic obstacles, etc., improving the environmental adaptability of the unmanned aerial vehicle.

[0074] 3) The single unmanned aerial vehicle autonomous perception and mapping system constructed by the present application uses reinforcement learning to enable the unmanned aerial vehicle to make autonomous decisions based on environmental feedback, thereby finding the best path in a complex environment, avoiding obstacles, and effectively covering the exploration area, enabling the unmanned aerial vehicle to learn the ability to autonomously explore in an unknown environment, thereby adapting to various complex and dynamic environments, improving exploration efficiency and perception accuracy. The unmanned aerial vehicle generates an explicit map during exploration using SLAM technology, and uses an efficient map representation and storage method, improving the efficiency of the map and improving the efficiency of data sharing and collaborative work between multiple unmanned aerial vehicles.

[0075] 4) The distributed multi-unmanned aerial vehicle task planning and navigation system constructed by the present application uses a large model driven reinforcement learning algorithm to comprehensively evaluate the capabilities of the unmanned aerial vehicle, the priority of the task and the environmental conditions, and the unmanned aerial vehicles achieve close cooperation through a self-organizing communication network, thereby enabling optimal dynamic allocation of tasks, improving resource utilization efficiency, and improving task execution collaboration and consistency. At the same time, the unmanned aerial vehicle quickly adjusts the strategy based on real-time environmental feedback, enabling real-time response to environmental changes and task requirements, improving the flexibility and adaptability of task execution, and reducing the time and cost of task execution. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 A reinforcement learning-based rejection environment collaborative autonomous perception and navigation method according to the present application;

[0077] Figure 2 A self-organizing robust communication topology network according to the present application;

[0078] Figure 3 Optimization diagram for scale estimation of image feature points using short-time motion parameters provided by IMU;

[0079] Figure 4 Diagram for unmanned aerial vehicle pose estimation using visual sensors and laser sensors. DETAILED DESCRIPTION

[0080] The application will be described in detail below with reference to the drawings and specific embodiments. The embodiments are implemented on the premise of the technical solutions of the application, and detailed implementation modes and specific operation processes are given, but the protection scope of the application is not limited to the following embodiments.

[0081] Embodiment 1

[0082] A reinforcement learning-based rejection environment collaborative autonomous perception and navigation method specifically includes the following key subsystems:

[0083] Self-organizing robust communication system. The application constructs a self-organizing robust communication system that can realize dynamic networking and data sharing among multiple unmanned aerial vehicles in a rejection environment. The system uses an adaptive network protocol that can automatically reconfigure under network topology changes or communication interference, ensuring the continuity and reliability of data transmission. In addition, the system also integrates efficient data encoding and error correction mechanisms to improve the accuracy and integrity of data transmission.

[0084] Multi-sensor fusion perception system. In view of the special needs of the rejection environment, the application designs a multi-sensor fusion perception system that integrates laser radar, visual sensor and inertial measurement unit (IMU) three sensors. The system can capture and process environmental information and unmanned aerial vehicle motion information in real time through data fusion algorithms. In order to improve the accuracy and robustness of perception, the application also introduces multi-sensor data synchronization and calibration technology, as well as adaptive feature extraction and information fusion strategy.

[0085] Single unmanned aerial vehicle autonomous perception and mapping system. Using the equipped sensors, the application constructs a single unmanned aerial vehicle autonomous perception and mapping system based on reinforcement learning. The system realizes autonomous perception and autonomous exploration of unmanned aerial vehicles in unknown environments through reinforcement learning algorithms. In the process of unmanned aerial vehicle exploration, SLAM technology is used to generate an explicit map to record the structure and features of the environment, and efficient map representation and storage methods are used. In addition, the system can also share and fuse the constructed map with other unmanned aerial vehicles to achieve more extensive environmental understanding and collaborative navigation.

[0086] Distributed multi-unmanned aerial vehicle task planning and navigation system, the application also proposes a large model driven reinforcement learning-based distributed multi-unmanned aerial vehicle task planning and navigation scheme. Through self-organizing communication network to realize information sharing, using large model according to the ability of unmanned aerial vehicle, task priority and environmental conditions to realize intelligent decomposition of task, using reinforcement learning algorithm to realize path planning and action decision. The cooperation between unmanned aerial vehicles in the navigation process, the system can also respond to environmental changes and task requirements in real time, and realize formation, obstacle avoidance and path following through collaborative control strategy.

[0087] Specifically, each part is implemented as follows:

[0088] As shown in Figure 1 , the self-organizing robust communication system regards the unmanned aerial vehicle as an independent communication node, and constructs an unmanned aerial vehicle self-organizing network (UANET); the UANET has the advantages of self-organization, strong flexibility, strong expansibility, and fast networking speed, and can meet the communication requirements of a multi-unmanned aerial vehicle system;

[0089] The distributed hash table (DHT) technology is adopted to quickly map the network topology structure, so that each unmanned aerial vehicle can know the existence and state of other unmanned aerial vehicles; for the unmanned aerial vehicle U i , the signal strength between the unmanned aerial vehicle U j and its adjacent unmanned aerial vehicle U ij is Q ij ; when Q ij meets a strength threshold value, the two unmanned aerial vehicles can establish stable communication; in the edge of the network topology structure, the signal strength Q ij between the corresponding two unmanned aerial vehicles and the gradient G ij of the signal strength with time are contained:

[0090]

[0091] When the signal strength meets the strength threshold value, a communication "strong edge" is formed between the two unmanned aerial vehicle nodes; when the signal strength Q ij is less than the strength threshold value and greater than a connectable strength, a communication "weak edge" is formed between the two unmanned aerial vehicles, as shown in Figure 2 ;

[0092] In a certain short time, the unmanned aerial vehicles in the subgraph connected by the strong edges in the topology network transmit the topology network information of the subgraph to each other, and explicitly indicate their own identities and topology connection relationships; meanwhile, each unmanned aerial vehicle predicts the change of the topology network at a future time according to the signal strength gradient, monitors the neighbor nodes connected by the weak edges, and deletes or converts the weak edges into strong edges when the communication strength changes, so as to realize dynamic topology network management;

[0093] The topology-based routing protocol is adopted, which is suitable for the characteristics of fast movement and dynamic topology change of the multi-unmanned aerial vehicle system; in a hostile environment, in order to cope with the problems of communication interference and spectrum congestion, the communication conditions can change at any time, and therefore the unmanned aerial vehicle must have the ability of dynamic routing and spectrum management; the unmanned aerial vehicle can perceive the wireless spectrum usage around it, and automatically adjusts its own communication frequency according to the occupation of the spectrum, so that the unmanned aerial vehicle can predict the communication interference and adjust the routing in real time;

[0094] The Q-learning algorithm is used to realize routing selection and routing establishment; first, the minimum hop factor J of the routing is calculated:

[0095]

[0096] where hop min is the minimum hop number of the breadth-first search, hop max is the maximum hop number allowed by the routing; the node load is evaluated using the queuing delay of the data packets within the network node, which is modeled using the M / M / 1 queuing model, and the expected number of data packets in the network layer queue is:

[0097]

[0098] where p i is the probability of having i data packets in the queue, λ is the rate of data packets received by the node, and μ is the rate of data packets processed by the node, and the average queuing delay of the data packets is:

[0099]

[0100] With the rate of data packets processed by the node μ being constant, the greater the rate of data packets received by the node, the greater the queuing delay of the data packets, and the higher the load of the node. The rate of data packets received by the node can be used to measure the load of the network node. The free degree factor of a node i with respect to its neighbor node j is defined as follows:

[0101]

[0102] where Z i , Z max is the current load of the network node i and the maximum load;

[0103] The instantaneous reward obtained by the current node i from the node j is defined according to the signal strength, hop factor and free degree factor as follows:

[0104] R(i,j) = ω Q ·Q(i,j) + ω J ·J(i,j) + ω F ·F(i,j)

[0105] where ω Q , ω J and ω F are the weights of the signal strength, hop factor and free degree factor.

[0106] The data sharing mechanism is an important part of the self-organizing robust communication system, and the system adopts a redundant design and fault-tolerant processing mechanism to ensure the accuracy and integrity of data transmission; in order to improve the robustness of data storage and sharing, each unmanned aerial vehicle can store a copy of the key data and synchronize information with other unmanned aerial vehicles when necessary;

[0107] To ensure smooth data sharing between drones, there are relative motion range constraints between drones that have established communication relationships until the data sharing ends, preventing communication interruptions. The relative motion range refers to the area within which the communication signal strength between the two drones can remain above a certain value. In the absence of obstacles, the communication signal strength is positively correlated with distance; maintaining a straight-line distance between the two drones ensures stable communication. In practice, the two drones can be kept relatively stationary to achieve stable communication.

[0108] Communication security is a critical aspect of this system design. All drones must undergo strict authentication upon joining the network to prevent unauthorized access. The system employs data encryption technology to ensure data security during transmission, and digital signature technology is used for communication between drones to verify data integrity and origin.

[0109] The multi-sensor fusion sensing system is as follows:

[0110] First, the lidar, vision sensor and IMU are jointly calibrated to obtain the pose transformation relationship between each sensor and to achieve spatial consistency of the data coordinates of each sensor.

[0111] By pre-integrating the IMU, the relative pose transformation relationship of the UAV at previous and subsequent time points is obtained, providing the LiDAR and visual sensors with the motion parameters of the UAV at a short time.

[0112] High-frame-rate pose changes obtained by IMU are used to correct motion distortion in LiDAR point clouds, resulting in more accurate LiDAR point clouds. During UAV movement, the coordinate systems of different point clouds generated in the same frame do not coincide; pose changes provided by the IMU are used to correct the point cloud positions. For a point cloud frame P, which contains points {p1, p2, p3, ..., p...} generated in chronological order... n}, which also corresponds to a series of IMU pose data {t1,t2,t3,…,t m}, using interpolation, a corresponding pose is generated for each point in the point cloud, resulting in a point-pose pair (p i ,t i ), i∈[1,n], and further obtain the corrected points:

[0113]

[0114] Environmental images are captured using a visual sensor, and feature extraction algorithms are used to obtain feature points in the images. Short-term motion parameters provided by the IMU are utilized to improve the matching speed and accuracy of feature points between consecutive frames of the visual sensor, aiding in pose estimation and feature point triangulation. Figure 3As shown, for the image feature point F captured by the camera at time t1, there is a corresponding vector pointing from the origin of the camera coordinate system to the spatial location of the feature point. In the IMU coordinate system:

[0115]

[0116] At time t2, the vector in the IMU coordinate system is:

[0117]

[0118] Then the vector in the camera coordinate system is:

[0119]

[0120] The vector projects a line segment in the camera image space. The strip-shaped region that the line segment passes through is the region where feature point F may exist, thus reducing the search space for feature points.

[0121] By utilizing the projection relationship between laser point clouds and environmental images, depth matching is performed to optimize the scale estimation of image feature points and eliminate scale drift in visual matching.

[0122] The specific scheme for autonomous perception exploration and mapping using a single unmanned aerial vehicle (UAV) is as follows:

[0123] A laser-based inertial navigation odometry system is constructed using a fusion system of laser and IMU (Inertial Measurement Unit), and a visual odometry system is constructed using a fusion system of visual sensor and IMU. These systems are used to estimate the UAV's trajectory. The two types of pose data are then combined to construct a pose optimization problem, yielding the optimal pose estimate. Figure 4 As shown, the UAV experiences a series of pose points {x1,x2,x3,…} during operation. The visual sensor and the laser sensor construct pose nodes based on their own perception data to form a pose graph. The pose estimation of the UAV is achieved by using a graph optimization algorithm.

[0124] Based on the optimal pose estimation, a point cloud map is created using laser point cloud and image feature points;

[0125] The bag-of-words data, which consists of line and surface features captured by laser point clouds and feature points from visual sensor images, is used as feature information for local maps to achieve loop closure detection.

[0126] Data compression is performed on the map, and dense point cloud structure data is represented by a planar patch to reduce data volume and improve data sharing efficiency. Meanwhile, key data for loop detection is shared, including line and surface features of laser point cloud and visual image features that can represent differences between locations. Using these shared feature data, the UAV can determine whether the current location has been explored according to its own observation data, form a path loop in the multi-UAV map, and enable loop detection to be performed across UAVs, thereby reducing mapping errors in the multi-UAV system and obtaining consistent map representation.

[0127] DDPG deep reinforcement learning is used to enable the UAV to learn local exploration capability in unknown space. The reward function focuses on four aspects, namely energy consumption, exploration area, path overlap degree and path danger degree, which correspond to energy factor (E), area factor (S), overlap factor (C) and danger factor (D) respectively.

[0128]

[0129] S = S t -S t-1

[0130]

[0131] D = ||p t -o t || 2 ·I(||p t -o t || 2 <TH D )

[0132] wherein E max is the total energy storage of the UAV, E t is the current remaining energy of the UAV; S t-1 is the mapping area at the previous time, S t is the mapping area at the current time; p t is the current position of the UAV, p i is the historical exploration trajectory position point, TH C is the stable sensing radius of the UAV; o t is the nearest obstacle point from the UAV at this time, TH D is the safe working radius of the UAV; the reward function is constructed according to these constraint factors:

[0133] R(t) = ω E ·E + ω S ·S + ω C ·C + ω D ·D

[0134] wherein ω E, ω S , ω C , ω D are weight parameters of each factor;

[0135] A single unmanned aerial vehicle uses its own equipped sensors to create a map when facing an unknown environment, uses reinforcement learning to plan an exploration path according to the current map, and autonomously explores and maps the environment under the constraints of energy, cost and exploration direction.

[0136] Distributed multi-unmanned aerial vehicle task planning and navigation, specifically as follows: a single unmanned aerial vehicle has the functions of autonomous perception, exploration and mapping, and uses a self-organizing robust communication system to realize information exchange and data sharing between unmanned aerial vehicles;

[0137] Through communication between the large model and the operator, the large model converts the natural language task into a specific exploration and navigation task and specifies the number and capability indicators of the unmanned aerial vehicles participating in the task according to the capabilities of the unmanned aerial vehicles, the priority of the task and the environmental conditions, realizes intelligent decomposition of the task, and transmits it to the multi-unmanned aerial vehicle system through the communication system; In order for the large model to have relevant capabilities, a good base model needs to be selected, and a certain amount of data on the capabilities of unmanned aerial vehicles, the priority of tasks and environmental conditions, as well as corresponding task decomposition example datasets, are needed to fine-tune the base model.

[0138] After the multi-unmanned aerial vehicle system receives the task, it uses a reinforcement learning algorithm to exchange and game among individuals in the specified specification of unmanned aerial vehicle cluster according to map information, their own positions and working states, and after the task planning converges, a consensus is reached among the unmanned aerial vehicles, realizing distributed task planning;

[0139] The unmanned aerial vehicle group includes multiple unmanned aerial vehicles {U1, U2, U3, …, U m}, the task includes multiple exploration direction targets {V1, V2, V3, …, V n}, and each unmanned aerial vehicle and each target can form a decision space for task allocation {U i , V j}; a strategy value function W is constructed to evaluate the expected return brought by the execution of each unmanned aerial vehicle's own strategy in a state, thereby constructing an m x n strategy value matrix; the strategy value function W includes a movement cost and a state cost, in the case of unknown map, the actual distance of the unmanned aerial vehicle to the exploration area of the target cannot be accurately calculated, only the planning path length H based on search between its own spatial position and the unknown area boundary is considered as the movement cost; the unmanned aerial vehicle is energy-constrained, and only the unmanned aerial vehicle with sufficient energy can complete the complete exploration task;

[0140] W = ω H ·H + ωP f(P)

[0141]

[0142] wherein ω H , ω P is the weight of path length, energy constraint, P TH is the upper threshold of energy required for a single task by the UAV;

[0143] Each UAV can only directly obtain the result of strategy value calculation between itself and each task, and the overall information can be obtained only by summarizing the calculation results of each UAV; each UAV is assigned with an order code in advance, and each UAV sends its own information and the information received from the UAV with lower order to the UAV with the highest order that can be communicated by the UAV, and the final information is summarized in the UAV with the highest order in the UAV group; after the information is summarized, the optimal allocation strategy P * is solved by using the strategy value matrix, and the mathematical model of the problem is described as follows:

[0144]

[0145] x ij =1 or 0

[0146] wherein x ij is 1 when the ith aircraft is assigned with the jth target, otherwise, x ij is 0; after the optimal allocation strategy is obtained, the UAV with the highest order forwards the task to the UAV with lower order, and the task can be executed after the task forwarding is completed.

[0147] The preferred embodiments of the present application are described in detail above. It should be understood that those skilled in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solution that can be obtained by logical analysis, reasoning or limited experiment based on the existing technology according to the concept of the present application shall be within the protection scope defined by the claims.

Claims

1. A collaborative autonomous perception and navigation method in a denied environment, characterized in that, The method steps include: Construct a self-organizing robust communication system to achieve dynamic networking and data sharing among multiple UAVs in a denied environment, and adopt an adaptive network protocol to achieve automatic configuration of the communication network under network topology changes or communication interference; A multi-sensor fusion perception system integrating lidar, vision sensors, and inertial measurement units is adopted to capture and process environmental information and the UAV's own motion information in real time through data fusion algorithms; Based on its equipped sensors, a single UAV constructs a reward function using reinforcement learning algorithms, taking into account the UAV's energy consumption, exploration area, path overlap, and path hazard level. This enables autonomous perception and exploration in unknown environments, and the completion of mapping. Distributed multi-UAV swarms utilize reinforcement learning algorithms to facilitate communication and game-playing among individual UAVs within a specified swarm size. This is based on map information, individual locations, and operational status, enabling task allocation, path planning, and action decision-making. The specific implementation of distributed task planning using reinforcement learning algorithms is as follows: Each drone and each exploration direction goal The decision space for task allocation is formed by combining pairs of tasks. ; Constructing the policy value function This is used to evaluate the expected benefits of executing each drone's strategy under different conditions, thereby constructing a... Strategy value matrix; strategy value function This includes movement costs and state costs. When the map is partially unknown, the actual distance the UAV travels to the target exploration area often cannot be accurately calculated. Therefore, we only consider the search-based planned path length from the UAV's spatial location to the boundary of the unknown area within the known map area. As a cost of mobility; drone operation is energy-dependent. Only drones with sufficient energy can complete a full exploration mission; in, , These are the weights of path length and energy constraints. It is the upper limit threshold of the energy required for a drone to perform a single mission; The drones are assigned sequential codes, and each drone sends its own information and information received from lower-order drones to the highest-order drone it can communicate with. Finally, the information is aggregated and sent to the highest-order drone in the swarm. After the information is aggregated, the optimal allocation strategy is solved using a policy value matrix. : Among them, when given the first The aircraft is designated as the first When there is a target, If it is 1, otherwise, =0; Once the optimal allocation strategy is obtained, the highest-order drone forwards the task to the lower-order drones, and the task can be executed after the forwarding is completed.

2. The collaborative autonomous perception and navigation method in a denied environment according to claim 1, characterized in that, The self-organizing robust communication system described above treats drones as independent communication nodes, constructing a self-organizing drone network: When drones adjacent drones The signal strength between When the strength threshold is met, the two drones establish stable communication. The edges of the network topology contain the signal strength between the corresponding two drones. and the gradient of signal strength over time ; When the signal strength meets the strength threshold, a strong communication edge is formed between the two UAV nodes. When the strength is less than the strength threshold but greater than the connectable strength, a weak communication edge is formed between the two drones; In a topological network, drones in subgraphs connected by strong edges exchange information about the current subgraph's topology, clarifying their respective identities and topological connections. Each UAV predicts future changes in the network topology based on the signal strength gradient, monitors neighboring nodes connected by weak edges, and deletes or transforms weak edges into strong edges when the communication strength changes.

3. The collaborative autonomous perception and navigation method in a denied environment according to claim 2, characterized in that, The aforementioned adaptive network protocol specifically involves the UAV sensing the surrounding wireless spectrum usage and automatically adjusting its communication frequency based on the spectrum occupancy. The Q-learning algorithm is used for route selection and route establishment, with a minimum hop factor. : in, This represents the minimum number of hops for breadth-first search. The maximum number of hops allowed for a route; Node load is assessed using the queuing delay of packets within a network node. Using an M / M / 1 queuing model, the expected number of packets in the network layer queue is: in, This indicates the number of data packets in the queue. The probability of that time. This indicates the rate at which a node receives data packets. This represents the rate at which a node processes data packets. The average queuing time for data packets is illustrated as follows: Use node receive rate A measure of the load on a network node, relative to its neighboring nodes. The estimated values ​​of the idleness factor are as follows: in, , The current load and maximum load of network node i; Based on signal strength Jump factor and idleness factor Set the current node From node Instant reward obtained for: in, , and It is the weight of signal strength, hop count factor, and idleness factor.

4. The collaborative autonomous perception and navigation method in a denied environment according to claim 1, characterized in that, In the self-organizing robust communication system described above: Each drone stores a copy of critical data and is able to synchronize information with other drones; Before data sharing ends, drones that have established communication with each other are subject to relative motion range constraints to avoid communication interruption. All drones must be authenticated when joining the network to prevent unauthorized access; the system uses data encryption technology, and communication between drones uses digital signature technology to verify the integrity and origin of the data.

5. The collaborative autonomous perception and navigation method in a denied environment according to claim 1, characterized in that, The multi-sensor fusion sensing system synchronizes and calibrates multi-sensor data as follows: The lidar, vision sensor and IMU are jointly calibrated to obtain the pose transformation relationship between each sensor, so as to make the spatial consistency of the data coordinates of each sensor. By pre-integrating the IMU, the relative pose transformation relationship of the UAV at previous and subsequent time points is obtained, providing the LiDAR and visual sensors with the motion parameters of the UAV at a short time. Using the pose changes obtained by the IMU, the corresponding pose is generated for each point in the lidar point cloud using interpolation, and motion distortion is corrected. The environment is captured by a visual sensor, and feature points in the image are obtained using a feature extraction algorithm. Using short-time motion parameters provided by the IMU, image feature points captured by the camera are analyzed. We obtain the vector pointing from the origin of the camera coordinate system to the spatial position corresponding to the feature point, and use the strip-shaped region traversed by the line segment projected by this vector in the camera image space as the feature point. For possible regions, match image feature points in consecutive frames from the visual sensor.

6. The collaborative autonomous perception and navigation method in a denied environment according to claim 1, characterized in that, The specific details of single-UAV mapping are as follows: A laser inertial navigation odometry system was constructed using a laser-IMU fusion system, and a visual odometry system was constructed using a visual sensor-IMU fusion system to estimate the UAV's trajectory. The two pose data were then combined to construct a pose optimization problem and obtain the optimal pose estimate. Based on the optimal pose estimation, a point cloud map is created using laser point cloud and image feature points; Loop closure detection is performed using bag-of-words data, which consists of line and surface features captured by laser point clouds and feature points from visual sensor images, as feature information of the local map. The map data is compressed, and dense point cloud structure data is represented by planar patches; Key data for shared loop closure detection is retained, including linear and planar features of laser point clouds that can characterize location differences, as well as visual image features. Path loops are formed within a multi-drone map, enabling loop closure detection to be performed across drones and allowing the constructed map to be shared and merged among multiple drones.

7. The collaborative autonomous perception and navigation method in a denied environment according to claim 1, characterized in that, The single UAV employs DDPG deep reinforcement learning, enabling it to learn local exploration capabilities in unknown spaces. The reward function includes an energy factor. Area factor Overlap factor Risk factors The structure is as follows: in, , , , These are the weight parameters for each factor; in, For the total energy storage of drones, This represents the drone's current remaining energy. This represents the mapping area at the previous moment. This represents the current mapping area. The current location of the drone. As a location point for historical exploration, To stabilize the sensing radius of the drone; This is the closest obstacle point to the drone at this moment. The safe operating radius for drones.

8. The collaborative autonomous perception and navigation method in a denied environment according to claim 1, characterized in that, The distributed multi-UAV system transforms natural language tasks into specific exploration and navigation tasks through a large model. Based on the UAV's capabilities, task priority, and environmental conditions, it specifies the number and capability indicators of the UAVs participating in the task, thereby achieving intelligent task decomposition. The task is then transmitted to the distributed multi-UAV system through a communication system. The large model is obtained by selecting a base model and fine-tuning the base model with a certain number of drone capabilities, task priorities, environmental condition data, and corresponding task decomposition example datasets.

9. The collaborative autonomous perception and navigation method in a denied environment according to claim 1, characterized in that, After receiving the task, the distributed multi-UAVs use reinforcement learning algorithms to communicate and compete among themselves in a UAV cluster of specified size based on map information, their respective locations and working statuses. After the task planning converges, the UAVs reach a consensus and realize distributed task planning.

Images