Unknown space collaborative exploration system based on air-ground heterogeneous robot

Abstract

The invention relates to an unknown space collaborative exploration system based on an air-ground heterogeneous robot, which comprises an unmanned vehicle and a plurality of unmanned aerial vehicles loaded on the unmanned vehicle, and is characterized in that the unmanned vehicle is provided with a first sensing module and a first computing power platform which are used for constructing a refined map for the environment; the unmanned aerial vehicle is provided with a second sensing module and a second computing power platform which are used for constructing a lightweight map for the environment; wireless communication modules are arranged in the unmanned vehicle and the unmanned aerial vehicle, and a semi-centralized self-organizing communication mode is adopted for communication; when the unmanned aerial vehicle performs collaborative operation to explore an unknown space, the unmanned aerial vehicle explores to an unknown area around by taking the unmanned vehicle as a center, constructs a lightweight map based on a second sensing module and a second computing power platform of the unmanned aerial vehicle and returns the lightweight map to the unmanned vehicle; and the unmanned vehicle fuses the lightweight map transmitted by the unmanned aerial vehicle and a refined map constructed by the unmanned vehicle through the first sensing module and the first computing power platform. Compared with the prior art, the method has the advantages of high environmental adaptability, high reliability and the like.

Description

Technical Field

[0001] The present invention relates to the field of unknown space exploration, and in particular to an unknown space collaborative exploration system based on air-ground heterogeneous robots. Background Art

[0002] Robots can not only improve the efficiency of exploring unknown spaces, but also ensure the safety of personnel in dangerous environments or environments that are difficult for humans to reach. They are of great significance to scientific research, military reconnaissance, disaster relief, etc.

[0003] However, due to the complexity and unpredictability of unknown space, traditional single robot systems often have problems such as incomplete information acquisition and low efficiency in task execution when facing complex and changing unknown environments. For example, drones, ground robots or underwater robots usually have specific functions and operating environments, but when facing complex unknown spaces, single robots often find it difficult to complete comprehensive and efficient exploration tasks. They can often only perform tasks in specific environments and find it difficult to cope with diverse exploration needs. For example, although drones can quickly cover large areas, their maneuverability and stability are limited in narrow or obstacle-filled environments; ground robots perform well in terrain adaptability, but are difficult to function in air or underwater environments.

[0004] Therefore, developing a robot system that can adapt to complex environments and has efficient collaborative capabilities has become an important direction of current technological development.

[0005] Heterogeneous robot systems, that is, systems composed of different types of robots, can play their respective advantages. They can complement each other's functions and achieve more flexible and efficient task execution. For example, ground robots have advantages in terrain adaptability and load capacity, while aerial robots have outstanding performance in field of view and maneuverability. The collaborative work of heterogeneous robot systems can achieve information complementarity and optimal task allocation. Heterogeneous robot collaborative exploration has great potential, but achieving this goal requires the joint support of multiple technologies.

[0006] Although existing heterogeneous robot systems have made some progress in some aspects, there are still some limitations. For example, the reliability and stability of the communication system still need to be improved, the accuracy and efficiency of environmental perception and mapping technology need to be further improved, and task allocation and path planning need to be more intelligent and adaptive. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide an unknown space collaborative exploration system based on air-ground heterogeneous robots with strong environmental adaptability, accurate and efficient environmental mapping, and high reliability.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] A system for collaborative exploration of unknown space based on air-ground heterogeneous robots, comprising an unmanned vehicle and a plurality of unmanned aerial vehicles mounted on the unmanned vehicle, wherein the unmanned vehicle is equipped with a first perception module and a first computing platform for building a refined map of the environment, and each of the unmanned aerial vehicles is equipped with a second perception module and a second computing platform for building a lightweight map of the environment;

[0010] When the system performs collaborative operations to explore unknown spaces, the drones explore the surrounding unknown areas with the unmanned vehicle as the center, and build a lightweight map based on the drone's own second perception module and the second computing power platform and send it back to the unmanned vehicle. The unmanned vehicle integrates the lightweight maps received from multiple drones and the refined map built by the unmanned vehicle itself through the first perception module and the first computing power platform;

[0011] The unmanned vehicle and the unmanned aerial vehicle are both provided with wireless communication modules, and the unmanned vehicle and the unmanned aerial vehicle communicate in a semi-centralized self-organizing communication manner.

[0012] Preferably, the unmanned vehicle and the unmanned aerial vehicle are both provided with wireless communication modules, and the unmanned vehicle and the unmanned aerial vehicle communicate using a semi-centralized self-organizing communication method, specifically:

[0013] With the unmanned vehicle as the central communication node and the drone as the communication node, the drone and the unmanned vehicle are directly or indirectly connected to form an unmanned vehicle central communication network;

[0014] When the communication between the UAV and the UAV center communication network is interrupted, the unmanned vehicle determines whether there is an idle UAV at this time; if there is an idle UAV, the idle UAV is dispatched to navigate to the set position as a communication relay node; if there is no idle UAV, the disconnected UAV spontaneously searches around for other communication nodes that can establish communication. If other communication nodes that can establish communication are found within the set time interval, the communication connection is directly established. If other communication nodes that can establish communication are still not found within the set time interval, the disconnected UAV returns to the previous position where it could communicate with the UAV center communication network; among them, other communication nodes that can establish communication are communication relay nodes.

[0015] Preferably, the communication relay node moves only in a known environment map, and uses a search-based algorithm to calculate the optimal path connecting the two previous and next communication nodes;

[0016] The communication node farthest from the central communication node on the communication link is taken as the terminal communication node. The terminal communication node is responsible for the perception task, and the target position of the relay communication node is as close to the next node as possible along the optimal path;

[0017] After calculating the connection path between the front and rear nodes, the UAV as a relay communication node starts from its current position and flies along the connection path from the position close to the upper node to the lower node. At the same time, the communication module monitors the strength of the communication signal. When the strength is less than a given threshold, the UAV as a relay communication node lands and performs the communication relay task.

[0018] Preferably, at most one drone acting as a communication relay node is allowed between the unmanned vehicle and the drone in front for perception.

[0019] Preferably, the first perception module includes a visual sensor for capturing visual information of the surrounding environment, a lidar for modeling the environment into a dense environmental point cloud map, and an inertial measurement unit for sensing short-term motion data; the second perception module includes a visual sensor for lightweight mapping of the entire environment, and an inertial measurement unit for sensing short-term motion data.

[0020] Preferably, the unmanned vehicle fuses the lightweight maps received from the multiple unmanned aerial vehicles and the refined maps constructed by the unmanned vehicle itself through the first perception module and the first computing platform. The specific fusion process is:

[0021] Based on the lightweight map, short-term motion data and key visual frame data sent back by the drone, dense visual low-precision mapping is performed;

[0022] The spatial point cloud scaling and registration method is used to adjust the pose in the lightweight map sent back by the drone, delete the low-precision map in the overlapping area, align the boundaries of the refined map constructed by the unmanned vehicle, and obtain a double-precision map;

[0023] As the unmanned vehicle moves towards the lightweight map, it continues to use its own first perception module to continuously build maps, generate new refined maps, and continuously move the boundary between the refined map and the lightweight map backwards.

[0024] Preferably, a low-frequency update strategy is adopted. Every time the unmanned vehicle travels a certain distance, the point cloud scaling and registration method is reused to correct the deviation at the boundary according to the high- and low-precision map boundaries.

[0025] Preferably, the double-precision map is represented as an octree map for path planning, and the undulating slope of the ground is calculated based on the ground data, and the undulating slope is used as a cost item in the path planning of the unmanned vehicle.

[0026] Preferably, when the collaborative operation explores the unknown space, the drone explores the unknown area around the unmanned vehicle, specifically including:

[0027] When multiple unknown boundaries appear in the map, the first computing platform on the unmanned vehicle calculates and outputs the unknown boundary number assigned to the unmanned vehicle for exploration in the next step through a reinforcement learning algorithm based on the fused map of the explored area and the set of current unknown boundaries, and assigns a drone to explore each of the remaining secondary unknown boundaries;

[0028] When the drone drives away from the unmanned vehicle, the unmanned vehicle stops and waits, and decides the next exploration boundary based on the drone’s exploration results;

[0029] Among them, the reinforcement learning algorithm uses the path length L and the time T passed by the unmanned vehicle as the reward and punishment items of reinforcement learning, dynamically optimizes the allocation of unknown boundaries, and through the structural distribution characteristics of the learning environment, enables the unmanned vehicle to select the unknown boundary with the highest exploration value score among multiple unknown boundaries for exploration.

[0030] Preferably, if there is only one unknown boundary in the map, the exploration is carried out in the single-unit exploration mode of the unmanned vehicle, and the unknown boundary tracking algorithm is used to complete the exploration task, specifically: the geometric center of the unknown boundary is used as the target point for path planning, and a search-based path planning method is used to obtain a walking path to the target boundary; wherein, during the continuous operation of the unmanned vehicle, the unknown boundary continues to retreat, and it is stipulated that the unmanned vehicle calculates the target position once every certain distance traveled

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] (1) Improving collaborative exploration capabilities: The present invention realizes comprehensive exploration of unknown space by constructing a collaborative system of ground unmanned vehicles and aerial drones. As a mobile base, the unmanned vehicle not only provides a platform for takeoff and landing for the drone, but also uses its onboard sensors and computing power platform to perform detailed modeling of the environment. The drone uses lightweight sensors and computing power platforms to operate for a long time with low power consumption, realizing lightweight mapping of areas outside the field of view of the unmanned vehicle. This collaborative operation mode enables the entire system to cover a wider space, provide richer environmental information, and significantly improve the comprehensiveness and efficiency of exploration.

[0033] (2) Stable communication and efficient data transmission guarantee: The centralized self-organizing communication system of the present invention ensures the stability and efficiency of data transmission between unmanned vehicles and drones. The unmanned vehicle serves as the communication center node, and the drone establishes communication with the unmanned vehicle in a direct or indirect manner to achieve real-time transmission of perception data and operation commands. Even if the communication between some drones and unmanned vehicles is blocked, the continuity of data transmission can be maintained by using other drones as communication relays, thereby ensuring the robustness and reliability of the entire system.

[0034] (3) Intelligent task allocation and dynamic path planning: The autonomous exploration system of the drone swarm can intelligently perform task allocation and path planning based on the real-time environmental information and the status of the drones. By segmenting unknown boundaries and dynamically allocating tasks, drones can collaborate efficiently and achieve rapid exploration of unknown areas. At the same time, drones complete lightweight mapping in real time during the exploration process and transmit key information back to the unmanned vehicle. The unmanned vehicle then performs map fusion and boundary information updates based on this information to further optimize the drone's exploration tasks and paths.

[0035] (4) Double-precision map generation and environmental understanding: The double-precision mapping and navigation system of the ground unmanned vehicle combines high-precision SLAM technology and visual SLAM technology to generate a double-precision map containing high-precision and lightweight information. This map not only provides fine details of the environment, but also covers a wider area, providing more comprehensive navigation information for the unmanned vehicle. The wide field of view of the drone helps the unmanned vehicle understand the environmental structure and the traffic conditions ahead, allowing the unmanned vehicle to make path planning and navigation decisions more accurately.

[0036] (5) High degree of autonomy and intelligent operation: The system of the present invention has a high degree of autonomy and intelligence, and can make autonomous decisions and execute tasks in unknown environments. The combination of lightweight mapping of drones and high-precision mapping of unmanned vehicles enables the system to adapt to changing environmental conditions and achieve more flexible and intelligent exploration strategies. The collaborative operation of unmanned vehicles and drones reduces dependence on human intervention, improves the system's autonomous operation capabilities, and also reduces the complexity and risk of operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the hardware composition of the unknown space collaborative exploration system based on air-ground heterogeneous robots in the embodiment;

[0038] Figure 2 A schematic diagram of a semi-centralized communication topology structure in an embodiment;

[0039] Figure 3 A schematic diagram of a communication relay node deployment method in an embodiment;

[0040] Figure 4 This is a schematic diagram of unknown boundary allocation in an embodiment. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0042] Example

[0043] This embodiment provides an unknown space collaborative exploration system based on air-ground heterogeneous robots, including an unmanned vehicle and multiple unmanned aerial vehicles mounted on the unmanned vehicle, the unmanned vehicle is equipped with a first perception module and a first computing power platform for building a refined map of the environment, and the unmanned aerial vehicle is equipped with a second perception module and a second computing power platform for building a lightweight map of the environment; both the unmanned vehicle and the unmanned aerial vehicle are provided with wireless communication modules, and the unmanned vehicle and the unmanned aerial vehicle communicate in a semi-centralized self-organizing communication mode;

[0044] When the system works collaboratively to explore unknown space, the drones explore the surrounding unknown areas with the unmanned vehicle as the center, build a lightweight map based on the drone's own second perception module and the second computing power platform and send it back to the unmanned vehicle. The unmanned vehicle integrates the lightweight maps received from multiple drones and the refined map built by the unmanned vehicle itself through the first perception module and the first computing power platform.

[0045] Next, the system of this embodiment is introduced in detail.

[0046] (1) Hardware architecture of the unknown space collaborative exploration system

[0047] like Figure 1 As shown, the hardware part of the unknown space collaborative exploration system includes an unmanned vehicle S01 and multiple unmanned aerial vehicles S02.

[0048] The unmanned vehicle S01 is equipped with a perception module. In this embodiment, the perception module adopts a multimodal perception device, including: a horizontally rotating 3D laser radar S04 for modeling the environment into a dense environmental point cloud map, a surround view camera S05 for capturing visual information of the surrounding environment, and an inertial measurement unit IMU for sensing short-term motion data. The surround view camera S05 is installed under the drone and can rotate the observation angle, allowing the drone to adjust the viewing angle in both perception and take-off and landing working conditions. The above three sensors complement each other and integrate perception to avoid the degradation of a single sensor.

[0049] A flat platform S03 for parking drones is set on the top of the unmanned vehicle S01. A QR code S06 for calculating relative posture is attached to the surface of the flat platform S03. When the drone S01 takes off and lands, it captures the QR code through the surround camera S05, calculates its posture relative to the unmanned vehicle, and plans a safe and fast movement path.

[0050] Both the unmanned vehicle S01 and the unmanned aerial vehicle S02 are equipped with powerful computing platforms, which are used to build refined maps and lightweight maps respectively.

[0051] Unmanned vehicles need to carry drones for long-distance operations. At the same time, as a communication center and perception data processing center, they consume a lot of energy and need to be equipped with larger energy storage batteries.

[0052] Both unmanned vehicles and drones are equipped with wireless communication modules.

[0053] (2) Ground-to-air semi-centralized self-organizing communication and communication node scheduling

[0054] When the system is working, the ground unmanned vehicle is always used as the working center. Both the unmanned vehicle and the drone are equipped with wireless communication modules to form communication network nodes, which are suitable for the use of a centralized network structure. The unmanned vehicle is used as the central node of the communication network, and the data of each drone is directly transmitted to the unmanned vehicle to form a centralized communication network. However, in a complex environment, the flying drone may lose communication with the central node unmanned vehicle, resulting in communication failure. For this reason, when the drone cannot communicate with the central node unmanned vehicle, it is required to be able to spontaneously search for communicative objects around and establish a connection with another communication node to form a local distributed network structure. In this way, a semi-centralized communication network is formed, and the drone explores around with the unmanned vehicle as the center, allowing communication interruption between edge nodes and central nodes within a certain period of time and local communication between edge nodes in a small range, thereby improving the flexibility and stability of the system. The drone or drone group that loses connection with the central node plans the time to return according to its own work needs, thereby restoring the connection with the central node.

[0055] Specifically, the unmanned vehicle is used as the central communication node, the drone is used as the communication node, and the drone and the unmanned vehicle are directly or indirectly connected to form an unmanned vehicle central communication network; when the communication between the drone and the drone central communication network is interrupted, the unmanned vehicle determines whether there is an idle drone at this time; if there is an idle drone, the idle drone is dispatched to navigate to the set position as a communication relay node; if there is no idle drone, the disconnected drone spontaneously searches around for other communication nodes that can establish communication, and if other communication nodes that can establish communication are found within the set time interval, the communication connection is directly established; if other communication nodes that can establish communication are still not found within the set time interval, the disconnected drone returns to the previous position when it could communicate with the drone central communication network; among them, the other communication nodes that can establish communication are communication relay nodes.

[0056] In this embodiment, the unmanned system (drone or unmanned vehicle) is set as a communication node, and UANET is used to build a self-organizing communication network; UANET has the advantages of self-organization, strong flexibility, and fast networking speed, and can meet the communication requirements of multiple unmanned systems.

[0057] like Figure 2 As shown in the figure, the unmanned vehicle U0 is the central node of the communication network, and the drone U i (i=1,…,n) is directly connected to the unmanned vehicle through wireless communication, forming a star topology network structure, which is expressed as:

[0058] G=<V,E>

[0059] V={U i |i=0,1,…,n}

[0060] E={(U0,U i )|i=1,…,n}

[0061] When an independent drone group appears, each drone in the group communicates with each other separately, forming a communication network G independent of the central node. ′ :

[0062] G ′ = <V ′ ,E ′ >

[0063] V ′ = {U p ,U q}

[0064] E ′ ={(U p ,U q )}

[0065] When the independent drone group moves away from or approaches the central network, the connection between the two drones that are close to each other is disconnected or reconnected; when the connection exists, a multi-hop chain topology is formed between the end drone and the center unmanned vehicle, and the drones in the chain play the role of communication relay between the center and the end; when drones fly in the environment, the communication signal strength is greatly affected by the environmental structure. In a complex environment, the signal is weak when the drone is blocked by obstacles during the exploration process, and data transmission requires good communication conditions. At this time, this type of communication network structure is often required;

[0066] When there are drones in the communication network that only play the role of communication relay, their relative positions relative to the nodes at both ends have a significant impact on the communication effect. They need to dynamically change their own positions according to the position changes of the nodes at both ends, that is, the autonomous scheduling of the communication nodes. The unmanned vehicle travels in the environment and uses its own sensors to create a map of the surroundings. The drone navigates based on the unmanned vehicle map and explores unknown boundaries to obtain more maps. When the local map is known, the drone can realize its own positioning and navigation based on the sensors and maps, and further, it can move autonomously to the specified position according to the instructions. This is the movement basis for realizing the autonomous scheduling of communication nodes. The unmanned system monitors the strength of the communication signal through the communication module, and can evaluate the connection status between each other, thereby providing decision-making information for the autonomous scheduling of communication nodes.

[0067] like Figure 3 As shown, the relay node U i Only move in the known environment map M, and use the search-based algorithm to calculate the connection between the two nodes U in the two-dimensional rasterized map. i-1 and U i+1 The optimal path; since the end node is responsible for the perception task and has a large range of movement, it is necessary to ensure good communication conditions during the large-scale movement as much as possible. Therefore, the target position of the relay node needs to be as close to the next node as possible along the optimal path; after calculating the connection path between the front and rear nodes, the relay node drone starts from the current position from a position close to the upper node and flies along the connection path to the lower node, while monitoring the communication signal strength. When the strength is less than a given threshold, the relay node lands to perform the communication relay task.

[0068] (3) Autonomous exploration of drones centered on ground unmanned vehicles

[0069] When a single unmanned vehicle performs an exploration mission, if a large blind spot appears in the environment, the unmanned vehicle needs to change its original direction of movement and go to a place that can cover the blind spot, or when a branch structure appears in the environment, the unmanned vehicle needs to go to different branches in turn to expand its field of vision and judge the road conditions ahead; in both cases, the unmanned vehicle needs to make multiple round trips; for a multi-unmanned system with ground-air collaboration, the unmanned vehicle carries the drone to explore together in an unknown environment. When the environment is empty and there are no branch paths, the sensors carried by the unmanned vehicle itself are enough to build a good map of the environment; in the face of a blind spot, the unmanned vehicle can maintain its original movement trajectory and the drone can go to cover it; in the face of a forked structure, the drone can go to roughly explore and help the unmanned vehicle choose a better path forward;

[0070] In a simple environment, there is only one major unknown boundary F in the map M. The system explores in single-unit mode and can complete the exploration task using only a simple algorithm for tracking the unknown boundary.

[0071] For the unknown boundary F, find its geometric center as the target point P for path planning target ;

[0072] P target =Center(F)

[0073] A search-based path planning method is used to obtain a walking path to the target boundary. During the continuous operation of the unmanned vehicle, the position boundary F will continue to retreat. Repeated calculation of the target position will increase the power consumption of the computer. At the same time, there are large or small differences in the target position each time, which may cause the unmanned vehicle to sway in the forward direction, resulting in energy loss in motion. Therefore, it is stipulated that the unmanned vehicle calculates the target position once every certain distance to reduce the update frequency of the target position.

[0074] like Figure 4 As shown in the figure, in a complex environment, blind spots or forks appear in the exploration area, and more than one unknown boundary F will appear in the map M. i (i=1,2,…), different unknown boundaries have different exploration costs and exploration values ​​(such as the length of the path to the unknown boundary, the possibility that the unknown boundary is a closed area and needs to turn back; the possibility of exploring the size, openness and shape of the space behind the unknown boundary), and the unknown boundaries need to be evaluated and sorted according to their status; the unmanned vehicle is the center of the exploration team, and the boundaries with the most exploration value should be assigned to the unmanned vehicle, and the secondary boundaries should be assigned to the drone for exploration;

[0075] The exploration cost and value of unknown boundaries are difficult to quantify explicitly. Based on the actual application scenario, a simulation scenario is designed, and reinforcement learning is used to learn the distribution characteristics of the environmental structure in the scenario, so as to select the boundary suitable for the unmanned vehicle to explore among multiple unknown boundaries.

[0076] In the virtual scene, the communication and perception process is simplified. It is assumed that there is no obstruction in the communication between unmanned systems. The perception of the unmanned system is equivalent to the field of view coverage, which speeds up the operation. The virtual ground-air collaborative system is used for exploration. The reinforcement learning model f(·) receives the map M of the explored area and the current unknown boundary set {F i |i=1,2,…}, output the unknown boundary number ID to be assigned to the unmanned vehicle for exploration in the next step, and the remaining unknown boundaries are left to the drone to explore; after exploring all scenes, the path length L and the time T taken by the unmanned vehicle are used as the reward and punishment items of reinforcement learning, so that the system can learn the selection strategy of the unknown boundary:

[0077] ID=f(M,{F i})

[0078] R=-α(L+T)

[0079] The secondary boundaries are explored by drones. Constrained by communication and movement capabilities, drones move within a range centered on the unmanned vehicle. One drone is assigned to explore each secondary boundary. When the drone leaves the unmanned vehicle, the unmanned vehicle stops and waits, which can not only maintain the stability of the communication environment, but also further determine the next boundary to be explored based on the exploration results of the drone. The unknown boundary tracking method is also used to first find the centroid of the unknown boundary, and then use the search-based path planning algorithm to generate a path to guide the drone to explore. The drone uses a camera for environmental perception, which has a narrow field of view and needs to move back and forth along the unknown boundary to cover the coverage area, gradually moving the unknown boundary back, while creating a map and recording key visual information, and transmitting it to the unmanned vehicle. The drone uses the existing visual-inertial sparse mapping method, which mainly serves its own positioning and perception of the general structure of the environment. When the drone is far away from the unmanned vehicle and the communication rate drops below the standard, the system will send an additional drone to undertake the communication relay task according to the method proposed in Example 1, thereby increasing the exploration range of the drone in front.

[0080] The number of drones is limited, and the maximum number of explorations of the secondary unknown boundary does not exceed the number of drones. When there are idle drones, at most one drone is allowed to act as a communication relay between the unmanned vehicle and the front perception drone. When there are no idle drones, the perception drone has no support from the relay drone and can only perform exploration tasks within the signal range of the unmanned vehicle.

[0081] The maximum range that the forward sensing drone can explore depends on the communication capabilities of the unmanned vehicle and the communication relay drone. The unknown area behind the unknown boundary is either a small closed space that can be completely covered by the exploration range and no new unknown boundaries will be generated, or a large closed space or open space that is beyond the exploration range and still has new unknown boundaries;

[0082] After the drone completes its exploration, the unmanned vehicle makes a decision based on the new unknown boundary and heads to the next unknown boundary with the greatest exploration value. The drone returns and lands on the unmanned vehicle, and they head there together, waiting for the next explorable multi-unknown boundary situation.

[0083] (4) Dual-precision mapping and navigation for ground unmanned vehicles

[0084] The ground unmanned vehicle is equipped with laser radar, visual sensors and IMU, and can autonomously create high-precision dense maps using existing multi-sensor fusion methods; however, this high-precision map can only be generated on the working path of the unmanned vehicle; the area explored by the drone alone only has sparse maps sent by the drone, which cannot support the navigation of the unmanned vehicle; the environment needs to be reconstructed visually from the key visual frames and their positions saved by the drone to form a dense map; the original information of the dense map comes from the monocular camera equipped by the drone, which is less accurate than the laser sensor and has the inevitable scale drift of the monocular system, so it is a low-precision map;

[0085] The two precision maps are generated by the perception data of unmanned vehicles and drones respectively. They cover different areas in space and overlap partially. Due to the mapping error, the two maps may have deviations in the positions of obstacle point clouds and often cannot be accurately aligned. In order to improve the overall representation performance of the dual-precision map, it is necessary to retain the high-precision map in the overlapping area as much as possible and delete the low-precision map. Therefore, it is necessary to adopt the spatial point cloud scaling and registration method to adjust the posture of the low-precision map so that the two maps can be accurately aligned at the boundary of the high-precision map.

[0086] The above two types of maps are dense maps, and can be converted into octree space occupancy maps using the same method to form a unified representation, and use the same algorithm to complete path planning; in the unified octree map representation, the ups and downs of the ground are recorded, and the ups and downs of the ground slope G are calculated using the local gradient changes of the ground part. The road slope is used as a cost item for path planning, which reduces the ups and downs of the planned path, provides a smooth passage path for the unmanned vehicle, and reduces energy consumption;

[0087] The initial slope at coordinate (x,y) is defined as g(x,y), where s is the slope calculation step size:

[0088]

[0089] The initial slope is greatly affected by the high-frequency information of the terrain and is not easy to represent the low-frequency terrain undulation information. The initial slope g(·) is subjected to Gaussian filtering using two-dimensional Gaussian convolution K and to eliminate the influence of high-frequency information to obtain the undulating slope G.

[0090] G(x,y)=conv(g(·) x-ns:x+ns,y-ns:y+ns ·K (2n+1)×(2n+1) )

[0091] Due to the difference in precision, the unmanned vehicle can achieve accurate self-positioning in the map it creates, but there will be large errors in the low-precision map; in the process of moving towards the low-precision map, the unmanned vehicle still uses its own sensors to continuously build maps and generate new high-precision maps, thereby further pushing the boundary between the high-precision and low-precision maps backwards; using a low-frequency update strategy similar to that in Example 3, every time the unmanned vehicle travels a certain distance, according to the boundary of the high- and low-precision maps, the point cloud scale scaling and alignment are reused to correct the deviation at the boundary, ensuring that the unmanned vehicle can continue to use the low-precision map for path planning.

[0092] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An unknown space collaborative exploration system based on air-ground heterogeneous robots, characterized in that: It includes an unmanned vehicle and a plurality of unmanned aerial vehicles mounted on the unmanned vehicle, wherein the unmanned vehicle is equipped with a first perception module and a first computing power platform for building a refined map of the environment, and the unmanned aerial vehicle is equipped with a second perception module and a second computing power platform for building a lightweight map of the environment; both the unmanned vehicle and the unmanned aerial vehicle are provided with a wireless communication module, and the unmanned vehicle and the unmanned aerial vehicle communicate in a semi-centralized self-organizing communication mode; When the system performs collaborative operations to explore unknown spaces, the drones explore the surrounding unknown areas with the unmanned vehicle as the center, build a lightweight map based on the drone's own second perception module and the second computing power platform and transmit it back to the unmanned vehicle. The unmanned vehicle integrates the lightweight maps received from multiple drones and the refined map built by the unmanned vehicle itself through the first perception module and the first computing power platform.

2. The unknown space collaborative exploration system based on air-ground heterogeneous robots according to claim 1 is characterized in that: The unmanned vehicle and the unmanned aerial vehicle are both provided with wireless communication modules, and the unmanned vehicle and the unmanned aerial vehicle communicate using a semi-centralized self-organizing communication method, specifically: With the unmanned vehicle as the central communication node and the drone as the communication node, the drone and the unmanned vehicle are directly or indirectly connected to form an unmanned vehicle central communication network; When the communication between the drone and the drone center communication network is interrupted, the unmanned vehicle determines whether there is an idle drone at this time; if there is an idle drone, the idle drone is dispatched to navigate to the set location as a communication relay node; If there are no idle drones, the disconnected drone will spontaneously search around for other communication nodes with which it can establish communication. If other communication nodes with which it can establish communication are found within the set time interval, the communication connection will be directly established. If other communication nodes with which it can establish communication are still not found within the set time interval, the disconnected drone will return to the previous position where it could communicate with the drone center communication network. Among them, other communication nodes with which communication can be established are communication relay nodes.

3. The unknown space collaborative exploration system based on air-ground heterogeneous robots according to claim 2 is characterized in that: The communication relay node moves only in a known environment map, and uses a search-based algorithm to calculate the optimal path connecting the two communication nodes before and after; The communication node farthest from the central communication node on the communication link is taken as the terminal communication node. The terminal communication node is responsible for the perception task, and the target position of the relay communication node is as close to the next node as possible along the optimal path; After calculating the connection path between the front and rear nodes, the UAV as a relay communication node starts from its current position and flies along the connection path from the position close to the upper node to the lower node. At the same time, the communication module monitors the strength of the communication signal. When the strength is less than a given threshold, the UAV as a relay communication node lands and performs the communication relay task.

4. The unknown space collaborative exploration system based on air-ground heterogeneous robots according to claim 2 is characterized in that: At most one drone acting as a communication relay node is allowed between the unmanned vehicle and the drone in front for perception.

5. The unknown space collaborative exploration system based on air-ground heterogeneous robots according to claim 1 is characterized in that: The first perception module includes a visual sensor for capturing visual information of the surrounding environment, a lidar for modeling the environment into a dense environmental point cloud map, and an inertial measurement unit for sensing short-term motion data; the second perception module includes a visual sensor for lightweight mapping of the entire environment, and an inertial measurement unit for sensing short-term motion data.

6. The unknown space collaborative exploration system based on air-ground heterogeneous robots according to claim 1 is characterized in that: The unmanned vehicle fuses the lightweight maps transmitted by the multiple unmanned aerial vehicles and the refined maps constructed by the unmanned vehicle itself through the first perception module and the first computing platform. The specific fusion process is as follows: Based on the lightweight map, short-term motion data and key visual frame data sent back by the drone, dense visual low-precision mapping is performed; The spatial point cloud scaling and registration method is used to adjust the pose in the lightweight map sent back by the drone, delete the low-precision map in the overlapping area, align the boundaries of the refined map constructed by the unmanned vehicle, and obtain a double-precision map; As the unmanned vehicle moves towards the lightweight map, it continues to use its own first perception module to continuously build maps, generate new refined maps, and continuously move the boundary between the refined map and the lightweight map backwards.

7. The unknown space collaborative exploration system based on air-ground heterogeneous robots according to claim 6 is characterized in that: A low-frequency update strategy is adopted. Every time the unmanned vehicle travels a certain distance, the point cloud scaling and registration methods are reused to correct the deviations at the boundaries according to the high- and low-precision map boundaries.

8. The unknown space collaborative exploration system based on air-ground heterogeneous robots according to claim 6 is characterized in that: The double-precision map is represented as an octree map for path planning. The undulating slope of the ground is calculated based on the ground data, and the undulating slope is used as the cost item in the path planning of the unmanned vehicle.

9. The unknown space collaborative exploration system based on air-ground heterogeneous robots according to claim 1 is characterized in that: When the collaborative operation explores the unknown space, the UAV explores the unknown area around the unmanned vehicle, specifically including: When multiple unknown boundaries appear in the map, the first computing platform on the unmanned vehicle calculates and outputs the unknown boundary number assigned to the unmanned vehicle for exploration in the next step through a reinforcement learning algorithm based on the fused map of the explored area and the set of current unknown boundaries, and assigns a drone to explore each of the remaining secondary unknown boundaries; When the drone drives away from the unmanned vehicle, the unmanned vehicle stops and waits, and decides the next exploration boundary based on the drone’s exploration results; Among them, the reinforcement learning algorithm uses the path length L and the time T passed by the unmanned vehicle as the reward and punishment items of reinforcement learning, dynamically optimizes the allocation of unknown boundaries, and through the structural distribution characteristics of the learning environment, enables the unmanned vehicle to select the unknown boundary with the highest exploration value score among multiple unknown boundaries for exploration.

10. The unknown space collaborative exploration system based on air-ground heterogeneous robots according to claim 1 is characterized in that: If there is only one unknown boundary on the map, the unmanned vehicle will be used for exploration in single-body exploration mode, and the unknown boundary tracking algorithm will be used to complete the exploration task. Specifically, the geometric center of the unknown boundary is used as the target point for path planning, and a search-based path planning method is used to obtain a walking path to the target boundary. During the continuous operation of the unmanned vehicle, the unknown boundary continues to retreat, and the unmanned vehicle is required to calculate the target position every time it travels a certain distance.

Images