An unknown space cooperative exploration system based on air-ground heterogeneous robots

Abstract

This invention relates to a collaborative exploration system for unknown spaces based on heterogeneous air-ground robots, comprising an unmanned vehicle (UAV) and multiple drones mounted on the UAV. The UAV is equipped with a first perception module and a first computing platform for constructing a detailed map of the environment, while the drones are equipped with a second perception module and a second computing platform for constructing a lightweight map of the environment. Both the UAV and drones are equipped with wireless communication modules, employing a semi-centralized self-organizing communication method. During collaborative exploration of unknown spaces, the drones explore the surrounding unknown areas from the UAV's center, constructing a lightweight map based on their own second perception module and second computing platform and transmitting it back to the UAV. The UAV then merges the lightweight map transmitted by the drones with its own detailed map constructed using the first perception module and first computing platform. Compared with existing technologies, this invention has advantages such as strong environmental adaptability and high reliability.

Description

Technical Field

[0001] This invention relates to the field of unknown space exploration, and in particular to an unknown space collaborative exploration system based on air-ground heterogeneous robots. Background Technology

[0002] Robots can not only improve the efficiency of exploring unknown spaces, but also ensure the safety of personnel in dangerous or inaccessible environments, which is of great significance for scientific research, military reconnaissance, disaster relief and other fields.

[0003] However, due to the complexity and unpredictability of unknown spaces, traditional single-robot systems often suffer from incomplete information acquisition and low task execution efficiency when facing complex and ever-changing unknown environments. Robots such as drones, ground robots, or underwater robots typically have specific functions and operating environments, but when faced with complex unknown spaces, single robots often struggle to complete comprehensive and efficient exploration tasks, often only able to perform tasks in specific environments and unable to cope with diverse exploration needs. For example, while drones can quickly cover large areas, their maneuverability and stability are limited in confined spaces or environments with dense obstacles; ground robots excel in terrain adaptability, but are less effective in aerial or underwater environments.

[0004] Therefore, developing a robotic system that can adapt to complex environments and has efficient collaborative capabilities has become an important direction for current technological development.

[0005] Heterogeneous robotic systems, composed of different types of robots, leverage their respective strengths and complement each other's functions to achieve more flexible and efficient task execution. For example, ground robots excel in terrain adaptability and payload capacity, while aerial robots stand out in terms of field of vision and maneuverability. Collaborative work among heterogeneous robotic systems enables information sharing and optimized task allocation. Heterogeneous robotic collaborative exploration holds immense potential, but achieving this goal requires the support of multiple technologies.

[0006] While existing heterogeneous robotic systems have made some progress in certain aspects, some limitations still exist. For example, the reliability and stability of communication systems need to be improved, the accuracy and efficiency of environmental perception and mapping technologies need to be further enhanced, and task allocation and path planning need to be more intelligent and adaptive. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a collaborative exploration system for unknown spaces based on air-ground heterogeneous robots, which has strong environmental adaptability, accurate and efficient environmental mapping, and high reliability.

[0008] The objective of this invention can be achieved through the following technical solutions:

[0009] An unknown space collaborative exploration system based on air-ground heterogeneous robots includes an unmanned vehicle and multiple drones mounted on the unmanned vehicle. The unmanned vehicle is equipped with a first perception module and a first computing platform for building a detailed map of the environment, and each drone is equipped with a second perception module and a second computing platform for building a lightweight map of the environment.

[0010] When the system conducts collaborative operations to explore unknown spaces, the drone explores the surrounding unknown areas with the unmanned vehicle as the center. It constructs a lightweight map based on the drone's own second perception module and second computing platform and transmits it back to the unmanned vehicle. The unmanned vehicle integrates the lightweight maps transmitted by multiple drones and the refined map constructed by the unmanned vehicle itself through the first perception module and the first computing platform.

[0011] Both the unmanned vehicle and the drone are equipped with wireless communication modules, and they communicate using a semi-centralized self-organizing communication method.

[0012] Preferably, both the unmanned vehicle and the drone are equipped with wireless communication modules, and the unmanned vehicle and the drone communicate using a semi-centralized self-organizing communication method, specifically:

[0013] With unmanned vehicles as the central communication node and drones as communication nodes, drones and unmanned vehicles are directly or indirectly connected to each other to form an unmanned vehicle central communication network.

[0014] When communication between the drone and the drone central communication network is interrupted, the unmanned vehicle determines whether there is an idle drone. If an idle drone exists, it is dispatched to navigate to a designated location to serve as a communication relay node. If no idle drone exists, the disconnected drone automatically searches for other communication nodes in its surroundings. If it finds another communication node within a set time interval, it establishes a communication connection directly. If it still cannot find another communication node within the set time interval, the disconnected drone returns to the location where it could previously communicate with the drone central communication network. The other communication nodes that can establish communication are communication relay nodes.

[0015] Preferably, the communication relay node only moves within a known environment map, and a search-based algorithm is used to calculate the optimal path connecting the two communication nodes.

[0016] The communication node furthest from the central communication node on the communication link is taken as the end communication node. The end communication node is responsible for the sensing task, and the target position of the relay communication node is as close as possible to the next node along the optimal path.

[0017] After calculating the connection path between the preceding and following nodes, the UAV, acting as a relay communication node, starts from its current position and flies down to the next lower-level node along the connection path from a position close to the upper-level node. At the same time, the communication module monitors the strength of the communication signal. When the strength is less than a given threshold, the UAV, acting as a relay communication node, lands and performs the communication relay task.

[0018] Preferably, at most one drone is allowed to act as a communication relay node between the unmanned vehicle and the drone used for perception in front.

[0019] Preferably, the first sensing module includes a visual sensor for capturing visual information of the surrounding environment, a lidar for modeling the environment into a dense environmental point cloud map, and an inertial measurement unit for sensing short-term motion data; the second sensing module includes a visual sensor for lightweight mapping of the overall environment and an inertial measurement unit for sensing short-term motion data.

[0020] Preferably, the unmanned vehicle fuses the lightweight map transmitted by multiple drones with the detailed map constructed by the unmanned vehicle itself through the first perception module and the first computing platform. The specific fusion process is as follows:

[0021] Based on the lightweight map, short-term motion data and key visual frame data transmitted back by the UAV, visually dense low-precision mapping is carried out.

[0022] The pose in the lightweight map transmitted by the UAV is adjusted by scaling and registration of spatial point cloud, low-precision maps in overlapping areas are deleted, and the boundaries of the fine map built by the UAV are aligned to obtain a double-precision map.

[0023] As the autonomous vehicle moves toward the lightweight map, it continuously uses its primary perception module to build a new, more detailed map, constantly shifting the boundary between the detailed map and the lightweight map.

[0024] Preferably, a low-frequency update strategy is adopted. Whenever the autonomous vehicle travels a certain distance, the deviation at the boundary is corrected by reusing the point cloud scaling and registration method based on the high and low precision map boundaries.

[0025] Preferably, the double-precision map is represented as an octree map for path planning. Based on the ground data, the undulation and slope of the ground are calculated, and the undulation and slope are used as a cost item in the autonomous vehicle path planning.

[0026] Preferably, when the collaborative operation explores unknown space, the drone explores the surrounding unknown areas from the unmanned vehicle as the center, specifically including:

[0027] When multiple unknown boundaries appear on the map, the first computing platform on the unmanned vehicle calculates and outputs the unknown boundary number to be allocated to the unmanned vehicle for the next step of exploration based on the fused map of the explored area and the current set of unknown boundaries through a reinforcement learning algorithm, and assigns a drone to explore each of the remaining minor unknown boundaries.

[0028] When the drone leaves the driverless car, the driverless car stops and waits, and the next exploration boundary is determined based on the drone's exploration results;

[0029] The reinforcement learning algorithm uses the path length L and time T taken by the autonomous vehicle as the reward and penalty terms for reinforcement learning, dynamically optimizes the allocation of unknown boundaries, and enables the autonomous vehicle to select the unknown boundary with the highest exploration value score from multiple unknown boundaries by learning the structural distribution characteristics of the environment.

[0030] Preferably, if the map has only one unknown boundary, the exploration is carried out in a single-vehicle exploration mode, using an unknown boundary tracking algorithm to complete the exploration task. Specifically, the geometric center of the unknown boundary is used as the target point for path planning, and a search-based path planning method is used to obtain the walking path to the target boundary. During continuous operation of the autonomous vehicle, the unknown boundary continuously recedes, and the target position is calculated every certain distance the autonomous vehicle travels.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] (1) Enhancing Collaborative Exploration Capabilities: This invention achieves comprehensive exploration of unknown spaces by constructing a collaborative system between ground-based unmanned vehicles (UAVs) and aerial drones (UAVs). The UAV, acting as a mobile base, not only provides a platform for the UAVs to take off and land but also performs detailed environmental modeling using its onboard sensors and computing platform. The UAV, utilizing lightweight sensors and computing platforms, operates for extended periods with low power consumption, enabling lightweight mapping of areas outside the UAV's field of vision. This collaborative operation mode allows the entire system to cover a wider area, provide richer environmental information, and significantly improve the comprehensiveness and efficiency of exploration.

[0033] (2) Stable Communication and Efficient Data Transmission Guarantee: The centralized self-organizing communication system of this invention ensures the stability and efficiency of data transmission between the unmanned vehicle and the drone. The unmanned vehicle acts as the central communication node, and the drone establishes communication with the unmanned vehicle directly or indirectly to achieve real-time transmission of perception data and operating commands. Even if communication between some drones and the unmanned vehicle is blocked, the continuity of data transmission can be maintained by using other drones as communication relays, thereby ensuring the robustness and reliability of the entire system.

[0034] (3) Intelligent task allocation and dynamic path planning: The UAV swarm autonomous exploration system can intelligently allocate tasks and plan paths based on real-time environmental information and UAV status. Through the segmentation of unknown boundaries and dynamic task allocation, UAVs can cooperate efficiently to achieve rapid exploration of unknown areas. At the same time, UAVs complete lightweight mapping in real time during the exploration process and transmit key information back to the unmanned vehicle. The unmanned vehicle then uses this information to perform map fusion and boundary information updates, further optimizing the UAVs' exploration tasks and paths.

[0035] (4) Double-precision map generation and environmental understanding: The double-precision mapping and navigation system for ground-based unmanned vehicles combines high-precision SLAM and visual SLAM technologies to generate a double-precision map containing both high-precision and lightweight information. This map not only provides fine details of the environment but also covers a wider area, providing more comprehensive navigation information for the unmanned vehicle. The wide field of view of the drone helps the unmanned vehicle understand the environmental structure and the traffic conditions ahead, enabling it to make more accurate path planning and navigation decisions.

[0036] (5) High degree of autonomy and intelligent operation: The system of this invention possesses a high degree of autonomy and intelligence, enabling it to make autonomous decisions and execute tasks in unknown environments. The combination of lightweight mapping by UAVs and high-precision mapping by unmanned vehicles allows the system to adapt to changing environmental conditions and achieve more flexible and intelligent exploration strategies. The collaborative operation of unmanned vehicles and UAVs reduces reliance on human intervention, improves the system's autonomous operation capability, and also reduces operational complexity and risk. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the hardware composition of the unknown space collaborative exploration system based on air-ground heterogeneous robots in the embodiment.

[0038] Figure 2 This is a schematic diagram of a semi-centralized communication topology in the embodiment;

[0039] Figure 3 This is a schematic diagram of the communication relay node deployment method in the embodiment;

[0040] Figure 4 This is a schematic diagram of the unknown boundary allocation in the embodiment. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0042] Example

[0043] This embodiment provides a collaborative exploration system for unknown spaces based on heterogeneous air-ground robots, including an unmanned vehicle and multiple drones mounted on the unmanned vehicle. The unmanned vehicle is equipped with a first perception module and a first computing platform for building a detailed map of the environment, and the drones are equipped with a second perception module and a second computing platform for building a lightweight map of the environment. Both the unmanned vehicle and the drones are equipped with wireless communication modules, and the unmanned vehicle and the drones communicate using a semi-centralized self-organizing communication method.

[0044] When the system conducts collaborative operations to explore unknown spaces, the drone explores the surrounding unknown areas with the unmanned vehicle as the center. It constructs a lightweight map based on its own second perception module and second computing platform and sends it back to the unmanned vehicle. The unmanned vehicle integrates the lightweight maps transmitted by multiple drones and the detailed map constructed by the unmanned vehicle itself through the first perception module and the first computing platform.

[0045] Next, the system of this embodiment will be described in detail.

[0046] (1) Hardware architecture of the collaborative exploration system for unknown space

[0047] like Figure 1 As shown, the hardware components of the collaborative exploration system for unknown space include an unmanned vehicle S01 and multiple drones S02.

[0048] The unmanned vehicle S01 is equipped with a perception module. In this embodiment, the perception module employs multimodal perception devices, including: a horizontally rotating 3D LiDAR S04 for modeling the environment into a dense environmental point cloud map, a surround-view camera S05 for capturing visual information of the surrounding environment, and an inertial measurement unit (IMU) for sensing short-term motion data. The surround-view camera S05 is mounted below the drone and can rotate its observation angle, allowing the drone to adjust its viewing angle during both perception and takeoff / landing operations. These three sensors complement each other, fusing perception to avoid the degradation of a single sensor.

[0049] The unmanned vehicle S01 has a planar platform S03 on its upper part for parking drones. The surface of the planar platform S03 is affixed with a QR code S06 for calculating relative pose. When the drone S01 takes off and lands, it captures the QR code through the surround-view camera S05, thereby calculating its own attitude relative to the unmanned vehicle and planning a safe and fast movement path.

[0050] Both the unmanned vehicle S01 and the drone S02 are equipped with powerful computing platforms, which are used to build detailed maps and lightweight maps respectively.

[0051] Unmanned vehicles need to carry drones for long-distance operations, and at the same time serve as communication centers and perception data processing centers, resulting in high energy consumption and requiring the installation of large energy storage batteries.

[0052] Both driverless cars and drones are equipped with wireless communication modules.

[0053] (2) Ground-to-air semi-centralized self-organizing communication and communication node scheduling

[0054] The system operates with the ground-based unmanned vehicle (UAV) as its central hub. Both the UAV and drones are equipped with wireless communication modules, forming communication network nodes suitable for a centralized network structure. With the UAV as the central node, data from each drone is directly transmitted to it, creating a centralized communication network. However, in complex environments, flying drones may lose communication with the central UAV, resulting in communication failure. To address this, drones, when unable to communicate with the central UAV, need to spontaneously search for communicable targets in their surroundings and establish connections with another communication node, forming a localized distributed network structure. This creates a semi-centralized communication network where drones explore outwards from the UAV, allowing for temporary communication interruptions between edge nodes and the central node, as well as localized communication between edge nodes within a small area, improving the system's flexibility and stability. Drones or drone groups that lose connection to the central node can plan their return time according to their operational needs, thereby restoring their connection to the central node.

[0055] Specifically, the unmanned vehicle (UAV) serves as the central communication node, and the drone serves as the communication node. The drone and the UAV are directly or indirectly connected to form a central communication network for the UAV. When communication between the drone and the drone central communication network is interrupted, the UAV determines whether there is an idle drone. If there is an idle drone, it is dispatched to navigate to a designated location to serve as a communication relay node. If there is no idle drone, the disconnected drone spontaneously searches for other communication nodes in its surroundings. If it finds another communication node within a set time interval, it establishes a communication connection directly. If it still cannot find another communication node within the set time interval, the disconnected drone returns to the location where it could previously communicate with the drone central communication network. The other communication nodes that can establish communication are communication relay nodes.

[0056] In this embodiment, the unmanned system (drone or unmanned vehicle) is set as the communication node, and a self-organizing communication network is built using UANET. UANET has the advantages of self-organization, high flexibility and fast networking speed, which can meet the communication requirements of multiple unmanned systems.

[0057] like Figure 2 As shown, the unmanned vehicle U0 is the central node of the communication network, and the drone U... i (i = 1, ..., n) are directly connected to the autonomous vehicle via wireless communication, forming a star topology network structure, which is represented as follows:

[0058] G =<V,E>

[0059] V={U i |i=0,1,…,n}

[0060] E = {(U0, U...} i |i=1,…,n}

[0061] When an independent drone group appears, each drone in the group communicates independently with each other, forming a communication network G independent of the central node. ′ :

[0062] G ′ = <V ′ E ′ >

[0063] V ′ ={U p U q}

[0064] E ′ ={(U p U q )}

[0065] When an independent drone group moves away from or nears the central network, the connection between two close drones is disconnected or reconnected. When the connection exists, a multi-hop chain topology is formed between the terminal drones and the central drones, with the drones in the chain acting as communication relays between the center and the terminal. As drones fly in the environment, the communication signal strength is greatly affected by the environmental structure. In complex environments, the signal is weak when drones are blocked by obstacles during exploration, while data transmission requires good communication conditions. This type of communication network structure is often needed in such cases.

[0066] When a drone acts solely as a communication relay in a communication network, its relative position to both endpoints significantly impacts communication effectiveness. It needs to dynamically adjust its position based on changes in the endpoints' positions—this is known as autonomous scheduling of communication nodes. Autonomous vehicles (RVs) navigate their environment, using their sensors to create maps of the surroundings. Drones then navigate based on these maps, exploring unknown boundaries to acquire more information. When a local map is known, the drone can achieve localization and navigation using sensors and the map. Furthermore, it can autonomously move to a designated location based on instructions; this forms the basis for autonomous scheduling of communication nodes. Unmanned systems monitor the strength of communication signals through their communication modules, evaluating the connection status between nodes and providing decision-making information for autonomous scheduling.

[0067] like Figure 3 As shown, relay node U i The system moves only within a known environment map M, and uses a search-based algorithm to calculate the connection between two consecutive nodes U in a 2D rasterized map. i-1 and U i+1 The optimal path is determined by the terminal node. Since the terminal node is responsible for the perception task and has a large range of movement, it is necessary to ensure good communication conditions during large-scale movement. Therefore, the target position of the relay node needs to be as close as possible to the next node along the optimal path. After calculating the connection path between the preceding and following nodes, the relay node UAV starts from its current position and flies to the next node along the connection path from the position close to the superior node. At the same time, it monitors the communication signal strength. When the strength is less than a given threshold, the relay node lands and performs the communication relay task.

[0068] (3) Autonomous exploration of unmanned aerial vehicles centered on ground unmanned vehicles

[0069] When a single autonomous vehicle (RV) is performing an exploration mission, if a large blind spot appears in the environment, the RV needs to change its original direction of movement to move to a location that can cover the blind spot. Alternatively, when there are branching structures in the environment, the RV needs to go to different branches in sequence to expand its field of vision and assess the road conditions ahead. In both cases, the RV needs to make multiple round trips. However, for a multi-unmanned system that coordinates ground and air operations, the RV carries a drone to explore the unknown environment together. When the environment is open and there are no branching paths, the sensors carried by the RV itself are sufficient to create a good map of the environment. When facing blind spots, the RV can maintain its original trajectory while the drone moves to cover them. When facing branching structures, the drone can move to explore roughly and help the RV choose a better path forward.

[0070] In a simple environment, there is only one main unknown boundary F in map M. The system explores in a single unmanned vehicle mode and can complete the exploration task using only a simple algorithm to track the unknown boundary.

[0071] For an unknown boundary F, find its geometric center and use it as the target point P for path planning. target ;

[0072] P target =Center(F)

[0073] A search-based path planning method is used to obtain the walking path to the target boundary. During the continuous operation of the autonomous vehicle, the position of the location boundary F will also continue to move backward. Repeatedly calculating the target position will increase the power consumption of the computer. At the same time, there will be large or small differences in the target position each time, which may cause the autonomous vehicle to sway in the forward direction, resulting in energy loss in motion. Therefore, it is stipulated that the autonomous vehicle calculates the target position once after traveling a certain distance, thereby reducing the update frequency of the target position.

[0074] like Figure 4 As shown, in complex environments, blind spots or forks in the path may appear in the exploration area, resulting in more than one unknown boundary F appearing on map M. i (i = 1, 2, ...), different unknown boundaries have different exploration costs and exploration values ​​(e.g., the path length to the unknown boundary, the possibility that the unknown boundary is a closed area and therefore a return is necessary; the possibility of the size, openness and shape of the exploration space behind the unknown boundary). They need to be evaluated and ranked according to the state of the unknown boundaries. The unmanned vehicle is the center of the exploration team, and the boundaries with the most exploration value should be assigned to the unmanned vehicle, while the secondary boundaries should be assigned to the drones for exploration.

[0075] The cost and value of exploring unknown boundaries are difficult to quantify explicitly. Based on the actual application scenario, a simulation scenario is designed, and reinforcement learning is used to learn the environmental structure distribution characteristics in the scenario, so as to select a suitable boundary for autonomous vehicles to explore from multiple unknown boundaries.

[0076] In the virtual scenario, the communication and perception processes are simplified. It is assumed that communication between unmanned systems is unimpeded, and the perception of the unmanned system is equivalent to field-of-view coverage, thus accelerating operation. A virtual ground-air cooperative system is used for exploration. The reinforcement learning model f(·) receives the map M of the explored area and the current set of unknown boundaries {F}. i |i=1,2,…}, output the ID of the unknown boundary to be assigned to the autonomous vehicle for exploration in the next step, while the remaining unknown boundaries are explored by the drone; after exploring the entire scene, use the path length L and the time T taken by the autonomous vehicle as the reward and penalty terms for reinforcement learning, so that the system learns the selection strategy for unknown boundaries:

[0077] ID = f(M, {F i})

[0078] R = -α(L + T)

[0079] Secondary boundaries are explored by drones. Constrained by communication and mobility, the drones move within a range centered on the autonomous vehicle. One drone is assigned to explore each secondary boundary. When a drone leaves the autonomous vehicle, the vehicle stops and waits, maintaining a stable communication environment and allowing the system to determine the next boundary to explore based on the drone's exploration results. The unknown boundary tracking method is also used: first, the centroid of the unknown boundary is determined, and then a search-based path planning algorithm is used to generate a path to guide the drone to explore. The drone uses a camera for environmental perception, but its field of view is narrow, requiring it to move back and forth along the unknown boundary to cover the area, gradually shifting the unknown boundary backward while creating a map and recording key visual information, which is then transmitted to the autonomous vehicle. The drone uses existing visual-inertial sparse mapping methods, primarily for self-localization and to perceive the general structure of the environment. When the drone is far from the autonomous vehicle and the communication rate drops below the standard, the system will dispatch another drone to act as a communication relay, as described in Example 1, thereby increasing the exploration range of the drone in front.

[0080] The number of drones is limited, and the maximum number of drones that can be explored in secondary unknown boundaries does not exceed the number of drones. When there are idle drones, at most one drone is allowed to act as a communication relay between the unmanned vehicle and the forward sensing drone. When there are no idle drones, the sensing drones do not have the support of relay drones and can only perform exploration tasks within the signal range of the unmanned vehicle.

[0081] The maximum range that a forward-sensing drone can explore depends on the communication capabilities of the unmanned vehicle and the communication relay drone. The unknown area beyond the unknown boundary is either a small, enclosed space that can be completely covered by the exploration range and no new unknown boundaries will be generated, or a large enclosed or open space that exceeds the exploration range and still has new unknown boundaries.

[0082] After the drone completes its exploration, the unmanned vehicle makes a decision based on the new unknown boundaries and moves to the next unknown boundary with the greatest exploration value. The drone returns and lands on the unmanned vehicle, and they move together to wait for the next multi-unknown boundary scenario to be explored.

[0083] (4) Dual-precision mapping and navigation for ground unmanned vehicles

[0084] Ground-based unmanned vehicles are equipped with lidar, visual sensors, and IMUs, enabling them to autonomously create high-precision dense maps using existing multi-sensor fusion methods. However, such high-precision maps can only be generated along the unmanned vehicle's working path. Areas explored by drones alone only have sparse maps sent by the drones, which cannot support unmanned vehicle navigation. It is necessary to reconstruct the environment visually from the key visual frames and poses saved by the drones to form a dense map. The original information of this dense map comes from the monocular camera equipped on the drone, which has lower accuracy compared to lidar sensors and suffers from the scale drift that is unavoidable in monocular systems, resulting in a low-precision map.

[0085] The two types of maps, each with different levels of precision, are generated from sensor data from autonomous vehicles and drones, respectively. They cover different areas in space and partially overlap. Due to mapping errors, the two maps may deviate in the position of obstacle point clouds, often failing to align precisely. To improve the overall representational performance of the dual-precision maps, it is necessary to retain the high-precision map in the overlapping area as much as possible and delete the low-precision map. Therefore, spatial point cloud scaling and registration methods are required to adjust the pose of the low-precision map so that the two maps can be precisely aligned at the boundary of the high-precision map.

[0086] Both types of maps with the above precision are dense maps. They can be converted into octree spatial occupancy maps using the same method to form a unified representation. The same algorithm can be used to complete the path planning. In the unified octree map representation, the undulation changes of the ground are recorded. The undulation slope G of the ground is calculated by using the local gradient changes of the ground part. The road slope is used as a cost item for path planning, reducing the up and down undulations of the planned path, providing a smooth passage path for autonomous vehicles, and reducing energy consumption.

[0087] The initial slope at coordinates (x, y) is defined as g(x, y), where s is the slope calculation step size:

[0088]

[0089] The initial slope is greatly affected by high-frequency terrain information and is not easy to represent low-frequency terrain undulation information. Two-dimensional Gaussian convolution K and Gaussian filtering are applied to the initial slope g(·) to eliminate the influence of high-frequency information and obtain the undulation slope G.

[0090] G(x,y)=conv(g(·) x-ns:x+ns,y-ns:y+ns ·K (2n+1)×(2n+1) )

[0091] Due to the difference in accuracy, the autonomous vehicle can achieve accurate self-positioning in the map it creates, but it will have a large error in the low-precision map. As the autonomous vehicle moves towards the low-precision map, it still uses its own sensors to continuously build maps and generate new high-precision maps, thereby further pushing the boundary between the high-precision and low-precision maps backward. Using a low-frequency update strategy similar to that in Example 3, every time the autonomous vehicle travels a certain distance, it re-utilizes the point cloud scale scaling and registration to correct the deviation at the boundary according to the boundary between the high-precision and low-precision maps, ensuring that the autonomous vehicle can continuously use the low-precision map for path planning.

[0092] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A collaborative exploration system for unknown spaces based on air-ground heterogeneous robots, characterized in that, The system includes an unmanned vehicle and multiple drones mounted on the unmanned vehicle. The unmanned vehicle is equipped with a first perception module and a first computing platform for building a detailed map of the environment, and the drones are equipped with a second perception module and a second computing platform for building a lightweight map of the environment. Both the unmanned vehicle and the drones are equipped with wireless communication modules, and the unmanned vehicle and the drones communicate using a semi-centralized self-organizing communication method. When the system conducts collaborative operations to explore unknown spaces, the drone explores the surrounding unknown areas with the unmanned vehicle as the center. It constructs a lightweight map based on the drone's own second perception module and second computing platform and transmits it back to the unmanned vehicle. The unmanned vehicle integrates the lightweight maps transmitted by multiple drones and the refined map constructed by the unmanned vehicle itself through the first perception module and the first computing platform. Both the unmanned vehicle and the drone are equipped with wireless communication modules. The unmanned vehicle and the drone communicate using a semi-centralized self-organizing communication method, specifically: With unmanned vehicles as the central communication node and drones as communication nodes, drones and unmanned vehicles are directly or indirectly connected to each other to form an unmanned vehicle central communication network. When communication between the drone and the drone center communication network is interrupted, the unmanned vehicle determines whether there is an idle drone at this time; if there is an idle drone, it dispatches the idle drone to navigate to the set location as a communication relay node. If no idle drone is found, the disconnected drone will automatically search for other communication nodes in its surroundings to establish communication. If other communication nodes are found within a set time interval, a communication connection will be established directly. If no other communication nodes are found within the set time interval, the disconnected drone will return to the position where it could previously communicate with the drone center communication network. The other communication nodes that can establish communication are communication relay nodes.

2. The unknown space collaborative exploration system based on air-ground heterogeneous robots according to claim 1, characterized in that, The communication relay node moves only within a known environment map, and a search-based algorithm is used to calculate the optimal path connecting the two communication nodes. The communication node furthest from the central communication node on the communication link is taken as the end communication node. The end communication node is responsible for the sensing task, and the target position of the relay communication node is as close as possible to the next node along the optimal path. After calculating the connection path between the preceding and following nodes, the UAV, acting as a relay communication node, starts from its current position and flies down to the next lower-level node along the connection path from a position close to the upper-level node. At the same time, the communication module monitors the strength of the communication signal. When the strength is less than a given threshold, the UAV, acting as a relay communication node, lands and performs the communication relay task.

3. The unknown space collaborative exploration system based on air-ground heterogeneous robots according to claim 1, characterized in that, The unmanned vehicle and the drone used for perception in front are allowed to have at most one drone acting as a communication relay node.

4. The unknown space collaborative exploration system based on air-ground heterogeneous robots according to claim 1, characterized in that, The first perception module includes a visual sensor for capturing visual information of the surrounding environment, a lidar for modeling the environment into a dense environmental point cloud map, and an inertial measurement unit for sensing short-term motion data; the second perception module includes a visual sensor for lightweight mapping of the overall environment and an inertial measurement unit for sensing short-term motion data.

5. A collaborative exploration system for unknown spaces based on heterogeneous air-ground robots according to claim 1, characterized in that, The unmanned vehicle integrates the lightweight maps transmitted from multiple drones with the detailed maps constructed by the unmanned vehicle itself through the first perception module and the first computing platform. The specific integration process is as follows: Based on the lightweight map, short-term motion data and key visual frame data transmitted back by the UAV, visually dense low-precision mapping is carried out. The pose in the lightweight map transmitted by the UAV is adjusted by scaling and registration of spatial point cloud, low-precision maps in overlapping areas are deleted, and the boundaries of the fine map built by the UAV are aligned to obtain a double-precision map. As the autonomous vehicle moves toward the lightweight map, it continuously uses its primary perception module to build a new, more detailed map, constantly shifting the boundary between the detailed map and the lightweight map.

6. A collaborative exploration system for unknown spaces based on heterogeneous air-ground robots according to claim 5, characterized in that, A low-frequency update strategy is adopted. Whenever the autonomous vehicle travels a certain distance, the deviation at the boundary is corrected by reusing point cloud scaling and registration methods based on the high and low precision map boundaries.

7. A collaborative exploration system for unknown spaces based on heterogeneous air-ground robots according to claim 5, characterized in that, The double-precision map is represented as an octree map for path planning. Based on the ground data, the undulation and slope of the ground are calculated and used as a cost item in the autonomous vehicle path planning.

8. A collaborative exploration system for unknown spaces based on heterogeneous air-ground robots according to claim 1, characterized in that, When exploring unknown spaces in the collaborative operation, the drone explores the surrounding unknown areas from the unmanned vehicle as the center, specifically including: When multiple unknown boundaries appear on the map, the first computing platform on the unmanned vehicle calculates and outputs the unknown boundary number to be allocated to the unmanned vehicle for the next step of exploration based on the fused map of the explored area and the current set of unknown boundaries through a reinforcement learning algorithm, and assigns a drone to explore each of the remaining minor unknown boundaries. When the drone leaves the driverless car, the driverless car stops and waits, and the next exploration boundary is determined based on the drone's exploration results; The reinforcement learning algorithm uses the path length L and time T taken by the autonomous vehicle as the reward and penalty terms for reinforcement learning, dynamically optimizes the allocation of unknown boundaries, and enables the autonomous vehicle to select the unknown boundary with the highest exploration value score from multiple unknown boundaries by learning the structural distribution characteristics of the environment.

9. A collaborative exploration system for unknown spaces based on heterogeneous air-ground robots according to claim 1, characterized in that, If the map has only one unknown boundary, the exploration is carried out in the single-vehicle exploration mode. The unknown boundary tracking algorithm is used to complete the exploration task. Specifically, the geometric center of the unknown boundary is used as the target point for path planning. A search-based path planning method is used to obtain the walking path to the target boundary. During the continuous operation of the autonomous vehicle, the unknown boundary continues to retreat. The target position is calculated once every certain distance the autonomous vehicle travels.

Images