An aerial-ground heterogeneous operation unit real-time map updating and pose correction system

Abstract

The application belongs to the technical field of cooperative navigation and autonomous positioning, and discloses an air-ground heterogeneous operation unit real-time map updating and pose correction system, which comprises a perception and mapping unit carried on an air operation unit and a ground operation unit and used for collecting environment perception data of air and ground perspectives respectively. The air top view and the ground local high-precision dual-perspective environment perception are realized through the perception and mapping unit, the environment change is detected in real time by a map maintenance unit, the map data is updated in an incremental manner, the space-time alignment and fusion of the air-ground dual-perspective map are completed through a cooperative processing unit, the poses of the two are jointly optimized, the latest map and the corrected pose are synchronized to each operation unit in an incremental manner through a distribution and execution unit, thereby replacing the mode that the air and the ground independently map respectively in the traditional scheme, and the off-line fusion after the task is completed, and the problem of cooperative failure caused by the lagging map updating and the accumulated pose drift is solved.

Description

Technical Field

[0001] This invention belongs to the field of cooperative navigation and autonomous positioning technology, specifically a real-time map update and pose correction system for air-ground heterogeneous operation units. Background Technology

[0002] Air-ground heterogeneous operation units refer to collaborative operation systems composed of aerial vehicles (such as drones) and ground mobile robots. The aerial units have the ability to conduct large-scale rapid inspections and overhead observations, while the ground units can perform detailed inspections or operations close to the target. The two share sensing data and positioning information in real time through a communication network, forming a heterogeneous operation formation with complementary functions and different spatial characteristics. This allows them to leverage their respective advantages in the same task and achieve integrated air-ground collaborative perception, decision-making, and execution.

[0003] During collaborative operations between air and ground heterogeneous work units, operators frequently utilize corresponding map building and pose correction systems to construct environmental maps and perform relative pose correction for both air and ground work units. While existing map updating and pose correction systems possess basic SLAM mapping and positioning capabilities, they typically employ a processing mode where air and ground units build maps independently and then merge them offline after the task is completed. This lack of real-time map updating capabilities results in poor adaptability to dynamic environments. Furthermore, these systems often rely on fixed priorities or simple coordinate transformations to handle pose relationships between heterogeneous platforms, making it difficult to address cumulative drift caused by communication delays or occlusion. In transitional indoor / outdoor scenarios and dynamically changing environments, inconsistencies between maps and poses can easily lead to collaborative failures, thus necessitating improvements. Summary of the Invention

[0004] The purpose of this invention is to provide a real-time map update and pose correction system for air-ground heterogeneous operation units to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a real-time map update and pose correction system for air-to-ground heterogeneous operation units, comprising: The perception and mapping unit, mounted on the aerial operation unit and the ground operation unit, is used to collect environmental perception data from aerial and ground perspectives respectively, and to construct their respective local maps. The map maintenance unit, connected to the perception and mapping unit, is used to compare the local map of each working unit at the current moment with the local map at the previous moment, detect map change areas, and generate map incremental data packets. The collaborative processing unit is connected to the perception and mapping unit and the map maintenance unit respectively. It is used to exchange the map incremental data packets and local maps of the aerial operation unit and the ground operation unit through the communication network, perform spatiotemporal alignment and fusion of map information from the aerial perspective and map information from the ground perspective to generate an updated global consistent map, and jointly optimize the pose of the aerial operation unit and the ground operation unit according to the global consistent map to obtain the corrected real-time pose. The distribution and execution unit, connected to the collaborative processing unit, is used to distribute the updated global consistency map and the corrected real-time pose to each work unit.

[0006] Preferably, the perception and mapping unit includes an aerial perception subunit and a ground perception subunit. The aerial perception subunit is mounted on the aerial operation unit and includes at least a lidar, a binocular vision camera, and an inertial measurement unit for collecting point cloud data, image data, and attitude data from an aerial perspective. The ground perception subunit is mounted on the ground operation unit and includes at least a lidar, a depth camera, and an odometer for collecting point cloud data, image data, and displacement data from a ground perspective.

[0007] Preferably, the perception and mapping unit further includes a local map construction subunit, which is used to receive multi-source perception data collected by the air perception subunit and the ground perception subunit respectively, and to perform spatiotemporal synchronization and fusion of lidar point cloud data, visual image data and inertial data through a multi-sensor fusion algorithm to generate air local maps and ground local maps respectively.

[0008] Preferably, the map maintenance unit includes a change detection subunit and an incremental update subunit; the change detection subunit is used to compare the local map of each work unit at the current moment with the local map at the previous moment grid by grid or feature comparison, calculate the map change amount, and mark the changed area and generate an update trigger signal when the change amount exceeds a preset threshold; the incremental update subunit is used to respond to the update trigger signal, extract the location information of the changed area, the map data before the change, the map data after the change, and the change type, and encapsulate them into a map incremental data package.

[0009] Preferably, the collaborative processing unit includes a map fusion subunit, which is used to receive the map incremental data packets and local maps sent by the aerial operation unit and the ground operation unit through the communication network, extract the same-name feature points or common viewing areas in the aerial view map and the ground view map for association matching, calculate the coordinate transformation matrix between the aerial map and the ground map, project the aerial view map and the ground local map onto a unified coordinate system for fusion, and generate a globally consistent map.

[0010] Preferably, the collaborative processing unit further includes a pose correction subunit, which is used to construct an air-ground collaborative pose factor map containing pose nodes of the air operation unit, pose nodes of the ground operation unit, and relative pose constraints between the two, based on the global consistency map, and to jointly estimate the poses of the air operation unit and the ground operation unit through a nonlinear optimization algorithm to eliminate cumulative drift and obtain the corrected real-time pose.

[0011] Preferably, the distribution and execution unit includes a map distribution subunit and a pose distribution subunit; the map distribution subunit is used to distribute the globally consistent map to the aerial operation unit and the ground operation unit, replacing the corresponding changed areas in the local map of each operation unit; the pose distribution subunit is used to distribute the corrected real-time pose to the aerial operation unit and the ground operation unit, so that each operation unit can update its local positioning results and perform subsequent operation tasks based on the updated map and pose.

[0012] Preferably, the system further includes an air-to-ground collaborative communication unit for establishing a real-time communication link between the airborne operation unit and the ground operation unit, and transmitting the map incremental data packets, local maps, and pose data; the air-to-ground collaborative communication unit has a communication quality monitoring function, and when the communication quality is detected to be lower than a preset threshold, it automatically reduces the transmission frequency of the map incremental data packets or compresses the amount of transmitted data to ensure the priority transmission of core pose data.

[0013] Preferably, the system further includes a data storage and playback unit, used to store the aerial local map, ground local map, map incremental data package, globally consistent map and corrected real-time pose in a time series, and to support historical data playback for environmental change trend analysis and system operation status review after the operation is completed.

[0014] The beneficial effects of this invention are as follows: Compared with existing technologies, this invention achieves high-precision dual-view environmental perception from both aerial and ground perspectives through a perception and mapping unit. The map maintenance unit detects environmental changes in real time and updates map data incrementally. The collaborative processing unit completes the spatiotemporal alignment and fusion of the air-ground dual-view maps and jointly optimizes the poses of both sides. Then, the distribution and execution unit synchronizes the latest map and corrected poses to each operation unit incrementally. This replaces the traditional approach of independent mapping in the air and on the ground, followed by offline fusion after the task is completed. It solves the problems of map update lag and collaborative failure caused by accumulated pose drift, and significantly improves the real-time response capability and positioning consistency of heterogeneous air-ground operation units in dynamic environments. Attached Figure Description

[0015] Figure 1 This is a flowchart of the system of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] like Figure 1 As shown, this embodiment of the invention provides a real-time map update and pose correction system for air-to-ground heterogeneous operation units, including: The perception and mapping unit, mounted on the aerial operation unit and the ground operation unit, is used to collect environmental perception data from aerial and ground perspectives respectively, and to construct their respective local maps. The map maintenance unit, connected to the perception and mapping unit, is used to compare the local map of each operation unit at the current moment with the local map at the previous moment, detect map change areas, and generate map incremental data packets. The collaborative processing unit is connected to the perception and mapping unit and the map maintenance unit respectively. It is used to exchange map incremental data packets and local maps between the aerial operation unit and the ground operation unit through the communication network, perform spatiotemporal alignment and fusion of map information from the aerial perspective and map information from the ground perspective, generate an updated global consistent map, and jointly optimize the pose of the aerial operation unit and the ground operation unit based on the global consistent map to obtain the corrected real-time pose. The distribution and execution unit, connected to the collaborative processing unit, is used to distribute the updated global consistency map and the corrected real-time pose to each work unit.

[0018] The perception and mapping unit realizes high-precision dual-view environmental perception from both aerial and local ground perspectives. The map maintenance unit detects environmental changes in real time and updates map data incrementally. The collaborative processing unit completes the spatiotemporal alignment and fusion of the aerial and ground dual-view maps, and jointly optimizes the poses of both sides. Then, the distribution and execution unit synchronizes the latest map and corrected poses to each operation unit, thereby replacing the traditional solution of independent mapping and offline fusion afterward, solving the problems of map update lag and cumulative pose drift.

[0019] The perception and mapping unit includes an aerial perception subunit and a ground perception subunit. The aerial perception subunit is mounted on the aerial operation unit and includes at least a lidar, a binocular vision camera, and an inertial measurement unit, used to collect point cloud data, image data, and attitude data from an aerial perspective. The ground perception subunit is mounted on the ground operation unit and includes at least a lidar, a depth camera, and an odometry, used to collect point cloud data, image data, and displacement data from a ground perspective.

[0020] By configuring sensor combinations adapted to the motion characteristics and operating height of the air and ground operation units respectively, the air unit focuses on wide-area top-down perception, while the ground unit focuses on close-range high-precision perception, thus achieving complementarity and synergy of air and ground dual-view perception capabilities.

[0021] The perception and mapping unit also includes a local map construction subunit, which receives multi-source perception data collected by the aerial perception subunit and the ground perception subunit respectively. Through a multi-sensor fusion algorithm, it performs spatiotemporal synchronization and fusion of lidar point cloud data, visual image data and inertial data to generate aerial local maps and ground local maps respectively.

[0022] The local map construction subunit overcomes the perception degradation problem of a single sensor in indoor-outdoor transition scenes and weak texture environments by multi-sensor fusion, providing a reliable data foundation for subsequent map updates and pose correction.

[0023] The map maintenance unit includes a change detection subunit and an incremental update subunit. The change detection subunit is used to compare the local map of each work unit at the current moment with the local map at the previous moment, and calculate the amount of map change. When the amount of change exceeds a preset threshold, the changed area is marked and an update trigger signal is generated. The incremental update subunit is used to respond to the update trigger signal, extract the location information of the changed area, the map data before the change, the map data after the change, and the change type, and encapsulate them into a map incremental data package.

[0024] The change detection subunit enables real-time perception of dynamic environmental changes, while the incremental update subunit transmits only the changed areas rather than the complete map, significantly reducing communication bandwidth usage and improving map update efficiency.

[0025] The collaborative processing unit includes a map fusion subunit, which is used to receive incremental map data packets and local maps sent by the aerial operation unit and the ground operation unit through the communication network, extract the same feature points or common viewing areas in the aerial view map and the ground view map for association matching, calculate the coordinate transformation matrix between the aerial map and the ground map, project the aerial top view map and the ground local map onto a unified coordinate system for fusion, and generate a globally consistent map.

[0026] The map fusion sub-unit addresses the shortcomings of single-view maps in terms of occluded areas and detail resolution by complementing the global aerial view and the local high-precision ground view, thus generating a fused map that has both global coverage and local accuracy.

[0027] The collaborative processing unit also includes a pose correction subunit, which is used to construct an air-ground collaborative pose factor map based on a global consistency map, which includes the pose nodes of the air operation unit, the pose nodes of the ground operation unit, and the relative pose constraints between them. The pose of the air operation unit and the ground operation unit is jointly estimated through a nonlinear optimization algorithm to eliminate cumulative drift and obtain the corrected real-time pose.

[0028] The pose correction subunit utilizes the relative observation constraints between the airborne and ground-based operation units to perform collaborative pose optimization, ensuring that the poses of both units remain consistent within the same reference frame, thus avoiding the accumulation of relative pose deviations caused by their independent positioning.

[0029] The distribution and execution unit includes a map distribution subunit and a pose distribution subunit. The map distribution subunit is used to distribute the globally consistent map to the aerial operation unit and the ground operation unit, replacing the corresponding changed areas in the local map of each operation unit. The pose distribution subunit is used to distribute the corrected real-time pose to the aerial operation unit and the ground operation unit, so that each operation unit can update its local positioning results and perform subsequent operation tasks based on the updated map and pose.

[0030] Map distribution and pose distribution update local data through incremental replacement rather than full coverage, reducing communication transmission volume and ensuring that each work unit performs tasks under the same spatiotemporal reference, avoiding decision conflicts caused by inconsistencies between map and pose.

[0031] The system also includes an air-to-ground collaborative communication unit, which is used to establish a real-time communication link between the air operation unit and the ground operation unit to transmit map incremental data packets, local maps and pose data. The air-to-ground collaborative communication unit has a communication quality monitoring function. When the communication quality is detected to be lower than a preset threshold, it automatically reduces the transmission frequency of map incremental data packets or compresses the amount of data transmitted to ensure the priority transmission of core pose data.

[0032] The air-ground collaborative communication unit uses an adaptive communication strategy to cope with signal attenuation and interference in complex indoor and outdoor environments, ensuring the continuous availability of the system's core functions under communication-limited conditions.

[0033] The system also includes a data storage and playback unit, which stores aerial local maps, ground local maps, map incremental data packets, globally consistent maps, and corrected real-time poses in time series, and supports historical data playback for environmental change trend analysis and system operation status review after the operation is completed.

[0034] The data storage and playback unit provides data support for system performance evaluation, algorithm optimization, and anomaly event tracing by fully recording the evolution of the map and pose.

[0035] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0036] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A real-time map update and pose correction system for an air-to-ground heterogeneous operation unit, characterized in that, include: The perception and mapping unit, mounted on the aerial operation unit and the ground operation unit, is used to collect environmental perception data from aerial and ground perspectives respectively, and to construct their respective local maps. The map maintenance unit, connected to the perception and mapping unit, is used to compare the local map of each working unit at the current moment with the local map at the previous moment, detect map change areas, and generate map incremental data packets. The collaborative processing unit is connected to the perception and mapping unit and the map maintenance unit respectively. It is used to exchange the map incremental data packets and local maps of the aerial operation unit and the ground operation unit through the communication network, perform spatiotemporal alignment and fusion of map information from the aerial perspective and map information from the ground perspective to generate an updated global consistent map, and jointly optimize the pose of the aerial operation unit and the ground operation unit according to the global consistent map to obtain the corrected real-time pose. The distribution and execution unit, connected to the collaborative processing unit, is used to distribute the updated global consistency map and the corrected real-time pose to each work unit.

2. The real-time map update and pose correction system for an air-to-ground heterogeneous operation unit according to claim 1, characterized in that: The perception and mapping unit includes an aerial perception subunit and a ground perception subunit. The aerial perception subunit is mounted on the aerial operation unit and includes at least a lidar, a binocular vision camera, and an inertial measurement unit, used to collect point cloud data, image data, and attitude data from an aerial perspective. The ground perception subunit is mounted on the ground operation unit and includes at least a lidar, a depth camera, and an odometer, used to collect point cloud data, image data, and displacement data from a ground perspective.

3. The real-time map update and pose correction system for a heterogeneous air-to-ground operation unit according to claim 1, characterized in that: The perception and mapping unit also includes a local map construction subunit, which is used to receive multi-source perception data collected by the air perception subunit and the ground perception subunit respectively, and to perform spatiotemporal synchronization and fusion of lidar point cloud data, visual image data and inertial data through a multi-sensor fusion algorithm to generate air local maps and ground local maps respectively.

4. The real-time map update and pose correction system for an air-to-ground heterogeneous operation unit according to claim 1, characterized in that: The map maintenance unit includes a change detection subunit and an incremental update subunit. The change detection subunit is used to compare the local map of each work unit at the current moment with the local map at the previous moment, and calculate the map change. When the change exceeds a preset threshold, the changed area is marked and an update trigger signal is generated. The incremental update subunit is used to respond to the update trigger signal, extract the location information of the changed area, the map data before the change, the map data after the change, and the change type, and encapsulate them into a map incremental data package.

5. The real-time map update and pose correction system for an air-to-ground heterogeneous operation unit according to claim 1, characterized in that: The collaborative processing unit includes a map fusion subunit, which is used to receive the map incremental data packets and local maps sent by the aerial operation unit and the ground operation unit through the communication network, extract the same feature points or common viewing areas in the aerial view map and the ground view map for association matching, calculate the coordinate transformation matrix between the aerial map and the ground map, project the aerial view map and the ground local map onto a unified coordinate system for fusion, and generate a globally consistent map.

6. The real-time map update and pose correction system for an air-to-ground heterogeneous operation unit according to claim 1, characterized in that: The collaborative processing unit also includes a pose correction subunit, which is used to construct an air-ground collaborative pose factor map containing pose nodes of the air operation unit, pose nodes of the ground operation unit, and relative pose constraints between the two, based on the global consistency map. The pose of the air operation unit and the ground operation unit is jointly estimated by a nonlinear optimization algorithm to eliminate cumulative drift and obtain the corrected real-time pose.

7. The real-time map update and pose correction system for an air-to-ground heterogeneous operation unit according to claim 1, characterized in that: The distribution and execution unit includes a map distribution subunit and a pose distribution subunit. The map distribution subunit is used to distribute the globally consistent map to the aerial operation unit and the ground operation unit, replacing the corresponding changed areas in the local map of each operation unit. The pose distribution subunit is used to distribute the corrected real-time pose to the aerial operation unit and the ground operation unit, so that each operation unit can update its local positioning results and perform subsequent operation tasks based on the updated map and pose.

8. The real-time map update and pose correction system for an air-to-ground heterogeneous operation unit according to claim 1, characterized in that: The system also includes an air-to-ground collaborative communication unit, used to establish a real-time communication link between the airborne operation unit and the ground operation unit, and to transmit the map incremental data packets, local maps and pose data; the air-to-ground collaborative communication unit has a communication quality monitoring function, and when the communication quality is detected to be lower than a preset threshold, it automatically reduces the transmission frequency of the map incremental data packets or compresses the amount of data transmitted to ensure the priority transmission of core pose data.

9. The real-time map update and pose correction system for an air-to-ground heterogeneous operation unit according to claim 1, characterized in that: The system also includes a data storage and playback unit, which stores the aerial local map, ground local map, map incremental data package, globally consistent map and corrected real-time pose in time series, and supports historical data playback for environmental change trend analysis and system operation status review after the operation is completed.

Images