A method and system for collaborative positioning and environment mapping of air-to-ground vehicles

Abstract

The present invention discloses a method and system for collaborative positioning and environmental mapping of air-ground vehicle networking, which relates to the field of future three-dimensional transportation technology. The method includes: determining the communication frequency by vehicle parameters; determining the optimal communication path according to the total score of communication quality on all possible paths; determining a number of key frames according to the visual data stream of the central vehicle, and generating a local map according to the key frames; sending the key frame information to the ground vehicle through the near-ground and air vehicle; extracting and parsing the key frame information; updating the local map according to the parsing information to obtain an updated local map of the ground vehicle; fusing the updated local map of the ground vehicle and the local map of the near-ground and air vehicle to obtain a global map; optimizing and adjusting the global map; and updating the location recognition database. The present invention ensures efficient collaboration between the two parties in data processing and map construction, and effectively improves the navigation and positioning accuracy of the three-dimensional transportation system.

Description

Technical Field

[0001] The present invention relates to the field of future three-dimensional transportation technology, and in particular to a method and system for collaborative positioning and environmental mapping of air-to-ground vehicles through networking. Background Art

[0002] With the rapid development of electric Vertical Take-off and Landing (eVTOL) technology and the demand for the development of the near-ground and air economy, ground vehicles and near-ground and air vehicles will become important components of the integrated three-dimensional transportation system. However, since the traditional ground vehicle positioning and navigation systems are usually of a single type and rely on the global positioning system (GPS) and other ground sensors, near-ground and air vehicles require high-precision air navigation technology. This has led to limited cooperation between the two and the problem of positioning and navigation in complex three-dimensional transportation environments.

[0003] Simultaneous Localization And Mapping (SLAM) is widely used in high-precision dynamic positioning and navigation scenarios of robots (near-ground and airborne vehicles, unmanned vehicles, etc.) in unknown environments. This technology gives sensor-equipped carriers the dual capabilities of mobile perception and map construction in environments without high-precision maps, while also calculating the carrier's accurate position on the map. At present, collaborative SLAM systems generally adopt a centralized architecture, including a front-end agent responsible for real-time processing of visual information and a back-end server responsible for integrating and optimizing map data. However, the centralized architecture has certain challenges in meeting the real-time requirements of positioning and mapping, and is highly dependent on road side units (RSUs) with powerful computing capabilities. In order to overcome these challenges, a distributed SLAM architecture has been proposed to make full use of the computing resources of various carriers, ensure real-time performance, improve safety, and avoid unnecessary duplication of environmental maps. The core idea of ​​distributed SLAM is to distribute the calculation and decision-making of the SLAM system among multiple carriers to achieve more efficient collaborative positioning and mapping. Whether it is a centralized or distributed architecture, the security of the air-to-ground data link must be guaranteed first. The current challenges are mainly reflected in the lack of dynamic requirements and adaptive strategies for data link security, which may make it difficult to meet the high-bandwidth data security transmission requirements required for collaborative SLAM.

[0004] In order to ensure coordinated, safe and accurate positioning and navigation in an integrated three-dimensional transportation network, a collaborative positioning and environmental mapping method for networking of ground vehicles and near-ground airborne vehicles is urgently needed. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for collaborative positioning and environmental mapping of air-to-ground vehicles, which can ensure collaborative, safe and accurate positioning and navigation in a comprehensive three-dimensional transportation network.

[0006] To achieve the above object, the present invention provides the following solution.

[0007] In a first aspect, the present invention provides a method for collaborative positioning and environment mapping of networked air-to-ground vehicles, comprising the following steps.

[0008] Determine the communication frequency by vehicle parameters; determine the optimal communication path based on the total communication quality score on all possible paths; vehicle parameters include the total capacity of the channel, the total number of near-ground and ground vehicles in the network, and the average distance between vehicles.

[0009] For each central vehicle, a number of key frames are determined according to the visual data stream of the central vehicle, and a local map corresponding to the central vehicle is generated according to the key frames; the central vehicle is a near-ground air vehicle or a ground vehicle; the visual data stream includes a number of video frames.

[0010] The key frame information is sent to the ground vehicle through the near-ground and airborne vehicle; the key frame information is parsed to obtain the extracted parsed information; the parsed information includes the key frame camera pose and map point location information; the key frame information is a message including the key frame and map point information.

[0011] Through the ground vehicle, the local map of the ground vehicle is updated according to the parsed information to obtain an updated local map of the ground vehicle; and the updated local map of the ground vehicle and the local map of the near-ground and airborne vehicle are merged to obtain a global map.

[0012] The global map is optimized and adjusted using the posture graph optimization method and the global bundling adjustment method to obtain an optimized global map.

[0013] The location identification database is updated according to the optimized global map to obtain an updated location identification database.

[0014] In a second aspect, the present invention provides a vehicle networked collaborative positioning and environment mapping system for implementing the method for networked collaborative positioning and environment mapping of air-to-ground vehicles described in the first aspect, comprising the following modules.

[0015] The networked collaboration module is used to: determine the communication frequency based on vehicle parameters; determine the optimal communication path based on the total communication quality score on all possible paths; the vehicle parameters include the total capacity of the channel, the total number of near-ground and airborne vehicles and ground vehicles in the network, and the average distance between vehicles.

[0016] A local map generation module is used to: for each central vehicle, determine a number of key frames based on the visual data stream of the central vehicle, and generate a local map corresponding to the central vehicle based on the key frames; the central vehicle is a near-ground air vehicle or a ground vehicle; the visual data stream includes a number of video frames.

[0017] The information transmission module is used to: send key frame information to a ground vehicle through a near-ground and airborne vehicle; extract and analyze the key frame information; the analyzed information includes the key frame camera pose and map point location information; the key frame information is a message including key frame and map point information.

[0018] The local map update module is used to: update the local map of the ground vehicle according to the parsed information through the ground vehicle to obtain the updated local map of the ground vehicle; and merge the updated local map of the ground vehicle with the local map of the near-ground and airborne vehicle to obtain the global map.

[0019] The global map optimization module is used to optimize and adjust the global map using a posture graph optimization method and a global bundling adjustment method to obtain an optimized global map.

[0020] The database updating module is used to update the location identification database according to the optimized global map to obtain an updated location identification database.

[0021] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the present invention provides a method and system for collaborative positioning and environmental mapping of networked air-ground vehicles, firstly determining the vehicle parameters to determine the communication frequency; determining the optimal communication path according to the total communication quality score on all possible paths; the vehicle parameters include the total capacity of the channel, the total number of near-ground and airborne vehicles and ground vehicles in the network, and the average distance between vehicles; for each central vehicle, determining a number of key frames according to the visual data stream of the central vehicle, and generating a local map corresponding to the central vehicle according to the key frames; the central vehicle is a near-ground and airborne vehicle or a ground vehicle; the visual data stream includes a number of video frames; and the key frame information is sent to the near-ground and airborne vehicle through the near-ground and airborne vehicle. Send to the ground vehicle; extract and analyze the key frame information to obtain the extracted analysis information; the analysis information includes the key frame camera pose and map point location information; the key frame information is a message including the key frame and map point information; the ground vehicle updates the local map of the ground vehicle according to the analysis information to obtain the updated local map of the ground vehicle; and fuses the updated local map of the ground vehicle with the local map of the near-ground and airborne vehicle to obtain the global map; optimize and adjust the global map using the posture graph optimization method and the global bundling adjustment method to obtain the optimized global map; update the location identification database according to the optimized global map to obtain the updated location identification database. The present invention ensures efficient collaboration between the two parties in data processing and map construction, and effectively improves the navigation and positioning accuracy of the three-dimensional transportation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A schematic flow chart of the vehicle networking collaborative positioning and environment mapping method provided in Example 1 of the present invention.

[0024] Figure 2 A schematic diagram of the respective operation processes in the networked collaborative positioning and environmental mapping of near-ground and airborne vehicles and ground vehicles provided in Example 1 of the present invention.

[0025] Figure 3 A block diagram of the vehicle networking collaborative positioning and environment mapping system provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] With the development of electric vertical take-off and landing (eVTOL) technology and near-ground and air economy, the importance of ground vehicles and near-ground and air vehicles in the three-dimensional transportation system has become increasingly prominent. The purpose of the present invention is to provide a method and system for networked collaborative positioning and environmental mapping of air-ground vehicles, aiming to overcome the limitations of traditional ground vehicles and near-ground and air vehicles in positioning and navigation, and improve the accuracy of dynamic positioning and navigation through collaborative synchronous positioning and mapping technology (SLAM). Near-ground and air vehicles focus on data capture and information transmission from a high-altitude perspective, including setting communication frequency, executing visual odometer (VO), sending keyframe information, and receiving and updating global map data optimized by ground vehicles. At the same time, the ground vehicle is responsible for data capture from the ground perspective, receiving aerial information, and performing map management and fusion, including processing received keyframes, applying multi-dimensional fusion positioning algorithms, optimizing global maps, and feeding back optimized data to air vehicles. This integrated process ensures efficient collaboration between the two parties in data processing and map construction, and effectively improves the navigation and positioning accuracy of the three-dimensional transportation system.

[0028] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] Example 1: Figure 1 and Figure 2 As shown, this embodiment is used to provide a method for networked collaborative positioning and environment mapping of air-to-ground vehicles, including the following steps.

[0030] S1: Determine the vehicle parameters to determine the communication frequency; determine the optimal communication path based on the total communication quality score on all possible paths; the vehicle parameters include the total capacity of the channel, the total number of near-ground and ground vehicles in the network, and the average distance between vehicles.

[0031] S2: For each central vehicle, a number of key frames are determined according to the visual data stream of the central vehicle, and a local map corresponding to the central vehicle is generated according to the key frames; the central vehicle is a near-ground air vehicle or a ground vehicle; the visual data stream includes a number of video frames.

[0032] S3: Send the key frame information to the ground vehicle through the near-ground air vehicle; extract the parsed information by parsing the key frame information; the parsed information includes the key frame camera pose and map point location information; the key frame information is a message including the key frame and map point information.

[0033] S4: By means of a ground vehicle, the local map of the ground vehicle is updated according to the parsed information to obtain an updated local map of the ground vehicle; and the updated local map of the ground vehicle is merged with the local map of the near-ground and airborne vehicle to obtain a global map.

[0034] S5: Optimize and adjust the global map using the posture graph optimization method and the global bundling adjustment method to obtain an optimized global map.

[0035] S6: updating the location identification database according to the optimized global map to obtain an updated location identification database.

[0036] Both vehicles (NEAV and ground vehicle) need to be equipped with communication modules that support self-organizing network technology and frequency adjustment, visual sensors (such as monocular cameras), computing modules, data storage modules, and database management modules. When the NEAV and ground vehicle with collaborative localization and mapping capabilities are within the specified operating range, the collaborative localization and mapping method begins to operate.

[0037] Step S1 is used to set the communication frequency , through self-organizing network technology, networked collaboration between near-ground and air vehicles and ground vehicles is achieved.

[0038] Step S1: Communication frequency setting and self-organizing network establishment process, including determining the communication frequency according to the distance between vehicles and communication requirements , optimize frequency selection to balance network capacity and physical distance, and adopt self-organizing network technology, with special emphasis on dynamically adjusting communication paths according to the real-time position and interaction of air and ground vehicles to achieve efficient and stable network connection.

[0039] Specifically: First, determine the frequency used for inter-vehicle communication. The key to selecting the communication frequency is to balance the transmission rate and channel quality to ensure the efficiency and stability of data transmission. The unit of frequency is usually Hertz (Hz), and its selection can be preliminarily estimated by formula (1).

[0040] (1).

[0041] in, Represents the selected communication center frequency (communication frequency), represents the total capacity of the channel, Represents the total number of near-ground airborne vehicles and ground vehicles in the network, represents the average inter-vehicle distance. Formula (1) aims to balance the network capacity and the physical distance between vehicles to optimize the choice of communication frequency.

[0042] In step S1, the optimal communication path is determined according to the total communication quality scores on all possible paths, specifically including: for each possible path, the total communication quality score of the possible path is calculated according to the communication quality scores of all nodes on the possible path; and the possible path corresponding to the maximum communication quality score is determined as the optimal communication path.

[0043] Then, self-organizing network technology is used to establish and maintain network connections between vehicles. The advantage is that it can dynamically select the optimal communication path based on the real-time location of the vehicle and the status of the surrounding available vehicles. The decision of path selection can be simplified by the following routing selection formula.

[0044] (2).

[0045] in, represents the optimal communication path, represents the set of all possible paths, represents the number of nodes on the possible path, Indicates nodes in possible paths The communication quality score on . Formula (2) helps the algorithm evaluate and select the path with the highest communication quality.

[0046] Step S2 performs the following process: the two vehicles perform keyframe-based visual odometry (VO), capture visual data and calculate the k Feature points in keyframes and map points A local map is created and maintained on a local server to represent the current location.

[0047] Through step S1, this embodiment ensures that the communication frequency is most suitable for the distance and needs between vehicles, and also fully utilizes the advantages of self-organizing network technology to achieve efficient and stable network connection, laying a solid foundation for subsequent steps.

[0048] Step S2, during the visual odometry execution and local map construction process, includes capturing visual data from different perspectives (from the perspectives of the near-ground air vehicle and the ground vehicle, respectively), dynamically selecting key frames, extracting and matching feature points, and updating the respective local maps based on these data.

[0049] In step S2, a number of key frames are determined based on the visual data stream of the central vehicle, specifically including: determining a number of key frames based on feature points on each video frame in the visual data stream of the central vehicle and the degree of visual change between any two video frames.

[0050] In step S2, a local map corresponding to the central vehicle is generated according to the key frame, specifically comprising: using a visual mileage calculation method to generate a local map corresponding to the central vehicle according to the key frame.

[0051] First, keyframes are dynamically selected based on the visual data streams of near-ground and ground vehicles, respectively. The keyframe selection criteria are based on the distribution and number of feature points, as well as the degree of visual change between frames; then, feature points are extracted in each keyframe and matched with the feature points of the previous keyframe; finally, based on the feature point matching results, the local map is updated, including the location of map points. The local map is maintained on the vehicle's local server to reflect the vehicle's current position and surrounding environment in real time. It can provide continuous and accurate location information, providing reliable data support for collaborative operations and environmental perception between vehicles.

[0052] Both the near-ground airborne vehicle and the ground vehicle perform keyframe-based visual odometry (VO) to collect visual data and calculate the local map. This process can be mathematically expressed as shown in formula (3).

[0053] (3).

[0054] In formula (3), For the A local map of keyframes, is the feature point of the key frame, The location of the map point.

[0055] In step S3, the near-ground vehicle The messages are packaged and transmitted to the ground vehicle.

[0056] The near-ground vehicle packages the message containing keyframes and map point information and transmits it to the ground vehicle according to the predetermined communication protocol. The message format is defined as , containing the camera poses of the keyframes and map point location After the ground vehicle receives the data, it needs to parse it to extract the key frame camera pose and map point location information. This process can be simplified by formula (4).

[0057] (4).

[0058] In formula (4), Indicates the parsed information of the ground vehicle.

[0059] The above steps ensure efficient and accurate data transmission and parsing to guarantee the accuracy and reliability of map fusion and location identification in subsequent steps.

[0060] In step S4, the ground vehicle performs map management tasks, processes new keyframes, and performs location recognition within the map. It uses a multidimensional fusion localization algorithm (MDLA) to solve the pose transformation relationship between the two sets of matching points and fuse the maps of the two vehicles into a comprehensive global map.

[0061] In step S4, during the map management and fusion process, a multi-dimensional fusion positioning algorithm is used to minimize the reprojection error and estimate the relative pose transformation between the two vehicles. The multi-dimensional fusion positioning algorithm is characterized by including a time-dependent dynamic scale factor based on the relative speed and distance adjustment between the vehicles, as well as accurately calculated rotation matrices and translation vectors to achieve high-precision fusion of map points between different vehicles.

[0062] The ground vehicle first integrates the received new keyframe information into its local map. This process involves adding the new map points and the keyframe's camera pose to the existing map structure.

[0063] In step S4, the updated local map of the ground vehicle and the local map of the vehicle near the ground and in the air are fused, specifically including: using a multi-dimensional fusion positioning algorithm to fuse the updated local map of the ground vehicle and the local map of the vehicle near the ground and in the air.

[0064] The multi-dimensional fusion localization algorithm (MDLA) is applied to process the map fusion between the two vehicles. The purpose of the MDLA algorithm is to estimate the relative pose transformation between the two vehicles by minimizing the reprojection error. The mathematical expression of the algorithm is shown in formula (5).

[0065] (5).

[0066] In formula (5), represents the reprojection error, and Represent the matching map points on the near-ground air vehicle and the ground vehicle respectively. is a time-dependent dynamic scaling factor that can be adjusted based on the relative speed and distance between vehicles. is the rotation matrix of the ground vehicle, is the translation vector. The pose transformation parameters calculated by the MDLA algorithm are used to fuse the maps of the two vehicles to build a comprehensive global map. This step ensures that it can adapt to different scenarios and conditions, making the algorithm more flexible and adaptable to changing environments, and ensuring the continuity and accuracy of the map, which is crucial for subsequent navigation and positioning tasks.

[0067] In the process of global map optimization and update in step S5, the posture graph optimization method and the global bundle adjustment method (BA) are used to carefully optimize the global map, especially by adjusting the camera pose and map point positions to minimize the reprojection error. This technology uses efficient numerical optimization methods to continuously adjust and improve the accuracy of the map in an iterative manner. The map accuracy can be further improved. The ground vehicle updates the optimized global map to the map stack of the near-ground and airborne vehicle.

[0068] The global map is optimized using pose graph optimization and bundle adjustment techniques to fine-tune the global map, thereby improving its accuracy and reliability. Pose graph optimization is a key technology used to improve map accuracy, which focuses on optimizing the camera pose (i.e., the position and orientation of each keyframe); global bundle adjustment is a comprehensive optimization process that aims to minimize the reprojection error by adjusting the camera pose and map point positions. The optimization process of the global map can be expressed by formula (6).

[0069] (6).

[0070] In formula (6), is the total reprojection error, It is k The first keyframe i The observed location of the map point, It is a map point In the k The predicted projected position in each keyframe is calculated. In order to minimize the reprojection error, the gradient descent method is used to continuously adjust the camera pose and map point position through an iterative process to reduce the overall error. The optimized global map is then updated to the map stack of the near-ground and airborne vehicle to ensure the latest and accurate map data.

[0071] In step S6, the location identification database update process involves integrating new map point locations and camera pose data into the existing location identification database. It is particularly emphasized that after the near-ground and airborne vehicle receives and processes the optimized global map data provided by the ground vehicle for collaborative positioning and environment mapping, the database is updated in real time to ensure that it contains the latest map information, so as to provide support for future navigation and positioning of the near-ground and airborne vehicle and the ground vehicle.

[0072] The location recognition database contains information such as map point location and camera pose, and its update formula is: , integrating new data into the location identification database to support future navigation and positioning.

[0073] The vehicle networked collaborative positioning and environmental mapping method provided in this embodiment requires the configuration of: a low-altitude air vehicle equipped with a high-definition camera and communication equipment and a ground rescue vehicle equipped with similar equipment. Its specific networked collaborative positioning and environmental mapping operation process includes the following steps.

[0074] Step 1: The ground rescue vehicle and the near-ground air vehicle establish a communication connection through a self-organizing network and set a suitable communication frequency.

[0075] Step 2: During the flight of the near-ground air vehicle, the visual data of the surrounding environment is captured through the visual odometry and sent to the ground rescue vehicle in real time.

[0076] Step 3: After receiving the data, the ground rescue vehicle processes the visual data of the two vehicles through a multi-dimensional fusion positioning algorithm unit, solves the posture transformation relationship, and fuses them to generate a global map.

[0077] Step 4: The global map is optimized and adjusted through the posture graph optimization and the global bundle adjustment unit to improve the accuracy and reliability of the map.

[0078] Step 5: The ground rescue vehicle updates the optimized global map to the location identification database and the map stack of the near-ground and airborne vehicle to provide accurate navigation and positioning support for the rescue operation.

[0079] The networked collaborative positioning and environmental mapping method of near-ground air vehicles and ground vehicles has effectively improved the operational efficiency and success rate in urban search and rescue missions, proving the practicality and effectiveness of the system.

[0080] The method provided in this embodiment realizes efficient collaborative network positioning and environment mapping through the above-mentioned computing and communication technologies, and is suitable for application scenarios of various near-ground and airborne vehicles and ground vehicles.

[0081] Compared with the prior art, this embodiment has the following beneficial effects.

[0082] 1) Multi-dimensional environmental perception and accurate map construction: This embodiment combines observation data from near-ground and airborne vehicles and ground vehicles to achieve multi-dimensional environmental perception from different angles and heights. This comprehensive perspective not only improves the accuracy of environmental mapping, but also enhances the ability to understand complex terrain and dynamic environments.

[0083] 2) Efficient data processing and optimized collaborative work: Through advanced data processing algorithms and self-organizing network technology, this embodiment can efficiently process large amounts of data and optimize the collaborative work between the two types of vehicles. This not only improves the speed and accuracy of data processing, but also ensures efficient collaboration in the execution of various tasks.

[0084] 3) Accurate positioning and powerful navigation support: This embodiment uses precise positioning algorithms and a continuously updated location identification database to provide accurate positioning information for various navigation tasks. This accuracy is critical for applications that require precise navigation and positioning, such as emergency response and urban planning.

[0085] Embodiment 2: In order to execute the method corresponding to the above embodiment 1 to achieve the corresponding functions and technical effects, the following provides a vehicle networked collaborative positioning and environment mapping system for the air-to-ground vehicle networked collaborative positioning and environment mapping method described in embodiment 1, such as Figure 3 As shown, it includes the following modules.

[0086] The networked collaborative module T1 is used to: determine the vehicle parameters to determine the communication frequency; determine the optimal communication path based on the total communication quality score on all possible paths; the vehicle parameters include the total capacity of the channel, the total number of near-ground and airborne vehicles and ground vehicles in the network, and the average distance between vehicles.

[0087] The local map generation module T2 is used to: for each central vehicle, determine a number of key frames based on the visual data stream of the central vehicle, and generate a local map corresponding to the central vehicle based on the key frames; the central vehicle is a near-ground air vehicle or a ground vehicle; the visual data stream includes a number of video frames.

[0088] The information transmission module T3 is used to: send the key frame information to the ground vehicle through the near-ground and airborne vehicle; extract and analyze the key frame information; the analyzed information includes the key frame camera pose and map point location information; the key frame information is a message including the key frame and map point information.

[0089] The local map update module T4 is used to: update the local map of the ground vehicle according to the parsed information through the ground vehicle to obtain the updated local map of the ground vehicle; and merge the updated local map of the ground vehicle with the local map of the near-ground and airborne vehicle to obtain the global map.

[0090] The global map optimization module T5 is used to optimize and adjust the global map using a posture graph optimization method and a global bundling adjustment method to obtain an optimized global map.

[0091] The database updating module T6 is used to update the location identification database according to the optimized global map to obtain an updated location identification database.

[0092] The near-ground and airborne vehicle module is equipped with communication equipment, visual acquisition devices and processors, which are used to execute visual odometers, capture visual data, and exchange data with ground vehicles through self-organizing network technology.

[0093] The ground vehicle module is also equipped with communication equipment, visual acquisition devices and processors to receive data transmitted by near-ground and airborne vehicles and perform map management tasks, including map fusion, posture transformation solution, and global map optimization and update.

[0094] Embedded in the processor of the ground vehicle module, it is used to process posture transformation relationships and achieve high-precision map fusion.

[0095] The posture map optimization and global bundle adjustment unit is integrated into the ground vehicle module and is responsible for the optimization and adjustment of the global map to ensure map accuracy.

[0096] The location identification database is connected to the near-earth air vehicle and ground vehicle modules to store updated global map data to support future navigation and positioning.

[0097] Among them, the networked collaborative module T1 includes an optimal communication path determination unit, which is used to: for each possible path, calculate the total communication quality score of the possible path according to the communication quality scores of all nodes on the possible path; and determine the possible path corresponding to the maximum total communication quality score as the optimal communication path.

[0098] The local map generation module T2 includes a local map generation unit, which is used to generate a local map corresponding to the central vehicle according to the key frame using a visual mileage calculation method.

[0099] The local map update module T4 includes a fusion unit, which is used to fuse the updated local map of the ground vehicle and the local map of the near-ground and airborne vehicle using a multi-dimensional fusion positioning algorithm.

[0100] The local map generation module T2 includes a key frame determination unit, which is used to determine a number of key frames based on the feature points on each video frame in the visual data stream of the central vehicle and the degree of visual change between two video frames.

[0101] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for collaborative positioning and environment mapping of air-ground vehicle networking, characterized in that: include: Determine the communication frequency based on vehicle parameters; determine the optimal communication path based on the total communication quality score on all possible paths; vehicle parameters include the total capacity of the channel, the total number of near-ground and ground vehicles in the network, and the average distance between vehicles; Make a preliminary estimate; among them, Represents the selected communication frequency, represents the total capacity of the channel, Represents the total number of near-ground airborne vehicles and ground vehicles in the network, represents the average inter-vehicle distance; For each central vehicle, a number of key frames are determined according to a visual data stream of the central vehicle, and a local map corresponding to the central vehicle is generated according to the key frames; the central vehicle is a near-ground air vehicle or a ground vehicle; the visual data stream includes a number of video frames; The key frame information is sent to the ground vehicle through the near-ground and airborne vehicle; the key frame information is parsed to obtain the extracted parsed information; the parsed information includes the key frame camera pose and the map point position information; the key frame information is a message including the key frame and the map point information; By means of a ground vehicle, the local map of the ground vehicle is updated according to the parsed information to obtain an updated local map of the ground vehicle; and the updated local map of the ground vehicle is merged with the local map of the vehicle near the ground and in the air to obtain a global map; The global map is optimized and adjusted by using the posture graph optimization method and the global bundle adjustment method to obtain the optimized global map. Indicates that, is the total reprojection error, It is k The first keyframe i The observed location of the map point, It is a map point In the k The predicted projected position in each key frame is calculated; the camera pose and map point position are adjusted through an iterative process using the gradient descent method to reduce the overall error; The location identification database is updated according to the optimized global map to obtain an updated location identification database.

2. The method for collaborative positioning and environment mapping of air-to-ground vehicles networked according to claim 1, characterized in that: The optimal communication path is determined based on the total communication quality scores on all possible paths, including: For each possible path, a total communication quality score of the possible path is calculated according to the communication quality scores of all nodes on the possible path; The possible path corresponding to the communication quality total score with the largest value is determined as the optimal communication path.

3. The method for collaborative positioning and environment mapping of air-to-ground vehicles networked according to claim 1, characterized in that: Generating a local map corresponding to the central vehicle according to the key frame specifically includes: A local map corresponding to the central vehicle is generated according to the key frames by using a visual odometry method.

4. The method for collaborative positioning and environment mapping of air-to-ground vehicles networked according to claim 1, characterized in that: The updated local map of the ground vehicle and the local map of the vehicle near the ground and in the air are merged, including: The updated local map of the ground vehicle and the local map of the near-ground and airborne vehicle are fused using a multi-dimensional fusion positioning algorithm.

5. The method for collaborative positioning and environment mapping of air-to-ground vehicles networked according to claim 1, characterized in that: A number of key frames are determined according to the visual data stream of the central vehicle, specifically including: A number of key frames are determined based on the feature points on each video frame in the visual data stream of the central vehicle and the degree of visual change between any two video frames.

6. A vehicle networked collaborative positioning and environment mapping system for implementing the air-to-ground vehicle networked collaborative positioning and environment mapping method of claim 1, characterized in that: include: The networked collaboration module is used to: determine the communication frequency based on vehicle parameters; determine the optimal communication path based on the total communication quality score on all possible paths; vehicle parameters include the total capacity of the channel, the total number of near-ground and ground vehicles in the network, and the average distance between vehicles; The local map generation module is used to: for each central vehicle, determine a number of key frames according to the visual data stream of the central vehicle, and generate a local map corresponding to the central vehicle according to the key frames; the central vehicle is a near-ground air vehicle or a ground vehicle; the visual data stream includes a number of video frames; The information transmission module is used to: send the key frame information to the ground vehicle through the near-ground and airborne vehicle; extract and analyze the key frame information to obtain the analyzed information; the analyzed information includes the key frame camera pose and the map point position information; the key frame information is a message including the key frame and the map point information; A local map update module is used to: update the local map of the ground vehicle according to the parsing information through the ground vehicle to obtain an updated local map of the ground vehicle; and merge the updated local map of the ground vehicle with the local map of the near-ground and airborne vehicle to obtain a global map; A global map optimization module is used to optimize and adjust the global map using a posture graph optimization method and a global bundling adjustment method to obtain an optimized global map; The database updating module is used to update the location identification database according to the optimized global map to obtain an updated location identification database.

7. The air-to-ground vehicle networked collaborative positioning and environment mapping system according to claim 6, characterized in that: The networked collaboration module includes an optimal communication path determination unit, which is used to: For each possible path, a total communication quality score of the possible path is calculated according to the communication quality scores of all nodes on the possible path; The possible path corresponding to the communication quality total score with the largest value is determined as the optimal communication path.

8. The air-to-ground vehicle networked collaborative positioning and environment mapping system according to claim 6, characterized in that: The local map generation module includes a local map generation unit, which is used to: generate a local map corresponding to the central vehicle according to the key frame using a visual mileage calculation method.

9. The air-to-ground vehicle networked collaborative positioning and environment mapping system according to claim 6, characterized in that: The local map update module includes a fusion unit, which is used to: use a multi-dimensional fusion positioning algorithm to fuse the updated local map of the ground vehicle and the local map of the near-ground and airborne vehicle.

10. The air-to-ground vehicle networked collaborative positioning and environment mapping system according to claim 6, characterized in that: The local map generation module includes a key frame determination unit, which is used to determine a number of key frames according to the feature points on each video frame in the visual data stream of the central vehicle and the degree of visual change between two video frames.

Images