Unmanned aerial vehicle group navigation architecture in satellite denial environment and collaborative navigation method

Abstract

The invention belongs to the technical field of unmanned aerial vehicle group collaborative navigation, and particularly relates to an unmanned aerial vehicle group navigation architecture and a collaborative navigation method in a satellite denial environment, and the method relates to establishment of the unmanned aerial vehicle group navigation architecture in the satellite denial environment and visual collaborative navigation in the satellite denial environment. The method is detailed description of an algorithm flow part in the architecture. Firstly, a framework is designed by taking vision as a core, and resource allocation in multiple aspects such as hardware, a sensor, a calculator, an algorithm and a flow is included, so that the system is relatively low in cost and has a collaborative navigation capability. And secondly, a method that the host constructs real-time reference data and the slaves utilize the reference data to assist in positioning is designed, so that the dependence of the slaves on the types, the number and the precision of the sensors is reduced, and the slaves can realize positioning and orientation only by relying on the camera sensors.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) swarm cooperative navigation, and specifically to a UAV swarm navigation architecture and cooperative navigation method in a satellite-denied environment. Background Technology

[0002] Unmanned aerial vehicle (UAV) swarm warfare is the mainstream of future intelligent warfare, a consensus reached by domestic and international weapons and equipment R&D institutions. Cooperative navigation in satellite-denied environments is a key technology that UAV swarms urgently need to overcome, and it remains a research challenge. Firstly, navigation and positioning in satellite-denied environments require a large number of auxiliary sensors, such as visual, geomagnetic, altimeter, and polarized light sensors, resulting in very high resource requirements and costs. Secondly, given the premise of individual UAV positioning, there is currently no mature technical solution for achieving relative positioning among UAV swarms.

[0003] Analysis of the above problems shows that the difficulties of cluster cooperative navigation in satellite-denied environments can be summarized into two points: (1) lack of cooperative navigation architecture in satellite-denied environments, that is, a navigation architecture with better comprehensive performance in terms of cost, performance, and stability; (2) lack of means of relative positioning and navigation.

[0004] In view of this, the present invention designs a navigation architecture and visual cooperative navigation method for unmanned aerial vehicle swarms in satellite-denied environments. The cooperative navigation system has low cost, high accuracy and high stability. Summary of the Invention

[0005] (a) Technical problems to be solved The technical problem to be solved by this invention is: how to provide a cooperative navigation architecture and navigation method for drone swarms that is low in cost, high in accuracy, and high in stability.

[0006] (II) Technical Solution To address the aforementioned technical problems, this invention provides a navigation architecture and cooperative navigation method for unmanned aerial vehicle (UAV) swarms in a satellite-denied environment. The method involves two parts: establishing a navigation architecture for UAV swarms in a satellite-denied environment and visual cooperative navigation in a satellite-denied environment.

[0007] Among them, the establishment of the UAV swarm navigation architecture in the satellite denial environment, in terms of hardware system: the UAV swarm is divided into 1 host and multiple slaves. The host carries a high-precision inertial navigation system, a downward-looking camera, and a radar altimeter, while the slaves carry a low-precision inertial navigation system and a downward-looking camera. All sensors have been calibrated. The host camera perceives the downward view image in real time and records it as the host downward view image. At the same time, the high-precision inertial navigation is used to mark the camera coordinate system corresponding to the host downward view image. Then, the host downward view image, along with the camera intrinsic parameters and camera coordinate system, is transmitted to all slave devices through the data link. The slave devices then receive the above-mentioned cameras for matching and navigation. The host computer, equipped with a high-performance computing calculator, executes a scene matching algorithm to generate baseline data. The slave computer, equipped with a low-performance computing calculator, matches the baseline data to achieve positioning.

[0008] In the visual cooperative navigation process under satellite denial, the sensor offline calibration data relied upon includes: the host camera intrinsic parameters denoted as K, and the host camera optical axis denoted as... The horizontal field of view of the main camera is denoted as FOV The horizontal dimension of the main camera image is denoted as The internal parameters of the camera are recorded as follows: The relative rotation matrix between the host inertial navigation system and the host camera is known, denoted as . The main and slave cameras are the same size, have the same field of view, and their optical axes are denoted as . ; The specific steps of the visual cooperative navigation process under satellite-denied conditions are as follows: Step A: The host perceives the lower view image, the... Frame image denoted as Scene matching is performed to achieve host positioning and obtain the optical axis of the host camera. The corresponding latitude and longitude are denoted as Continuous attitude information is provided by high-precision inertial navigation. The pose matrix corresponding to the frame image is denoted as: The scene matching described is a mature method in the field of visual navigation. Step B: Image annotation, including resolution, localization information, and pose information; including: Step B.1: Mark the bottom view image resolution of the host camera based on the host camera's field of view, host image resolution, and radar altimeter information:

[0009] Step B.2: Mark the positioning information of the optical axis based on the scene matching results in Step A; Step B.3: According to step A Inertial navigation attitude information is used to annotate the attitude matrix between the camera coordinate system and the navigation coordinate system. :

[0010] Step C: Data Link Transfer , , , Give each slave aircraft; Step D: Each slave device performs the same steps after receiving data, and the real-time image perceived by the slave device is denoted as... Execute in sequence: Step D.1: For and Performing a feature extraction process based on the SURF algorithm and a feature matching process based on brute-force matching yields coarse matching point pairs. ; Step D.2: For Perform a feature selection process based on the RANSAC algorithm to obtain exact matching point pairs. Given the fundamental matrix F, calculate the relative rotation matrix based on the fundamental matrix. ; Step E: Attitude calculation, based on step D.2 With step C The attitude matrix of the slave machine is calculated according to the chain rule:

[0011] Step F: Locating and solving, first based on K from step A and In step D.2 Calculate the optical axis of the slave machine exist pixel position :

[0012] Furthermore, based on K in step A, In step B.1, s is calculated. Compared to Translation amount:

[0013] Further according to step E Calculate the optical axis center of the slave machine Distance difference between the slave and slave carriers:

[0014] Furthermore, according to step A Calculate carrier positioning : ; G: Execute step A for the (i+1)th frame image.

[0015] in, It is a constant of 6378137.

[0016] In step D, both the feature extraction process and the feature matching process are mature methods in the field of image matching.

[0017] In step D.2, the RANSAC algorithm is used to calculate the exact matching point pairs and the fundamental matrix F, which is a mature method in the field of image matching.

[0018] In step D.2, calculating the relative rotation matrix based on the fundamental matrix F is a mature method in the image processing field.

[0019] (III) Beneficial Effects Compared with the prior art, the present invention has the following advantages: (1) This invention provides a navigation architecture for unmanned aerial vehicle swarms in a satellite-denied environment, with reasonable allocation of computing resources and sensor resources, and low system cost.

[0020] (2) Based on a visual cooperative navigation method for UAV swarms under satellite denial environment, this method cleverly utilizes the high-precision reference data provided by the host to assist the slave in completing the positioning, providing a new method for the field of UAV swarm cooperative navigation. Attached Figure Description

[0021] Figure 1 This is the overall flowchart of the present invention. Detailed Implementation

[0022] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0023] To address the aforementioned technical problems, this invention provides a navigation architecture and cooperative navigation method for unmanned aerial vehicle (UAV) swarms in a satellite-denied environment. The method involves two parts: establishing a navigation architecture for UAV swarms in a satellite-denied environment and visual cooperative navigation in a satellite-denied environment.

[0024] Among them, the establishment of the UAV swarm navigation architecture in the satellite denial environment, in terms of hardware system: the UAV swarm is divided into 1 host and multiple slaves. The host carries a high-precision inertial navigation system, a downward-looking camera, and a radar altimeter, while the slaves carry a low-precision inertial navigation system and a downward-looking camera. All sensors have been calibrated. The host camera perceives the downward view image in real time and records it as the host downward view image. At the same time, the high-precision inertial navigation is used to mark the camera coordinate system corresponding to the host downward view image. Then, the host downward view image, along with the camera intrinsic parameters and camera coordinate system, is transmitted to all slave devices through the data link. The slave devices then receive the above-mentioned cameras for matching and navigation. The host computer, equipped with a high-performance computing calculator, executes a scene matching algorithm to generate baseline data. The slave computer, equipped with a low-performance computing calculator, matches the baseline data to achieve positioning.

[0025] In the visual cooperative navigation process under satellite denial, the sensor offline calibration data relied upon includes: the host camera intrinsic parameters denoted as K, and the host camera optical axis denoted as... The horizontal field of view of the main camera is denoted as FOV The horizontal dimension of the main camera image is denoted as The internal parameters of the camera are recorded as follows: The relative rotation matrix between the host inertial navigation system and the host camera is known, denoted as . The main and slave cameras are the same size, have the same field of view, and their optical axes are denoted as . ; The specific steps of the visual cooperative navigation process under satellite-denied conditions are as follows: Step A: The host perceives the downward-facing image, and the i-th frame image is denoted as... Scene matching is performed to achieve host positioning and obtain the optical axis of the host camera. The corresponding latitude and longitude are denoted as Continuous attitude information is provided by a high-precision inertial navigation system, and the attitude matrix corresponding to the i-th frame image is denoted as . The scene matching described is a mature method in the field of visual navigation. Step B: Image annotation, including resolution, localization information, and pose information; including: Step B.1: Mark the bottom view image resolution of the host camera based on the host camera's field of view, host image resolution, and radar altimeter information:

[0026] Step B.2: Mark the positioning information of the optical axis based on the scene matching results in Step A; Step B.3: According to step A Inertial navigation attitude information is used to annotate the attitude matrix between the camera coordinate system and the navigation coordinate system. :

[0027] Step C: Data Link Transfer , , , Give each slave aircraft; Step D: Each slave device performs the same steps after receiving data, and the real-time image perceived by the slave device is denoted as... Execute in sequence: Step D.1: For and Performing a feature extraction process based on the SURF algorithm and a feature matching process based on brute-force matching yields coarse matching point pairs. ; Step D.2: For Perform a feature selection process based on the RANSAC algorithm to obtain exact matching point pairs. Given the fundamental matrix F, calculate the relative rotation matrix based on the fundamental matrix. ; Step E: Attitude calculation, based on step D.2 With step C The attitude matrix of the slave machine is calculated according to the chain rule:

[0028] Step F: Locating and solving, first based on K and... In step D.2 Calculate the optical axis of the slave machine exist pixel position :

[0029] Furthermore, based on K in step A, In step B.1, s is calculated. Compared to Translation amount:

[0030] Further according to step E Calculate the optical axis center of the slave machine Distance difference between the slave and slave carriers:

[0031] Furthermore, according to step A Calculate carrier positioning : ; G: Execute step A for the (i+1)th frame of the image.

[0032] in, It is a constant of 6378137.

[0033] In step D, both the feature extraction process and the feature matching process are mature methods in the field of image matching.

[0034] In step D.2, the RANSAC algorithm is used to calculate the exact matching point pairs and the fundamental matrix F, which is a mature method in the field of image matching.

[0035] In step D.2, calculating the relative rotation matrix based on the fundamental matrix F is a mature method in the image processing field.

[0036] Example 1 This embodiment provides a navigation architecture for a drone swarm. In terms of hardware, the drone swarm consists of one host and multiple slave drones. The host carries a high-precision inertial navigation system (INS), a downward-looking camera, and a radar altimeter, while the slave drones carry low-precision INS and downward-looking cameras. All sensors are calibrated. Regarding the algorithm flow, the host's downward-looking camera perceives the downward-looking image in real time and records it as the host's downward-looking image. Simultaneously, the high-precision INS is used to annotate the camera coordinate system corresponding to the host's downward-looking image. Then, the host's downward-looking image, along with camera intrinsic parameters and the camera coordinate system, is transmitted to all slave drones via a data link. The slave drones then receive the aforementioned camera data for matching and navigation. In terms of computing processors, the host is equipped with a high-performance calculator to execute an image matching algorithm to generate reference data. The slave drones are equipped with low-performance calculators to match the reference data to achieve positioning. On the other hand, this embodiment provides a visual cooperative navigation method, wherein the offline calibration data of the sensors on which the method relies is denoted as: the intrinsic parameter of the host camera is denoted as K, and the optical axis of the host camera is denoted as... The horizontal field of view of the main camera is denoted as FOV The horizontal dimension of the main camera image is denoted as The internal parameters of the camera are recorded as follows: The relative rotation matrix between the host inertial navigation system and the host camera is known, denoted as . The main and slave cameras are the same size, have the same field of view, and their optical axes are denoted as . The specific steps of the method are as follows: A. The host perceives the image below, and the i-th frame image is denoted as... Scene matching is performed to achieve host positioning and obtain the optical axis of the host camera. The corresponding latitude and longitude are denoted as Continuous attitude information is provided by a high-precision inertial navigation system, and the attitude matrix corresponding to the i-th frame image is denoted as . The scene matching described is a mature method in the field of visual navigation. B. Image annotation, including resolution, location information, and pose information; B.1 Based on the host camera's field of view, host image resolution, and radar altimeter information, mark the host's bottom view image resolution:

[0037] B.2 Mark the optical axis positioning information based on the scene matching results in step A; B.3 According to step A Inertial navigation attitude information is used to annotate the attitude matrix between the camera coordinate system and the navigation coordinate system. :

[0038] C. Data Link Transmission , , , Give each slave aircraft; D. Each slave device performs the same steps after receiving data; let the real-time image perceived by the slave device be denoted as . Execute in sequence: D.1 pair and Performing a feature extraction process based on the SURF algorithm and a feature matching process based on brute-force matching yields coarse matching point pairs. Both the feature extraction process and the feature matching process are mature methods in the field of image matching; D.2 Perform a feature selection process based on the RANSAC algorithm to obtain exact matching point pairs. Given the fundamental matrix F, calculate the relative rotation matrix based on the fundamental matrix. The exact matching point pairs and the fundamental matrix F calculated using the RANSAC algorithm are... A mature method in the field of image matching is to calculate the relative rotation matrix based on the fundamental matrix F. E. Attitude calculation, based on step D.2 With step C The attitude matrix of the slave machine is calculated according to the chain rule:

[0039] F. Location calculation, first based on K in step A and... In step D.2 Calculate the optical axis of the slave machine exist pixel position :

[0040] Furthermore, based on K in step A, In step B.1, s is calculated. Compared to Translation amount:

[0041] Further according to step E Calculate the optical axis center of the slave machine Distance difference between the slave and slave carriers:

[0042] Furthermore, according to step A Calculate carrier positioning :

[0043] in, It is a constant of 6378137.

[0044] G. Execute step A for the (i+1)th frame image.

[0045] Example 2 One aspect of this embodiment provides a navigation architecture for an unmanned aerial vehicle (UAV) swarm. In terms of hardware, the UAV swarm consists of one master unit and multiple slave units. The master unit carries a high-precision inertial navigation system (INS), a downward-looking camera, and a radar altimeter, while the slave units carry low-precision INS and downward-looking cameras. All sensors are calibrated. In terms of algorithm flow, the master unit's downward-looking camera perceives the downward-looking image in real time and records it as the master unit's downward-looking image. Simultaneously, the high-precision INS is used to annotate the camera coordinate system corresponding to the master unit's downward-looking image. Then, the master unit's downward-looking image, along with camera intrinsic parameters and camera coordinate system, is transmitted to all slave units via a data link. The slave units then receive the aforementioned camera data for matching and navigation. In terms of computing processors, the master unit, equipped with a high-performance computing processor, executes an image matching algorithm to generate reference data. The slave units, equipped with low-performance computing processors, match the reference data to achieve positioning.

[0046] On the other hand, this embodiment provides a visual cooperative navigation method, wherein the offline calibration data of the sensors on which the method relies is denoted as: the intrinsic parameter of the host camera is denoted as K, and the optical axis of the host camera is denoted as... The horizontal field of view of the main camera is denoted as FOV The horizontal dimension of the main camera image is denoted as The internal parameters of the camera are recorded as follows: The relative rotation matrix between the host inertial navigation system and the host camera is known, denoted as . The main and slave cameras are the same size, have the same field of view, and their optical axes are denoted as . The method flow is as follows Figure 1 As shown, the specific steps are as follows: A. The host perceives the image below, and the i-th frame image is denoted as... Scene matching is performed to achieve host positioning and obtain the optical axis of the host camera. The corresponding latitude and longitude are denoted as Continuous attitude information is provided by a high-precision inertial navigation system, and the attitude matrix corresponding to the i-th frame image is denoted as... The scene matching described is a mature method in the field of visual navigation.

[0047] B. Image annotation, including resolution, location information, and pose information.

[0048] B.1 Based on the host camera's field of view, host image resolution, and radar altimeter information, mark the host's bottom view image resolution:

[0049] B.2 Mark the optical axis positioning information based on the scene matching results in step A.

[0050] B.3 According to step A Inertial navigation attitude information is used to annotate the attitude matrix between the camera coordinate system and the navigation coordinate system. :

[0051] C. Data Link Transmission , , , Give it to each slave aircraft.

[0052] D. Each slave device performs the same steps after receiving data; let the real-time image perceived by the slave device be denoted as . Execute in sequence: D.1 pair and Performing a feature extraction process based on the SURF algorithm and a feature matching process based on brute-force matching yields coarse matching point pairs. Both the feature extraction process and the feature matching process are mature methods in the field of image matching.

[0053] D.2 Perform a feature selection process based on the RANSAC algorithm to obtain exact matching point pairs. Given the fundamental matrix F, calculate the relative rotation matrix based on the fundamental matrix. The exact matching point pairs and the fundamental matrix F calculated using the RANSAC algorithm are... A mature method in the field of image matching is to calculate the relative rotation matrix based on the fundamental matrix F.

[0054] E. Attitude calculation, based on step D.2 With step C The attitude matrix of the slave machine is calculated according to the chain rule:

[0055] F. Location calculation, first based on K in step A and... In step D.2 Calculate the optical axis of the slave machine exist pixel position :

[0056] Furthermore, based on K in step A, In step B.1, s is calculated. Compared to Translation amount:

[0057] Further according to step E Calculate the optical axis center of the slave machine Distance difference between the slave and slave carriers:

[0058] Furthermore, according to step A Calculate carrier positioning :

[0059] in, It is a constant of 6378137.

[0060] G. Execute step A for the (i+1)th frame image.

[0061] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A navigation architecture and cooperative navigation method for unmanned aerial vehicle (UAV) swarms in a satellite-denied environment, characterized in that, The method involves two parts: establishing a navigation architecture for unmanned aerial vehicle swarms under satellite denial and visual cooperative navigation under satellite denial.

2. The UAV swarm navigation architecture and cooperative navigation method under satellite denial environment as described in claim 1, characterized in that, The establishment of the unmanned aerial vehicle (UAV) swarm navigation architecture under satellite denial environment involves the following hardware system: the UAV swarm consists of one host and multiple slaves. The host carries a high-precision inertial navigation system, a downward-looking camera, and a radar altimeter, while the slaves carry a low-precision inertial navigation system and a downward-looking camera. The host camera perceives the downward view image in real time and records it as the host downward view image. At the same time, the high-precision inertial navigation is used to mark the camera coordinate system corresponding to the host downward view image. Then, the host downward view image, along with the camera intrinsic parameters and camera coordinate system, is transmitted to all slave devices through the data link. The slave devices then receive the above-mentioned cameras for matching and navigation. The host computer, equipped with a high-performance computing calculator, executes a scene matching algorithm to generate baseline data. The slave computer, equipped with a low-performance computing calculator, matches the baseline data to achieve positioning.

3. The UAV swarm navigation architecture and cooperative navigation method under satellite denial environment as described in claim 2, characterized in that, During the visual cooperative navigation process in the satellite-denied environment, the offline calibration data of the sensors relied upon includes: the host camera intrinsic parameters denoted as K, and the host camera optical axis denoted as... The horizontal field of view of the main camera is denoted as FOV, and the horizontal size of the main camera image is denoted as... The internal parameters of the camera are recorded as follows: The relative rotation matrix between the host inertial navigation system and the host camera is known, denoted as . The main and slave cameras are the same size, have the same field of view, and their optical axes are denoted as . ; The specific steps of the visual cooperative navigation process under satellite-denied conditions are as follows: Step A: The host perceives the downward-view image, and the i-th frame image is denoted as... Scene matching is performed to achieve host positioning and obtain the optical axis of the host camera. The corresponding latitude and longitude are denoted as Continuous attitude information is provided by a high-precision inertial navigation system, and the attitude matrix corresponding to the i-th frame image is denoted as . The scene matching described is a mature method in the field of visual navigation. Step B: Image annotation, including resolution, localization information, and pose information; including: Step B.1: Mark the bottom view image resolution of the host camera based on the host camera's field of view, host image resolution, and radar altimeter information: Step B.2: Mark the positioning information of the optical axis based on the scene matching results in Step A; Step B.3: According to step A Inertial navigation attitude information is used to annotate the attitude matrix between the camera coordinate system and the navigation coordinate system. : Step C: Data Link Transfer , , , Give each slave aircraft; Step D: Each slave device performs the same steps after receiving data, and the real-time image perceived by the slave device is denoted as... Execute in sequence: Step D.1: For and Performing a feature extraction process based on the SURF algorithm and a feature matching process based on brute-force matching yields coarse matching point pairs. ; Step D.2: For Perform a feature selection process based on the RANSAC algorithm to obtain exact matching point pairs. Given the fundamental matrix F, calculate the relative rotation matrix based on the fundamental matrix. ; Step E: Attitude calculation, based on step D.2 With step C The attitude matrix of the slave machine is calculated according to the chain rule: Step F: Locating and solving, first based on K from step A and In step D.2 Calculate the optical axis of the slave machine exist pixel position : Furthermore, based on K in step A, In step B.1, S is calculated. Compared to Translation amount: Further according to step E Calculate the optical axis center of the slave machine Distance difference between the slave and slave carriers: Furthermore, according to step A Calculate carrier positioning : ； G: Execute step A for the (i+1)th frame image.

4. The UAV swarm navigation architecture and cooperative navigation method under satellite denial environment as described in claim 3, characterized in that, It is a constant of 6378137.

5. The UAV swarm navigation architecture and cooperative navigation method under satellite denial environment as described in claim 3, characterized in that, In step D, both the feature extraction process and the feature matching process are mature methods in the field of image matching.

6. The UAV swarm navigation architecture and cooperative navigation method under satellite denial environment as described in claim 3, characterized in that, In step D.2, the RANSAC algorithm is used to calculate the exact matching point pairs and the fundamental matrix F, which is a mature method in the field of image matching.

7. The UAV swarm navigation architecture and cooperative navigation method under satellite denial environment as described in claim 3, characterized in that, In step D.2, calculating the relative rotation matrix based on the fundamental matrix F is a mature method in the image processing field.

8. The UAV swarm navigation architecture and cooperative navigation method under satellite denial environment as described in claim 3, characterized in that, The method described belongs to the field of UAV swarm cooperative navigation technology.

9. The navigation architecture and cooperative navigation method for unmanned aerial vehicle swarms in a satellite-denied environment as described in claim 3, characterized in that, The method provides a navigation architecture for unmanned aerial vehicle swarms in satellite-denied environments, with reasonable allocation of computing and sensor resources and low system cost.

10. The navigation architecture and cooperative navigation method for unmanned aerial vehicle swarms in a satellite-denied environment as described in claim 3, characterized in that, The method cleverly utilizes high-precision reference data provided by the host to assist the slave in completing positioning, providing a new approach for the field of UAV swarm collaborative navigation.

Images