Unmanned aerial vehicle scene matching navigation positioning method and device based on infrared image

Abstract

The invention discloses an unmanned aerial vehicle scene matching navigation positioning method and device based on an infrared image. The method comprises the following steps: S1, connecting an unmanned aerial vehicle with an infrared camera in a strapdown manner; s2, connecting an airborne equipment processor with the infrared camera and an unmanned aerial vehicle flight control system, wherein the airborne equipment processor is pre-loaded with a satellite map of a to-be-flied area; s3, in the cruising stage of the unmanned aerial vehicle, judging whether a satellite navigation signal is valid or not; s4, if the satellite navigation signal fails, acquiring an infrared camera image in real time by the airborne equipment processor through the infrared camera, selecting a to-be-matched area within a threshold range according to a positioning result of the unmanned aerial vehicle at the previous moment, and caching the satellite image of the area into the airborne equipment processor; s5, carrying out structural feature extraction on the infrared camera image and the cached satellite map, and carrying out matching; and S6, calculating actual latitude and longitude coordinates of the infrared camera image according to the optimal matching position to obtain the latitude and longitude of the unmanned aerial vehicle at the current moment. The autonomy of the unmanned aerial vehicle navigation positioning method is improved, the method does not depend on the assistance of external signals, and the safety of the unmanned aerial vehicle in the cruising process is improved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicles (UAVs), and in particular to a method and device for scene matching navigation and positioning of UAVs based on infrared images. Background Art

[0002] With the increasing popularity of drones, drones need to consider navigation and positioning solutions in complex electromagnetic environments, such as those where differential satellite navigation fails or satellite signal interruptions, to ensure high reliability and safety. Most existing drones obtain their position by receiving satellite navigation signals within a specific frequency band. During navigation, they calculate their current position and compare it with pre-set waypoints to complete the navigation process. In such situations, if the satellite navigation signal is lost, interfered with, or even spoofed, the drone may lose its positioning, potentially causing the navigation system to diverge and crash, posing a threat to personnel, facilities, and the drone itself within the flight area.

[0003] Therefore, it is urgent to provide a solution for UAV navigation and positioning. Summary of the Invention

[0004] In order to solve the above problems, the technical solution of the present invention provides a method and device for UAV scene matching navigation and positioning based on infrared images, which can ensure the normal positioning and navigation of the UAV.

[0005] According to a first embodiment of the technical solution of the present invention, a method for scene matching navigation and positioning of a UAV based on infrared images is provided, comprising:

[0006] S1. Strapdown the drone and the infrared camera;

[0007] S2. Connecting an onboard device processor to the infrared camera and the UAV flight control system, wherein the onboard device processor is pre-loaded with a satellite map of the area to be flown;

[0008] S3. During the UAV cruising phase, determining whether the satellite navigation signal received by the UAV is valid;

[0009] S4. If the satellite navigation signal fails, the onboard device processor collects infrared camera images in real time through the infrared camera, selects a to-be-matched area within a threshold range based on the UAV's last positioning result, and caches the satellite image of the area from a pre-loaded satellite map into the onboard device processor;

[0010] S5. Extracting structural features from the infrared camera image and the cached satellite map, and performing matching;

[0011] S6. Calculate the actual latitude and longitude coordinates of the infrared camera image based on the best matching position, and then calculate the latitude and longitude of the drone at the current moment.

[0012] In the above solution, in step S1, the optical axis of the infrared camera is perpendicular to the drone body and faces downward.

[0013] In the above solution, step S2 includes:

[0014] S2.1. The onboard device processor is connected to the UAV flight control system to receive instructions and information from the UAV flight control system;

[0015] S2.2. The airborne device processor is connected to the infrared camera for collecting the infrared camera image in real time.

[0016] In the above scheme, in step S2.1, the onboard device processor receives instructions and information from the UAV flight control system including: the attitude, position and speed of the UAV, the air pressure altitude of the UAV's location and the validity of the satellite navigation status.

[0017] In the above solution, step S4 includes: the onboard device processor acquires the infrared camera image in real time through the image acquisition card and performs orthorectification on the infrared camera image according to the posture of the drone.

[0018] In the above solution, in step S4, orthorectifying the infrared camera image based on the posture of the drone includes:

[0019] S4.1. Select four corner points and a center point of the infrared camera image as control points, and obtain pixel coordinates of the control points;

[0020] S4.2. Calculate the attitude matrix R based on the attitude of the UAV;

[0021] S4.3. Calculate the relative geographic coordinates between the control point and the UAV using a digital elevation model based on the UAV's attitude matrix R;

[0022] S4.4, using the least squares method to obtain the affine matrix M of the pixel coordinates and the real relative geographic coordinates A , through the affine matrix M A An orthorectified image of the infrared camera image is obtained.

[0023] In the above solution, in step S5, a Log-Gabor filter is used to calculate the structural feature maps of the orthorectified infrared camera image and the satellite map respectively, and feature matching is performed between the orthorectified infrared camera image and the satellite map.

[0024] In the above scheme, in step S6, the relative geographic coordinates of the orthorectified infrared camera image are converted into the actual longitude and latitude according to the coordinates of the satellite map of the best matching position, and then the actual longitude and latitude coordinates of the infrared camera are calculated as the longitude and latitude of the drone at the current moment.

[0025] According to a second aspect of the technical solution of the present invention, a device for navigation and positioning of a drone based on infrared images is provided. The device is used to implement the method for navigation and positioning of a drone based on infrared images according to any one of the above-mentioned solutions. The device includes:

[0026] Strapdown module, used to strapdown the drone and infrared camera;

[0027] A pre-loading module is used to connect the airborne device processor to the image acquisition card and the UAV flight control system, wherein the airborne device processor is pre-loaded with a satellite map of the area to be flown;

[0028] The judgment module is used to judge whether the satellite navigation signal is valid during the UAV cruising phase;

[0029] a cache module configured to, if the satellite navigation signal fails, cause the onboard device processor to acquire infrared camera images in real time through the image acquisition card, select a to-be-matched area within a threshold range based on the UAV's last positioning result, and cache the satellite image of the area from a pre-loaded satellite map into the onboard device processor;

[0030] A feature extraction and matching module is used to extract structural features from the infrared camera image and the cached satellite map, and perform matching;

[0031] The calculation module is used to calculate the actual latitude and longitude coordinates of the control point in the infrared camera image according to the best matching position, and then calculate the latitude and longitude of the drone at the current moment.

[0032] According to a third aspect of the technical solution of the present invention, an electronic device is provided, comprising:

[0033] a memory storing executable instructions;

[0034] A processor, wherein the processor runs the executable instructions in the memory to implement the method of any one of the above solutions.

[0035] Beneficial effects of the present invention:

[0036] The present invention discloses a method and device for scene matching navigation and positioning of a UAV based on infrared images. The method obtains the real-time position information of the UAV by matching infrared images taken by the UAV with satellite maps, thereby improving the autonomy of the UAV navigation and positioning method, not relying on the assistance of external signals, improving the safety of the UAV during cruising, and being suitable for navigation and positioning in situations and areas without satellite guidance signals. The method has a high degree of automation, minimizes manual participation throughout the process, and ensures timely and effective handling. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0038] Figure 1 This is a flow chart of the infrared image-based UAV scene matching navigation and positioning method of the present invention;

[0039] Figure 2 This is a flow chart of image acquisition and matching in the present invention;

[0040] Figure 3 This is a flowchart for performing orthorectification in the present invention;

[0041] Figure 4 A schematic diagram of selecting control points in the present invention;

[0042] Figure 5 Schematic diagram of feature extraction between infrared camera images and satellite maps in the present invention;

[0043] Figure 6 This is a schematic diagram of the feature matching results between the infrared camera image and the satellite map in the present invention.

[0044] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0045] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0046] The terms "first," "second," and the like in the description and claims of the present disclosure are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present disclosure described herein can, for example, be implemented in orders other than those illustrated or described herein.

[0047] In addition, the terms "comprises" and "having" and any variations thereof are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product or apparatus.

[0048] Multiple includes two or more.

[0049] It should be understood that the term "and / or" as used in this disclosure simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0050] like Figures 1 to 3 As shown, an embodiment of the technical solution of the present invention provides a method for scene matching navigation and positioning of a UAV based on infrared images, comprising:

[0051] S1. Strapdown the drone and the infrared camera;

[0052] S2. Connecting the onboard device processor to the infrared camera and the UAV flight control system, wherein the onboard device processor is pre-loaded with a satellite map of the area to be flown;

[0053] S3. During the UAV cruising phase, determine whether the satellite navigation signal received by the UAV is valid;

[0054] S4. If the satellite navigation signal fails, the onboard device processor uses the infrared camera to collect infrared camera images in real time, selects a matching area within a threshold range based on the drone's previous positioning result, and caches the satellite image of the area from the pre-loaded satellite map into the onboard device processor;

[0055] S5. Extract structural features from the infrared camera image and the cached satellite map, and perform matching;

[0056] S6. Calculate the actual latitude and longitude coordinates of the infrared camera image based on the best matching position, and then calculate the latitude and longitude of the drone at the current moment.

[0057] In step S1, the infrared camera's optical axis is perpendicular to the drone's body and points downward. There is no relative motion between the infrared camera and the drone's body, so the calculated position of the infrared camera is the drone's position.

[0058] Step S2 specifically includes the following steps:

[0059] S2.1. The onboard device processor is connected to the UAV flight control system to receive instructions and information from the UAV flight control system;

[0060] Specifically, the onboard device processor receives commands and information from the drone flight control system, including: the drone's attitude, such as pitch angle θ, roll angle φ, yaw angle ψ, position P and speed V, and the pressure altitude h at the drone's location. b and the validity of satellite navigation status.

[0061] S2.2. The airborne device processor is connected to the infrared camera to collect infrared camera images in real time.

[0062] In step S3, it is determined whether the satellite navigation signal is valid. If the satellite navigation signal is valid, satellite navigation is continued. If the satellite navigation signal is invalid, the infrared camera image is used for navigation and positioning. The method for determining whether the satellite navigation signal is valid is to determine the validity of the satellite navigation status information sent by the UAV flight control system.

[0063] Step S4 includes: the onboard device processor acquires the infrared camera image in real time through the image acquisition card and performs orthorectification on the infrared camera image according to the posture of the UAV.

[0064] Through orthorectification, image distortion caused by factors such as terrain undulation, sensor tilt and atmospheric refraction can be eliminated, thereby generating high-precision maps and geographic information system data.

[0065] like Figure 4 As shown in the figure, specifically, the orthorectification of the infrared camera image based on the drone's posture includes:

[0066] S4.1. Select the four corner points and the center point of the infrared camera image as control points, P i (x i ,y i ), i = 1, 2…5, get the pixel coordinates of the control points, which are: P1(0, 0), P2(w, 0), P3(0, h), P4(w, h) and P5(w / 2, h / 2), where w is the pixel width of the image and h is the pixel height of the image;

[0067] S4.2. Calculate the attitude matrix R based on the attitude of the drone. The formula is as follows:

[0068]

[0069] R is the unit orthogonal matrix, R -1 =R T , assuming

[0070]

[0071] S4.3. Calculate the relative geographic coordinates between the control point and the UAV using the digital elevation model based on the UAV's attitude matrix R;

[0072] Digital elevation model (DEM) is a three-dimensional data used to describe the terrain. The average height of the ground below the drone is h gi , calculate the relative geographic coordinates P of the control point di ,i=1,…5.

[0073]

[0074] S4.4. Obtain pixel coordinates P using the least squares method i , i=1,2…5 and relative geographic coordinates P di , the affine matrix M with i=1,…5 A , through the affine matrix M A Get the orthorectified image of the infrared camera image.

[0075] like Figure 5 and Figure 6 As shown, in step S5, the Log-Gabor filter is used to calculate the structural feature maps of the orthorectified infrared camera image and the satellite map respectively, and feature matching is performed between the orthorectified infrared camera image and the satellite map.

[0076] Specifically, M(x,y) is the image to be extracted structural features

[0077]

[0078] E so (x,y) and O so (x, y) are the even and odd components obtained by convolving the 2D-LogGabor filter with the image M(x, y) in direction o and scale s, respectively. is the odd-symmetric filter of Log-Gabor; is an even-symmetric filter of Log-Gabor.

[0079]

[0080] Then, find the maximum value A k (x,y)=max{A o(x,y)} and its direction index k, the maximum index mapping value I can be obtained MIM The maximum value k at the maximum index of (x,y).

[0081] I MIM (x,y)=k where A k (x,y)=max{A o (x,y)} (7)

[0082] In step S6, the relative geographic coordinates of the orthorectified infrared camera image are converted into actual longitude and latitude positions according to the coordinates of the satellite map of the best matching position, and then the actual longitude and latitude coordinates of the infrared camera are calculated.

[0083] Specifically, the pixel coordinates of the control points in the infrared camera image and the actual latitude and longitude of the control points are used to obtain the camera's latitude and longitude using the PNP method. These are the latitude and longitude of the infrared camera, but since the drone and infrared camera are strapdown and there is no relative motion between them, the infrared camera's location is the same as the drone's.

[0084] According to a second aspect of the technical solution of the present invention, there is provided a device for drone scene matching navigation and positioning based on infrared images, the device being used to implement the drone scene matching navigation and positioning method based on infrared images described in the above solution, the device comprising:

[0085] Strapdown module, used to strapdown the drone and infrared camera;

[0086] A pre-loaded module is used to connect the airborne device processor to the image acquisition card and the UAV flight control system, and the airborne device processor is pre-loaded with a satellite map of the area to be flown;

[0087] The judgment module is used to judge whether the satellite navigation signal is valid during the UAV cruising phase;

[0088] A cache module is used to collect infrared camera images in real time through the image acquisition card if the satellite navigation signal fails. The onboard device processor selects a matching area within a threshold range based on the UAV's previous positioning result and caches the satellite image of the area from the pre-loaded satellite map into the onboard device processor;

[0089] Feature extraction and matching module, used to extract structural features from infrared camera images and cached satellite maps, and perform matching;

[0090] The calculation module is used to calculate the actual latitude and longitude coordinates of the control point in the infrared camera image based on the best matching position, and then calculate the latitude and longitude of the drone at the current moment.

[0091] According to a third aspect of the technical solution of the present invention, there is provided an electronic device, comprising:

[0092] a memory storing executable instructions;

[0093] The processor runs the executable instructions in the memory to implement the method described.

[0094] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0095] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0096] Through the description of the above embodiments, those skilled in the art can clearly understand that the above implementation method can be implemented by means of software plus the necessary general hardware platform, or of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0097] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. A method for UAV scene matching navigation and positioning based on infrared images, characterized in that: include: S1. Strapdown the drone and the infrared camera; S2. Connecting an onboard device processor to the infrared camera and the UAV flight control system, wherein the onboard device processor is pre-loaded with a satellite map of the area to be flown; S3. During the UAV cruising phase, determining whether the satellite navigation signal received by the UAV is valid; S4. If the satellite navigation signal fails, the onboard device processor collects infrared camera images in real time through the infrared camera, selects a to-be-matched area within a threshold range based on the UAV's last positioning result, and caches the satellite image of the area from a pre-loaded satellite map into the onboard device processor; S5. Extracting structural features from the infrared camera image and the cached satellite map, and performing matching; S6. Calculate the actual longitude and latitude coordinates of the infrared camera image based on the best matching position, thereby obtaining the longitude and latitude coordinates of the drone at the current moment.

2. The method for UAV scene matching navigation and positioning based on infrared images according to claim 1 is characterized in that: In step S1, the optical axis of the infrared camera is set vertically and downwardly to the body of the drone.

3. The method for UAV scene matching navigation and positioning based on infrared images according to claim 1 is characterized in that: Step S2 includes: S2.

1. The onboard device processor is connected to the UAV flight control system to receive instructions and information from the UAV flight control system; S2.

2. The airborne device processor is connected to the infrared camera for collecting the infrared camera image in real time.

4. The method for UAV scene matching navigation and positioning based on infrared images according to claim 3 is characterized in that: In step S2.1, the onboard device processor receives instructions and information from the UAV flight control system, including: the attitude, position and speed of the UAV, the barometric altitude of the UAV's location, and the validity of the satellite navigation status.

5. The method for UAV scene matching navigation and positioning based on infrared images according to claim 1, characterized in that: Step S4 includes: the onboard device processor acquires the infrared camera image in real time through the image acquisition card and performs orthorectification on the infrared camera image according to the posture of the UAV.

6. The method for UAV scene matching navigation and positioning based on infrared images according to claim 5 is characterized in that: In step S4, orthorectifying the infrared camera image based on the posture of the drone includes: S4.

1. Select four corner points and a center point of the infrared camera image as control points, and obtain pixel coordinates of the control points; S4.

2. Calculate the attitude matrix R based on the attitude of the UAV; S4.

3. Calculate the relative geographic coordinates between the control point and the drone using a digital elevation model based on the drone's attitude matrix R; S4.4, using the least squares method to obtain the affine matrix M of the pixel coordinates and the relative geographic coordinates A , through the affine matrix M A An orthorectified image of the infrared camera image is obtained.

7. The method for UAV scene matching navigation and positioning based on infrared images according to claim 6 is characterized in that: In step S5, a Log-Gabor filter is used to calculate structural feature maps of the orthorectified infrared camera image and the satellite map, and feature matching is performed between the orthorectified infrared camera image and the satellite map.

8. The method for UAV scene matching navigation and positioning based on infrared images according to claim 6 is characterized in that: In step S6, the relative geographic coordinates of the orthorectified infrared camera image are converted into the actual longitude and latitude according to the coordinates of the satellite map of the best matching position, and then the actual longitude and latitude coordinates of the infrared camera are calculated as the longitude and latitude of the drone at the current moment.

9. A drone scene matching navigation and positioning device based on infrared images, characterized in that: The device is used to implement the infrared image-based unmanned aerial vehicle scene matching navigation and positioning method according to any one of claims 1 to 8, and the device includes: Strapdown module, used to strapdown the drone and infrared camera; A pre-loading module is used to connect the airborne device processor to the image acquisition card and the UAV flight control system, wherein the airborne device processor is pre-loaded with a satellite map of the area to be flown; The judgment module is used to judge whether the satellite navigation signal is valid during the UAV cruising phase; a cache module configured to, if the satellite navigation signal fails, cause the onboard device processor to acquire infrared camera images in real time through the image acquisition card, select a to-be-matched area within a threshold range based on the UAV's last positioning result, and cache the satellite image of the area from a pre-loaded satellite map into the onboard device processor; A feature extraction and matching module is used to extract structural features from the infrared camera image and the cached satellite map, and perform matching; The calculation module is used to calculate the actual longitude and latitude coordinates of the control point in the infrared camera image according to the best matching position, and then calculate the longitude and latitude of the drone at the current moment.

10. An electronic device, characterized in that: The electronic device comprises: a memory storing executable instructions; A processor, wherein the processor runs the executable instructions in the memory to implement the method according to any one of claims 1 to 8.

Images