An unmanned aerial vehicle cruise vision positioning method based on photoelectric load

Abstract

This invention provides a visual positioning method for UAV cruise based on an optoelectronic payload. The method includes: using the optoelectronic payload to perform downward-looking imaging of the ground to acquire an original image; correcting the original image according to the attitude of the optoelectronic payload and the flight altitude to acquire a corrected image; and performing feature extraction and matching on a reference image and the corrected image respectively to achieve visual positioning during the UAV cruise phase. Specifically, correcting the original image according to the attitude of the optoelectronic payload and the flight altitude includes: performing orthorectification on the original image according to the attitude of the optoelectronic payload to acquire an orthorectified image; and performing scale correction on the orthorectified image according to the flight altitude and the spatial resolution of the reference image to acquire a corrected image. This invention solves the technical problems of complex structure and high cost associated with using dedicated imaging devices for visual positioning during the UAV cruise phase in existing technologies.

Description

Technical Field

[0001] This invention belongs to the field of visual navigation technology, and particularly relates to a visual positioning method for UAV cruise based on photoelectric payload. Background Technology

[0002] Computer vision, also known as machine vision, is a discipline that studies how to use computers to simulate human visual functions, extract valuable information from images of objective things, process and recognize it, and ultimately use it for practical detection, testing, navigation or control. It involves fields such as image processing, computer science, physiological psychology and photogrammetry.

[0003] Visual navigation uses visual images of the objective world acquired by image sensors as its navigation information source. It simulates human visual function, recognizing and understanding images to obtain navigation information for the carrier. It features strong autonomy, rich information, and a high level of intelligence, enabling it to meet the demands of high-precision relative navigation. As an emerging navigation system under development, visual navigation systems are gaining increasing attention due to their intelligent and autonomous characteristics.

[0004] Image matching is an important research hotspot in the field of computer vision. It is widely used in aerospace, satellite mapping, image retrieval, 3D reconstruction, target tracking and many other fields. In recent years, it has been widely used in visual navigation during the cruise phase of UAVs.

[0005] To achieve visual positioning during the cruise phase of a drone, a dedicated imaging device is generally required. However, the following problems exist: dedicated imaging devices for visual navigation increase the overall weight of the drone, and the accompanying optical windows increase the structural complexity of the drone, affecting its stealth performance and increasing costs; some drones with weight constraints cannot be equipped with dedicated imaging devices. Summary of the Invention

[0006] The present invention aims to solve at least one of the technical problems existing in the prior art.

[0007] This invention provides a visual positioning method for the cruise phase of an unmanned aerial vehicle (UAV) based on an optoelectronic payload. The method includes:

[0008] Using an optoelectronic payload to perform downward-looking imaging of the ground and acquire the original image;

[0009] The original image is corrected based on the attitude and flight altitude of the photoelectric payload to obtain the corrected image;

[0010] Feature extraction and matching are performed on the baseline image and the corrected image respectively to achieve visual positioning of the UAV during the cruise phase;

[0011] in,

[0012] Correcting the original image based on the attitude and flight altitude of the photoelectric payload specifically includes:

[0013] The original image is orthorectified according to the photoelectric load attitude to obtain an orthorectified image;

[0014] The orthophoto image is scaled based on the flight altitude and the spatial resolution of the reference image to obtain the corrected image.

[0015] Furthermore, orthorectification is performed on the original image I1 according to the following formula to obtain the orthorectified image I2:

[0016]

[0017] in, Let P be the image coordinates in the orthophoto image I2, K be the intrinsic parameter matrix of the photoelectric payload imaging device, C be the orthophoto rotation matrix, and P be the image coordinates in the orthophoto image I2. i Let P be the image coordinates of the spatial point P in the original image I1.

[0018] Furthermore, the orthophoto image I2 is scaled according to the following formula to obtain the corrected image I3:

[0019]

[0020] Where N is the scaling factor of the orthophoto I2, and k is the spatial resolution of the orthophoto I2. d represents the spatial resolution of the reference image I0, f represents the pixel size of the photoelectric payload imaging device, H represents the focal length of the photoelectric payload imaging device, and H represents the flight phase altitude when the original image is imaged.

[0021] This invention provides a visual positioning method for UAVs during the cruise phase based on an optoelectronic payload. This method utilizes the optoelectronic payload to perform downward-looking imaging of the ground, and then corrects the acquired image by combining it with attitude sensor information to obtain an image with the same effect as a dedicated visual positioning imaging device. This image is then used for visual positioning during the UAV's cruise phase. This invention achieves the combined use of the UAV's observation payload and navigation device, reducing the overall structural complexity and cost. Compared with existing technologies, this invention solves the technical problems of structural complexity and high cost associated with using dedicated imaging devices for visual positioning during the UAV's cruise phase. Attached Figure Description

[0022] The accompanying drawings, which form part of this specification, are provided to further illustrate embodiments of the invention and, together with the textual description, explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0023] Figure 1 A flowchart illustrating a visual positioning method for the cruise phase of an unmanned aerial vehicle (UAV) based on an optoelectronic payload, according to a specific embodiment of the present invention, is shown. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0026] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0027] like Figure 1 As shown in the figure, a visual positioning method for the cruise phase of an unmanned aerial vehicle (UAV) based on an optoelectronic payload is provided according to a specific embodiment of the present invention. The method includes:

[0028] Using an optoelectronic payload to perform downward-looking imaging of the ground and acquire the original image;

[0029] The original image is corrected based on the attitude and flight altitude of the photoelectric payload to obtain the corrected image;

[0030] Feature extraction and matching are performed on the baseline image and the corrected image respectively to achieve visual positioning of the UAV during the cruise phase;

[0031] Specifically, the correction of the original image based on the attitude and flight altitude of the photoelectric payload includes:

[0032] The original image is orthorectified according to the photoelectric load attitude to obtain an orthorectified image;

[0033] The orthophoto image is scaled based on the flight altitude and the spatial resolution of the reference image to obtain the corrected image.

[0034] This configuration provides a visual positioning method for UAVs during the cruise phase based on an optoelectronic payload. This method utilizes the optoelectronic payload to perform downward-looking imaging of the ground, and then corrects the acquired image by combining it with attitude sensor information to obtain an image with the same effect as a dedicated visual positioning imaging device. This image is then used for visual positioning during the UAV's cruise phase. The technical solution of this invention achieves the combined use of the UAV's observation payload and navigation device, reducing the overall structural complexity and cost.

[0035] Furthermore, in this invention, by adjusting the servo mechanism of the photoelectric load to make it look vertically downward, it images the ground and forms the original image I1.

[0036] The image coordinates of spatial point P in the original image I1 obtained by downward imaging with a photoelectric load are P. i The original image I1 is orthorectified according to the following formula to obtain the orthorectified image I2:

[0037]

[0038] in, Let I be the image coordinates in the orthophoto image I2, K be the intrinsic parameter matrix of the photoelectric payload imaging device, and C be the orthophoto rotation matrix.

[0039] Using the above formula, the image coordinates P in the original image I1 are... i Transformed into image coordinates in orthophoto I2 Obtain the orthophoto image I2.

[0040] Furthermore, in this invention, the pixel resolution of the orthophoto image I2 and the reference image I0 are unified and scaled to a certain spatial resolution.

[0041] The spatial resolution of the orthophoto I2 is

[0042]

[0043] Where k is the spatial resolution in the orthophoto image I2, d is the pixel size of the photoelectric payload imaging device, f is the focal length of the photoelectric payload imaging device, and H is the flight phase altitude when the original image is imaged.

[0044] Assume the spatial resolution of the reference image I0 is (Unit: meters / pixel), the orthophoto image I2 needs to be scaled by the following factors to match the pixel resolution of the reference image I0. The scaled image will be used as the corrected image I3:

[0045]

[0046] Where N is the scaling factor of the orthophoto I2.

[0047] In this invention, after acquiring the corrected image, a mature visual matching navigation method can be used to extract and match features of the reference image I0 and the corrected image I3 respectively, thereby achieving visual positioning.

[0048] To address the issue of relying on dedicated imaging devices for visual positioning during the cruise phase of unmanned aerial vehicles (UAVs), this invention proposes a visual positioning method based on an optoelectronic payload. An optoelectronic payload is a standard configuration for reconnaissance UAVs, typically including an imaging device, a servo mechanism, and an attitude sensor. By locking the attitude of the imaging device within the optoelectronic payload using the servo mechanism, ground imaging can be achieved. Based on this, the attitude sensor performs attitude correction on the image, achieving the same effect as a dedicated imaging device for visual positioning. Therefore, this method can be used for visual positioning during the UAV's cruise phase. This invention fully utilizes the structure and electrical systems of existing optoelectronic payloads on UAVs, enabling in-depth utilization of sensor information.

[0049] To gain a further understanding of the present invention, the visual positioning method for the cruise phase of an unmanned aerial vehicle based on photoelectric payload of the present invention will be described in detail below with reference to specific embodiments.

[0050] Example

[0051] By utilizing the structure and electrical system of existing optoelectronic payloads on UAVs, a visual positioning method for the cruise phase of UAVs based on optoelectronic payloads is provided, which specifically includes the following steps.

[0052] Step 1: Downward-looking imaging of the ground by the photoelectric payload:

[0053] Adjust the servo mechanism of the photoelectric load to make it look vertically downward. After the imaging device is locked in attitude, it images the ground. The resulting original image is denoted as I1.

[0054] Step 2: Perform orthorectification on the image based on the photoelectric load attitude:

[0055] according to Perform orthorectification on the original image I1 to obtain the orthorectified image I2.

[0056] Step 3: Perform dimensional corrections based on flight altitude.

[0057] according to Obtain the scaling factor, scale the orthophoto image I2 according to the scaling factor, unify the pixel resolution with the reference image I0, and obtain the corrected image I3.

[0058] Step 4, Feature matching and localization:

[0059] Feature extraction and matching are performed on the baseline image and the corrected image respectively to achieve visual positioning of the UAV during the cruise phase.

[0060] In summary, this invention provides a visual positioning method for UAVs during the cruise phase based on an optoelectronic payload. This method utilizes the optoelectronic payload to perform downward-looking imaging of the ground, and then corrects the acquired image by combining it with attitude sensor information to obtain an image with the same effect as a dedicated visual positioning imaging device. This image is then used for visual positioning of the UAV during the cruise phase. The technical solution of this invention achieves the combined use of the UAV's observation payload and navigation device, reducing the overall structural complexity and cost of the aircraft.

[0061] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A visual positioning method for the cruise phase of an unmanned aerial vehicle (UAV) based on an optoelectronic payload, characterized in that, The UAV cruise phase visual positioning method based on photoelectric payload includes: Using an optoelectronic payload to perform downward-looking imaging of the ground and acquire the original image; The original image is corrected based on the attitude and flight altitude of the photoelectric payload to obtain the corrected image; Feature extraction and matching are performed on the baseline image and the corrected image respectively to achieve visual positioning of the UAV during the cruise phase; in, Correcting the original image based on the attitude and flight altitude of the photoelectric payload specifically includes: The original image is orthorectified according to the photoelectric load attitude to obtain an orthorectified image; The orthophoto image is scaled based on the flight altitude and the spatial resolution of the reference image to obtain the corrected image.

2. The visual positioning method for the cruise phase of an unmanned aerial vehicle based on photoelectric payload according to claim 1, characterized in that, Orthorectification is performed on the original image I1 according to the following formula to obtain the orthorectified image I2: in, Let P be the image coordinates in the orthophoto image I2, K be the intrinsic parameter matrix of the photoelectric payload imaging device, C be the orthophoto rotation matrix, and P be the image coordinates in the orthophoto image I2. i Let P be the image coordinates of the spatial point P in the original image I1.

3. The visual positioning method for the cruise phase of an unmanned aerial vehicle based on photoelectric payload according to claim 1, characterized in that, The orthophoto image I2 is scaled according to the following formula to obtain the corrected image I3: Where N is the scaling factor of the orthophoto I2, and k is the spatial resolution of the orthophoto I2. d represents the spatial resolution of the reference image I0, f represents the pixel size of the photoelectric payload imaging device, H represents the focal length of the photoelectric payload imaging device, and H represents the flight phase altitude when the original image is imaged.

Images