A method and system for correcting and optimizing prior position coordinates of relative positioning points of a fuel dispenser

Abstract

The present application belongs to the field of visual navigation technology of aircraft in-flight refueling, and discloses a method for correcting and optimizing prior position coordinates of relative positioning points of a tanker, establishes an optical distortion mapping relationship of a refueling camera cabin of a receiver aircraft, obtains pixel original coordinates of the relative positioning points of the tanker as input of the optical distortion mapping relationship, calculates pixel corrected coordinates of the relative positioning points of the tanker after optical distortion correction, calculates a posture rotation matrix and a translation matrix of the refueling camera relative to the tanker under all frames of images, and further establishes a re-projection error cost function of the relative positioning points of the tanker, so as to obtain calibrated prior position coordinates of the relative positioning points of the tanker through optimization. The present application can correct the prior position coordinates of the relative positioning points of the tanker according to actually collected tanker images, further reduce the re-projection error of the prior position coordinates of the relative positioning points of the tanker when solving the posture, and improve the visual posture solving precision of the receiver aircraft relative to the tanker.

Description

Technical Field

[0001] This invention belongs to the field of visual navigation technology for aerial refueling of aircraft, and relates to a method for calculating the prior position coordinates of the relative positioning point of a refueling aircraft, specifically to a method and system for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling aircraft. Background Technology

[0002] In the current process of autonomous in-flight refueling, the receiving aircraft mostly uses visual navigation to achieve precise relative positioning of the tanker and the drogue. Visual navigation needs to detect the relative positioning point of the tanker and calculate the relative pose of the receiving aircraft and the tanker based on the prior position coordinates of the tanker's relative positioning point in the refueling reference coordinate system. Therefore, the accuracy of the prior position coordinates of the tanker's relative positioning point will directly affect the accuracy of visual navigation in the autonomous refueling process.

[0003] The prior position coordinates of the refueling aircraft relative to the positioning point are usually selected and measured on the digital model of the refueling aircraft. Considering the uncertainties in the actual aircraft manufacturing and elastic deformation, there will be a certain positioning error between the measured value of the digital model and the actual value, which will directly affect the accuracy of the attitude calculation of the receiver aircraft relative to the refueling aircraft. Summary of the Invention

[0004] The purpose of this invention is to address the aforementioned problems by providing a method for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling machine. This method utilizes images of the refueling machine captured by an airborne refueling camera during the refueling process to correct and optimize the prior position coordinates of the relative positioning point of the refueling machine in the refueling reference coordinate system.

[0005] The technical solution of this invention: A method for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling machine includes the following steps: S1, Establish the optical distortion mapping relationship between the cockpit of the onboard refueling camera of the receiving aircraft; S2, obtain the original pixel coordinates of the relative positioning point of the refueling aircraft as the input of the cockpit optical distortion mapping relationship, and calculate the corrected pixel coordinates of the relative positioning point of the refueling aircraft after optical distortion correction; S3. Using the pixel correction coordinates of the relative positioning point of the fuel dispenser and the prior position coordinates of the relative positioning point of the fuel dispenser selected on the digital model of the fuel dispenser, calculate the attitude rotation matrix and translation matrix of the receiving camera relative to the fuel dispenser in all frame images. S4. Establish the reprojection error cost function of the refueling machine relative positioning point. Using the measured prior position coordinates, attitude rotation matrix and translation matrix of the refueling machine relative positioning point on the digital model of the refueling machine obtained in S3 as the initial values ​​of the optimization variables, perform nonlinear optimization on the prior position coordinates of the refueling machine relative positioning point to obtain the calibrated prior position coordinates of the refueling machine relative positioning point.

[0006] Furthermore, in S1, the fixed checkerboard pattern is positioned to cover the field of view of the airborne refueling camera outside the cockpit. The airborne refueling camera then captures images of the fixed checkerboard pattern under both open and closed cockpit conditions, extracts the pixel coordinates of the checkerboard corner points, and obtains the cockpit optical distortion mapping relationship.

[0007] Furthermore, in S1, the fixed checkerboard pattern is positioned to cover the field of view of the airborne refueling camera outside the cockpit. Images of the fixed checkerboard pattern are acquired by the airborne refueling camera. The corner pixel coordinates of the checkerboard pattern images acquired with the canopy open are extracted. Corner pixel coordinates extracted from checkerboard pattern images with the canopy closed. A cockpit optical distortion mapping table was obtained using a cubic spline two-dimensional interpolation method. .

[0008] Furthermore, in S2, for all frames of tanker images acquired during the tanker's flight, a trained neural network model is used to extract the original pixel coordinates of the tanker's relative positioning point, which are then used as input for the cockpit optical distortion mapping relationship.

[0009] Furthermore, in S2, the relative positioning points of the refueling aircraft in all frames of images acquired during the receiver aircraft's flight are extracted using the YoloPose model. The original pixel coordinates of the j-th relative positioning point of the refueling aircraft in the i-th frame of the refueling aircraft image are... Calculate the pixel-corrected coordinates of the relative positioning point of the refueling machine after optical distortion correction. .

[0010] Furthermore, in S3, the pixel correction coordinates of the relative positioning point of the j-th refueling pump in the i-th frame image are: The prior position coordinates of the j-th refueling machine relative positioning point selected in the digital model are: By establishing the correspondence between the pixel coordinates of the relative positioning points of multiple refueling pumps and the prior position coordinates in the refueling reference coordinate system, the attitude rotation matrix of the receiving camera relative to the refueling pump is calculated using the PnP method. Translation matrix .

[0011] Furthermore, in S4, a reprojection error cost function for the refueling pump relative positioning points is established for a total of m frames of refueling pump images and n refueling pump relative positioning points. , where K is the camera intrinsic parameter matrix.

[0012] Furthermore, in S4, the attitude rotation matrix R and translation matrix t of the receiving camera relative to the refueling machine in each frame of the image calculated by EPnP, and the prior position coordinates p of the relative positioning point of the refueling machine selected on the digital model of the refueling machine are used as the initial values ​​of the optimization variables. The Gauss-Newton iteration method is used to iteratively calculate the optimized prior position coordinates of the relative positioning point of the refueling machine. As a calibration value.

[0013] A priori position coordinate correction and optimization system for the relative positioning point of a fuel dispenser includes software for correcting and optimizing the priori position coordinates of the relative positioning point of the fuel dispenser installed in the computer of the receiving fuel dispenser. The software for correcting and optimizing the priori position coordinates of the relative positioning point of the fuel dispenser runs the aforementioned method for correcting and optimizing the priori position coordinates of the relative positioning point of the fuel dispenser.

[0014] The beneficial effects of this invention are: This invention realizes the optimization of the prior position coordinate correction of the relative positioning point of the refueling machine. It can correct the prior position coordinate of the relative positioning point of the refueling machine based on the actual acquired image of the refueling machine, further reducing the reprojection error of the prior position coordinate of the relative positioning point of the refueling machine in pose calculation, and improving the accuracy of visual pose calculation of the receiving machine relative to the refueling machine. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the prior position coordinate correction and optimization process for the relative positioning point of the refueling machine according to the present invention. Detailed Implementation

[0017] This section describes embodiments of the present invention, used to explain and illustrate the technical solutions of the present invention. Unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0018] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating directions or positional relationships, are given in the accompanying drawings and are used only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or device referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include more than one of those features. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0019] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integrated connections; they can refer to mechanical connections or point connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0020] Example 1: A method for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling machine includes the following steps: S1, Establish the optical distortion mapping relationship between the cockpit of the onboard refueling camera of the receiving aircraft; S2, obtain the original pixel coordinates of the relative positioning point of the refueling aircraft as the input of the cockpit optical distortion mapping relationship, and calculate the corrected pixel coordinates of the relative positioning point of the refueling aircraft after optical distortion correction; S3. Using the pixel correction coordinates of the relative positioning point of the fuel dispenser and the prior position coordinates of the relative positioning point of the fuel dispenser selected on the digital model of the fuel dispenser, calculate the attitude rotation matrix and translation matrix of the receiving camera relative to the fuel dispenser in all frame images. S4. Establish the reprojection error cost function of the refueling machine relative positioning point. Using the measured prior position coordinates, attitude rotation matrix and translation matrix of the refueling machine relative positioning point on the digital model of the refueling machine obtained in S3 as the initial values ​​of the optimization variables, perform nonlinear optimization on the prior position coordinates of the refueling machine relative positioning point to obtain the calibrated prior position coordinates of the refueling machine relative positioning point.

[0021] In S1, the fixed checkerboard pattern is positioned to cover the field of view of the airborne refueling camera outside the cockpit. The airborne refueling camera acquires images of the fixed checkerboard pattern under both open and closed cockpit conditions. The pixel coordinates of the checkerboard corner points are extracted to obtain the cockpit optical distortion mapping relationship.

[0022] In S1, the fixed checkerboard pattern is positioned to cover the field of view of the airborne refueling camera outside the cockpit. Images of the fixed checkerboard pattern are acquired using the airborne refueling camera. The corner pixel coordinates of the checkerboard pattern images acquired with the canopy open are extracted. Corner pixel coordinates extracted from checkerboard pattern images with the canopy closed. A cockpit optical distortion mapping table was obtained using a cubic spline two-dimensional interpolation method. .

[0023] In S2, for all frames of tanker images acquired during the receiver aircraft's flight, a trained neural network model is used to extract the original pixel coordinates of the tanker's relative positioning points, which are then used as input for the cockpit optical distortion mapping relationship.

[0024] In S2, the relative positioning points of the refueling aircraft in all frames of images acquired during the receiver aircraft's flight are extracted using the YoloPose model. The original pixel coordinates of the j-th relative positioning point of the refueling aircraft in the i-th frame image are... Calculate the pixel-corrected coordinates of the relative positioning point of the refueling machine after optical distortion correction. .

[0025] In S3, the pixel correction coordinates of the relative positioning point of the j-th refueling machine in the i-th frame image are: The prior position coordinates of the j-th refueling machine relative positioning point selected in the digital model are: By establishing the correspondence between the pixel coordinates of the relative positioning points of multiple refueling pumps and the prior position coordinates in the refueling reference coordinate system, the attitude rotation matrix of the receiving camera relative to the refueling pump is calculated using the PnP method. Translation matrix .

[0026] In S4, a reprojection error cost function for the refueling machine relative positioning points is established for a total of m frames of refueling machine images and n refueling machine relative positioning points. , where K is the camera intrinsic parameter matrix.

[0027] In S4, the attitude rotation matrix R and translation matrix t of the receiving camera relative to the refueling machine in each frame image calculated by EPnP, and the prior position coordinates p of the relative positioning point of the refueling machine selected from the measured digital model of the refueling machine are used as the initial values ​​of the optimization variables. The Gauss-Newton iteration method is used to iteratively calculate the optimized prior position coordinates of the relative positioning point of the refueling machine. As a calibration value.

[0028] A priori position coordinate correction and optimization system for the relative positioning point of a fuel dispenser includes software for correcting and optimizing the priori position coordinates of the relative positioning point of the fuel dispenser installed in the computer of the receiving fuel dispenser. The software for correcting and optimizing the priori position coordinates of the relative positioning point of the fuel dispenser runs the aforementioned method for correcting and optimizing the priori position coordinates of the relative positioning point of the fuel dispenser.

[0029] Example 2: This invention is a method for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling machine, comprising the following steps: Step 1: Establish the optical distortion mapping relationship of the cockpit of the onboard refueling camera on the receiving aircraft: The fixed checkerboard pattern is positioned outside the cockpit to cover the field of view of the airborne refueling camera. Images of the fixed checkerboard pattern are acquired using the airborne refueling camera. The corner pixel coordinates of the checkerboard pattern images acquired with the canopy open are extracted. Corner pixel coordinates extracted from checkerboard pattern images with the canopy closed. A cockpit optical distortion mapping table was obtained using a cubic spline two-dimensional interpolation method. ; Step 2: Obtain the pixel-corrected coordinates of the relative positioning point of the fuel dispenser: The relative positioning points of the refueling aircraft in multiple frames of images acquired during the receiver aircraft's flight were extracted using the YoloPose model. The original pixel coordinates of the j-th relative positioning point of the refueling aircraft in the i-th frame of the refueling aircraft image are: Calculate the pixel-corrected coordinates of the relative positioning point of the refueling machine after optical distortion correction. ; Step 3: Calculate the pose of the receiving camera relative to the refueling machine: Calculate the attitude rotation and translation matrices of the receiving camera relative to the refueling truck in each frame image. The pixel correction coordinates of the j-th refueling truck relative to the positioning point in the i-th frame image are: The prior position coordinates of the j-th refueling machine relative positioning point selected in the digital model are: By establishing the correspondence between the pixel coordinates of the relative positioning points of multiple refueling pumps and the prior position coordinates in the refueling reference coordinate system, the attitude rotation matrix of the receiving camera relative to the refueling pump is calculated using the EPnP method. Translation matrix ; Step 4: Correct the prior position coordinates of the fuel dispenser's relative positioning point in the fuel dispenser reference coordinate system: For a total of m frames of fuel dispenser images and n relative positioning points of the fuel dispensers, establish a cost function for the reprojection error of the relative positioning points of the fuel dispensers. , where K is the camera intrinsic parameter matrix; The attitude rotation matrix R and translation matrix t of the receiving camera relative to the refueling truck in each frame of the image calculated by EPnP, and the prior position coordinates p of the relative positioning point of the refueling truck selected from the measured digital model of the refueling truck are used as the initial values ​​of the optimization variables. The Gauss-Newton iteration method is used to iteratively calculate the optimized prior position coordinates of the relative positioning point of the refueling truck. As calibration values. The attitude rotation matrix R, translation matrix t, etc. in this step are a combination of all frames.

[0030] The above description is merely a specific embodiment of the present invention, providing a detailed description of the invention. Parts not covered herein are conventional techniques. However, the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling machine, characterized in that, Includes the following steps: S1, Establish the optical distortion mapping relationship between the cockpit of the onboard refueling camera of the receiving aircraft; S2, obtain the original pixel coordinates of the relative positioning point of the refueling aircraft as the input of the cockpit optical distortion mapping relationship, and calculate the corrected pixel coordinates of the relative positioning point of the refueling aircraft after optical distortion correction; S3. Using the pixel correction coordinates of the relative positioning point of the fuel dispenser and the prior position coordinates of the relative positioning point of the fuel dispenser selected on the digital model of the fuel dispenser, calculate the attitude rotation matrix and translation matrix of the receiving camera relative to the fuel dispenser in all frame images. S4. Establish the reprojection error cost function of the refueling machine relative positioning point. Using the measured prior position coordinates, attitude rotation matrix and translation matrix of the refueling machine relative positioning point on the digital model of the refueling machine obtained in S3 as the initial values ​​of the optimization variables, perform nonlinear optimization on the prior position coordinates of the refueling machine relative positioning point to obtain the calibrated prior position coordinates of the refueling machine relative positioning point.

2. The method for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling machine according to claim 1, characterized in that, In S1, the fixed checkerboard pattern is positioned to cover the field of view of the airborne refueling camera outside the cockpit. The airborne refueling camera acquires images of the fixed checkerboard pattern under both open and closed cockpit conditions. The pixel coordinates of the checkerboard corner points are extracted to obtain the cockpit optical distortion mapping relationship.

3. The method for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling machine according to claim 2, characterized in that, In S1, the fixed checkerboard pattern is positioned to cover the field of view of the airborne refueling camera outside the cockpit. Images of the fixed checkerboard pattern are acquired using the airborne refueling camera. The corner pixel coordinates of the checkerboard pattern images acquired with the canopy open are extracted. Corner pixel coordinates extracted from checkerboard pattern images with the canopy closed. A cockpit optical distortion mapping table was obtained using a cubic spline two-dimensional interpolation method. .

4. The method for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling machine according to claim 1, characterized in that, In S2, for all frames of tanker images acquired during the receiver aircraft's flight, a trained neural network model is used to extract the original pixel coordinates of the tanker's relative positioning points, which are then used as input for the cockpit optical distortion mapping relationship.

5. The method for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling machine according to claim 4, characterized in that, In S2, the relative positioning points of the refueling aircraft in all frames of images acquired during the receiver aircraft's flight are extracted using the YoloPose model. The original pixel coordinates of the j-th relative positioning point of the refueling aircraft in the i-th frame image are... Calculate the pixel-corrected coordinates of the relative positioning point of the refueling machine after optical distortion correction. .

6. The method for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling machine according to claim 5, characterized in that, In S3, the pixel correction coordinates of the relative positioning point of the j-th refueling machine in the i-th frame image are: The prior position coordinates of the j-th refueling machine relative positioning point selected in the digital model are: By establishing the correspondence between the pixel coordinates of the relative positioning points of multiple refueling pumps and the prior position coordinates in the refueling reference coordinate system, the attitude rotation matrix of the receiving camera relative to the refueling pump is calculated using the EPnP method. Translation matrix .

7. The method for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling machine according to claim 6, characterized in that, In S4, a reprojection error cost function for the refueling machine relative positioning points is established for a total of m frames of refueling machine images and n refueling machine relative positioning points. , where K is the camera intrinsic parameter matrix.

8. The method for correcting and optimizing the prior position coordinates of the relative positioning point of a refueling machine according to claim 1, characterized in that, In S4, the attitude rotation matrix R and translation matrix t of the receiving camera relative to the refueling aircraft in all frame images calculated by EPnP, and the prior position coordinates p of the relative positioning point of the refueling aircraft selected from the measured digital model of the refueling aircraft are used as the initial values ​​of the optimization variables. The Gauss-Newton iteration method is used to iteratively calculate the optimized prior position coordinates of the relative positioning point of the refueling aircraft. As a calibration value.

9. A priori position coordinate correction and optimization system for the relative positioning point of a refueling machine, comprising software for correcting and optimizing the priori position coordinates of the relative positioning point of the refueling machine installed in the computer of the receiving machine, wherein the software for correcting and optimizing the priori position coordinates of the relative positioning point of the refueling machine runs the priori position coordinate correction and optimization method for the relative positioning point of the refueling machine as described in any one of claims 1-8.

Images