Method for calculating the six-degree-of-freedom position and attitude of an aircraft using a single aerial observation image

Abstract

The present invention discloses a method for calculating the six-degree-of-freedom posture of an aircraft through a single aerial observation image, which belongs to the field of aircraft posture estimation and includes the steps of extracting and matching feature points between a single aerial observation image of an aircraft and a satellite map, and then calculating the six-degree-of-freedom posture of the aircraft according to the matching results, thereby calculating the position and attitude of the aircraft. The present invention can not only obtain more accurate position data, but also obtain the precise rotation angle of the aircraft.

Description

Technical Field

[0001] The present invention relates to the field of aircraft posture estimation, and more specifically, to a method for calculating the six-degree-of-freedom posture of an aircraft through a single aerial observation image. Background Art

[0002] Aircraft visual positioning is a technology that uses image data from an onboard camera to locate the aircraft. This technology is extremely important for the safe operation of the aircraft when GPS fails. Most aircraft visual positioning systems can only estimate the relative motion of the aircraft, which will continue to generate cumulative errors during long-term flight, resulting in increasing positioning errors. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a method for calculating the six-degree-of-freedom position and attitude of an aircraft through a single aerial observation image, which can not only obtain more accurate position data, but also obtain the precise rotation angle of the aircraft, etc.

[0004] The object of the present invention is achieved through the following solutions:

[0005] A method for calculating the six-degree-of-freedom position and attitude of an aircraft through a single aerial observation image comprises the steps of extracting and matching feature points between the single aerial observation image of the aircraft and a satellite map, and then calculating the six-degree-of-freedom position and attitude of the aircraft according to the matching results, thereby calculating the position and attitude of the aircraft.

[0006] Further, the method comprises the following sub-steps:

[0007] S1, calculating a first position of the aircraft according to a visual odometer or an inertial measurement unit, extracting a satellite map of the first position according to a GPS of the first position, and then respectively extracting and matching feature points of the satellite map and the observed image;

[0008] S2, after matching the feature points between the aircraft observation image and the satellite map based on step S1, according to the pixel coordinates of the feature points on the satellite map Calculate the longitude and latitude coordinates of each feature point on the satellite map according to the following formula

[0009]

[0010]

[0011] Among them, w and h are the width and height of the local satellite map image, respectively, and C lon and C lat is the longitude and latitude of the pixel at the center of the local satellite map; ρ lon and ρlat are the longitude and latitude spans of each pixel on the satellite map, respectively;

[0012] S3, transform the latitude and longitude coordinates of the feature point into the geocentric coordinate system, and then transform the geocentric coordinate system into the northeast celestial coordinate system with the aircraft take-off point as the station. The northeast celestial coordinate system coordinates of the feature point are expressed as:

[0013]

[0014] Where N is the number of matching feature points;

[0015] S4, taking the rotation matrix R and the three-dimensional translation t as the unknown quantities to be solved, the coordinates of the feature points on the satellite map in the camera coordinate system are obtained, which can be expressed as:

[0016]

[0017] Matching feature points on the camera observation image Transformed to the camera coordinate system through the camera intrinsic parameter matrix K, it is expressed as:

[0018]

[0019] The line connecting the matching feature points on the observed image and the camera center forms a vector. and The direction is the same, so as to construct a linear equation, and use the method of solving the linear equation group to solve the position and attitude of the aircraft in the northeast celestial coordinate system.

[0020] Further, in step S4, the step of constructing a linear equation and solving a linear equation group to obtain the position and attitude of the aircraft in the northeast sky coordinate system includes the following sub-steps:

[0021] is a three-degree-of-freedom vector, and there are two orthogonal direction vectors. These two orthogonal direction vectors exist in In the null space of , it is recorded as:

[0022]

[0023] Among them, r and s are For two direction vectors that are orthogonal to each other, then:

[0024]

[0025] exist and In the same direction, there exists:

[0026]

[0027] In the above formula, there are 12 unknown quantities including R and t. Each pair of matching feature points provides two equality constraints. The camera coordinates can be solved by using at least 6 pairs of matching feature points. Then, the general method of solving linear equations is used to solve the position and attitude of the aircraft in the northeast celestial coordinate system.

[0028] Furthermore, in step S4, the sub-step is included: setting the rotation matrix R to:

[0029]

[0030] Further, in step S1, the feature points include SURF feature points.

[0031] Furthermore, the aircraft includes a drone.

[0032] The beneficial effects of the present invention are:

[0033] The present invention provides a method, which matches feature points between aircraft observation images and satellite maps, and then directly solves the six-degree-of-freedom absolute position of the aircraft in real-world coordinates. This method can not only obtain more accurate position data (longitude, latitude, altitude), but also obtain the precise rotation angle (heading angle, roll angle, pitch angle) of the aircraft. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 It is a schematic diagram of the method principle of the present invention;

[0036] In the figure, 1-airborne camera coordinate system, 2-airborne camera image plane, 3-depth normalized plane, 4-projection point of satellite map point on the image plane, 5-actual observation point of satellite map point on the image plane, 6-direction vector of satellite map point projection, 7-direction vector from camera origin to observation point, 8-first tangent vector of direction vector 7 from camera origin to observation point on depth normalized plane 3, 9-second tangent vector of direction vector 7 from camera origin to observation point on depth normalized plane 3, 10-ground. DETAILED DESCRIPTION

[0037] All features disclosed in all embodiments of this specification, or steps in all methods or processes disclosed implicitly, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any way. Figure 1 , the technical concept, working principle, efficacy and working process of the present invention are further described in detail.

[0038] The technical solution provided by the present invention only needs to use a single image observed by an aircraft (such as a drone) during aerial photography, and extract and match feature points between the aircraft observation image and the satellite map to establish a Figure 1 The matching relationship shown includes the onboard camera coordinate system 1, the onboard camera image plane 2, the depth normalization plane 3, the projection point of the satellite map point on the image plane 4, the actual observation point of the satellite map point on the image plane 5 and the direction vector of the satellite map point projection 6. Then, based on the matching results, the position and attitude of the aircraft can be calculated more accurately. In the specific implementation process, the six-degree-of-freedom posture of the aircraft in the northeast sky coordinate system with the take-off point as the station can be calculated, so that the position and attitude of the aircraft can be calculated more accurately.

[0039] In practical applications, when an aircraft flies to a new location, the approximate location of the aircraft (the first position of the present invention) is first inferred based on the visual odometer or IMU (inertial measurement unit), and the nearby satellite map is extracted based on the GPS of the approximate location. Then, the feature points of the satellite map and the observed image (such as SURF feature points) are extracted and matched respectively.

[0040] After obtaining the feature point matching between the aircraft observation image and the satellite map, the present invention firstly matches the pixel coordinates of the feature points on the satellite map. Calculate the latitude and longitude coordinates of each feature point on the satellite map The calculation method is:

[0041]

[0042]

[0043] Among them, w and h are the width and height of the local satellite map image, respectively, and C lon and C lat is the longitude and latitude of the pixel at the center of the local satellite map; ρ lon and ρ lat are the longitude and latitude spans of each pixel on the satellite map, respectively;

[0044] The latitude and longitude coordinates of the feature points can be transformed into the geocentric coordinate system, and then transformed from the geocentric coordinate system to the northeast celestial coordinate system with the aircraft take-off point as the station. The northeast celestial coordinate system coordinates of these feature points are expressed as:

[0045]

[0046] In the specific implementation process of the present invention, for example, solving the attitude of the aircraft in the northeast sky coordinate system includes a rotation matrix R and a three-dimensional translation t=(t x ,t y ,t z ). Taking R and t as the unknown quantities to be solved, the coordinates of the feature points on the satellite map in the camera coordinate system can be obtained, which can be expressed as:

[0047]

[0048] The matching feature points on the camera observation image can also be transformed into the camera coordinate system through the camera intrinsic parameter matrix K:

[0049]

[0050] The lines connecting these points and the camera center (i.e., the camera origin) form vectors, such as Figure 1 The direction vector from the camera origin to the observation point in 7. Theoretically, when the camera posture is correct, the vector and The direction is the same, that is Figure 1 The direction vector 6 of the map point projection in coincides with the direction vector 7 from the camera origin to the observation point, and the present invention constructs a linear equation based on this. is a three-degree-of-freedom vector, and there are two direction vectors orthogonal to it, such as Figure 1 The first tangent vector 8 of the direction vector 7 from the camera origin to the observation point on the depth normalized plane 3 and the second tangent vector 9 of the direction vector 7 from the camera origin to the observation point on the depth normalized plane 3, these two direction vectors exist in In the null space of , it is recorded as:

[0051]

[0052] where r and s are For two direction vectors that are orthogonal to each other, there exists:

[0053]

[0054] So, if you want and The direction is the same, then it needs to satisfy:

[0055]

[0056] Let the rotation matrix R be:

[0057]

[0058] Therefore, there are 12 unknown quantities in equation (8), including R and t. Each pair of matching feature points provides two equation constraints, so at least 6 pairs of matching feature points are needed to solve the camera coordinates. 6 pairs of matching feature points get 6 sets of equations (8), a total of 12 equations. The general method of solving linear equations can be used to solve the position and attitude of the aircraft in the northeast sky coordinate system.

[0059] Embodiment 1: A method for calculating the six-degree-of-freedom position and attitude of an aircraft using a single aerial observation image, comprising the steps of: extracting and matching feature points between a single aircraft observation image and a satellite map, and then calculating the six-degree-of-freedom position and attitude of the aircraft based on the matching results, thereby calculating the position and attitude of the aircraft.

[0060] On the basis of Example 1, the method comprises the following sub-steps:

[0061] S1, calculating a first position of the aircraft according to a visual odometer or an inertial measurement unit, extracting a satellite map of the first position according to a GPS of the first position, and then respectively extracting and matching feature points of the satellite map and the observed image;

[0062] S2, after matching the feature points between the aircraft observation image and the satellite map based on step S1, according to the pixel coordinates of the feature points on the satellite map Calculate the longitude and latitude coordinates of each feature point on the satellite map according to the following formula

[0063]

[0064]

[0065] Among them, w and h are the width and height of the local satellite map image, respectively, and C lon and C lat is the longitude and latitude of the pixel at the center of the local satellite map; ρ lon and ρ lat are the longitude and latitude spans of each pixel on the satellite map, respectively;

[0066] S3, transform the latitude and longitude coordinates of the feature point into the geocentric coordinate system, and then transform the geocentric coordinate system into the northeast celestial coordinate system with the aircraft take-off point as the station. The northeast celestial coordinate system coordinates of the feature point are expressed as:

[0067]

[0068] Where N is the number of matching feature points;

[0069] S4, taking the rotation matrix R and the three-dimensional translation t as the unknown quantities to be solved, the coordinates of the feature points on the satellite map in the camera coordinate system are obtained, which can be expressed as:

[0070]

[0071] Matching feature points on the camera observation image Transformed to the camera coordinate system through the camera intrinsic parameter matrix, expressed as:

[0072]

[0073] The line connecting the matching feature points on the observed image and the camera center forms a vector. and The direction is the same, so as to construct a linear equation, and use the method of solving the linear equation group to solve the position and attitude of the aircraft in the northeast celestial coordinate system.

[0074] Embodiment 3: Based on Embodiment 2, in step S4, a linear equation is constructed, and the position and attitude of the aircraft in the northeast sky coordinate system are solved by using a method for solving a linear equation system, including the following sub-steps:

[0075] is a three-degree-of-freedom vector, and there are two orthogonal direction vectors. These two orthogonal direction vectors exist in In the null space of , it is recorded as:

[0076]

[0077] Among them, r and s are For two direction vectors that are orthogonal to each other, then:

[0078]

[0079] exist and In the same direction, there exists:

[0080]

[0081] In the above formula, there are 12 unknown quantities including R and t. Each pair of matching feature points provides two equality constraints. The camera coordinates can be solved by using at least 6 pairs of matching feature points. Then, the general method of solving linear equations is used to solve the position and attitude of the aircraft in the northeast celestial coordinate system.

[0082] Embodiment 4: Based on Embodiment 3, in step S4, the following sub-steps are included: setting the rotation matrix R to:

[0083]

[0084] In actual application, in step S1, the feature points include SURF feature points.

[0085] In practical applications, the aircraft includes drones, but is not limited to drones.

[0086] If the functions of the present invention are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium, and all or part of the steps of the methods described in various embodiments of the present invention are executed in a computer device (which can be a personal computer, a server, or a network device, etc.) and corresponding software. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, or an optical disk, and the test or actual data exists in a read-only memory (Random Access Memory, RAM), a random access memory (Random Access Memory, RAM), etc. in the program implementation.

Claims

1. A method for calculating the six-degree-of-freedom position and posture of an aircraft using a single aerial observation image, characterized in that: The method comprises the following steps: extracting and matching feature points between a single aerial observation image of an aircraft and a satellite map, and then calculating the six-degree-of-freedom position and attitude of the aircraft according to the matching results, thereby calculating the position and attitude of the aircraft; and specifically comprising the following sub-steps: S1, calculating a first position of the aircraft according to a visual odometer or an inertial measurement unit, extracting a satellite map of the first position according to a GPS of the first position, and then respectively extracting and matching feature points of the satellite map and the observed image; S2, after matching the feature points between the aircraft observation image and the satellite map based on step S1, according to the pixel coordinates of the feature points on the satellite map Calculate the longitude and latitude coordinates of each feature point on the satellite map according to the following formula Among them, w and h are the width and height of the local satellite map image, respectively, and C lon and C lat is the longitude and latitude of the pixel at the center of the local satellite map; ρ lon and ρ lat are the longitude and latitude spans of each pixel on the satellite map, respectively; S3, transform the latitude and longitude coordinates of the feature point into the geocentric coordinate system, and then transform the geocentric coordinate system into the northeast celestial coordinate system with the aircraft take-off point as the station. The northeast celestial coordinate system coordinates of the feature point are expressed as: Where N is the number of matching feature points; S4, taking the rotation matrix R and the three-dimensional translation t as the unknown quantities to be solved, the coordinates of the feature points on the satellite map in the camera coordinate system are obtained, which can be expressed as: Matching feature points on the camera observation image Transformed to the camera coordinate system through the camera intrinsic parameter matrix K, it is expressed as: The line connecting the matching feature points on the observed image and the camera center forms a vector. and The direction is the same, so as to construct a linear equation, and use the method of solving the linear equation group to solve the position and attitude of the aircraft in the northeast celestial coordinate system.

2. The method for calculating the six-degree-of-freedom position and attitude of an aircraft through a single aerial observation image according to claim 1, characterized in that: In step S4, the step of constructing a linear equation and solving a linear equation group to obtain the position and attitude of the aircraft in the northeast sky coordinate system includes the following sub-steps: is a three-degree-of-freedom vector, and there are two orthogonal direction vectors. These two orthogonal direction vectors exist in In the null space of , it is recorded as: Among them, r and s are For two direction vectors that are orthogonal to each other, then: exist and In the same direction, there exists: In the above formula, there are 12 unknown quantities including R and t. Each pair of matching feature points provides two equality constraints. The camera coordinates can be solved by using at least 6 pairs of matching feature points. Then, the general method of solving linear equations is used to solve the position and attitude of the aircraft in the northeast celestial coordinate system.

3. The method for calculating the six-degree-of-freedom posture of an aircraft through a single aerial observation image according to claim 2, characterized in that: In step S4, the sub-step is included: setting the rotation matrix R to:

4. The method for calculating the six-degree-of-freedom posture of an aircraft through a single aerial observation image according to claim 1, characterized in that: In step S1, the feature points include SURF feature points.

5. The method for calculating the six-degree-of-freedom position and attitude of an aircraft through a single aerial observation image according to any one of claims 1 to 4, characterized in that: The aircraft includes a drone.

Images