A monocular UAV absolute vision matching and positioning method based on a geographic base map

Abstract

The present invention discloses a monocular UAV absolute vision matching and positioning scheme based on a geographic base map. The method includes: constructing a geographic base map of the UAV flight area and performing feature extraction to obtain a template image; acquiring an airborne camera photo and performing feature extraction to obtain a target image to be measured; performing motion constraint on the UAV based on an inertial measurement unit, matching the template image with the target image to be measured to obtain a matching area; constructing a visual inertial odometer based on the output of the airborne camera and the output of the inertial measurement unit and obtaining the local pose of the UAV; using an extended Kalman filter algorithm to perform state joint estimation on the matching area and the local pose of the UAV to obtain the real-time pose of the UAV. By using the present invention, precise positioning of the UAV in a complex environment can be achieved. As a monocular UAV absolute vision matching and positioning scheme based on a geographic base map, the present invention can be widely applied to the field of UAV vision positioning.

Description

Technical Field

[0001] The present invention relates to the field of UAV vision positioning, and particularly to a monocular UAV absolute vision matching and positioning scheme based on a geographic base map. Background Art

[0002] Unmanned aerial vehicles (UAVs) have the characteristics of low manufacturing cost, long endurance time, good concealment, strong vitality, fearlessness of casualties, simple take-off and landing, convenient operation, flexibility and mobility, etc. They are suitable for performing more onerous tasks in complex and dangerous environments. Therefore, they have extremely broad application prospects in both military and civilian fields. With the continuous increase in the application fields of UAVs, the performance requirements for UAVs are becoming more and more stringent, especially the positioning performance of UAVs. For UAVs to perform tasks in a changing environment, the current positioning and navigation schemes are difficult to meet the requirements. At present, UAV positioning and navigation mainly rely on the position information provided by the global positioning scheme GNSS. However, the GNSS signal is relatively weak and is easily interfered. In outdoor environments with severe occlusion and indoor environments, GNSS cannot work properly and cannot provide stable speed and position information, which will directly cause the UAV to be unable to fly normally. This has stimulated the development of new methods to supplement or replace satellite navigation in GNSS signal denied environments.

[0003] Regarding the UAV positioning problem in the case of satellite positioning scheme denial, the existing methods mainly use visual images to match with the onboard reference map to obtain the absolute position information of the UAV. However, there are differences in height, time, and perspective between the reference image and the information collected by the UAV in real time. The corresponding relationship of the features in the image will be destroyed due to this difference, resulting in the difficulty of the existing methods to complete the precise positioning of the UAV in the real complex environment. It is necessary to study a highly reliable matching method to complete the precise positioning of the UAV in the real complex environment. Summary of the Invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a monocular UAV absolute vision matching and positioning scheme based on a geographic base map, which can achieve precise positioning of the UAV in a complex environment.

[0005] A monocular UAV absolute vision matching and positioning scheme based on a geographic base map includes the following steps:

[0006] Construct a geographic base map of the UAV flight area and perform feature extraction to obtain a template image;

[0007] Obtain the photos taken by the onboard camera and perform feature extraction to obtain a target image to be measured;

[0008] Based on the inertial measurement unit, perform motion constraint on the UAV, and match the template image with the target image to be measured to obtain a matching area;

[0009] Construct a visual-inertial odometer based on the outputs of the airborne camera and the inertial measurement unit, and obtain the local pose of the UAV.

[0010] Use the extended Kalman filter algorithm to perform state joint estimation on the matching region and the local pose of the UAV to obtain the real-time pose of the UAV.

[0011] Further, the step of constructing the geographic base map of the UAV flight area and performing feature extraction to obtain the template image specifically includes:

[0012] Select a download source according to the longitude and latitude coordinates, image zoom level, and image style of the UAV flight area to obtain image tiles.

[0013] Unify and fuse the image tile coordinates to obtain a coordinate scheme.

[0014] Write the coordinate scheme into the world coordinates of the four corner points of the template to be processed to obtain the geographic base map.

[0015] Extract feature points from the geographic base map, determine the directions of the feature points, and construct feature point descriptions to obtain the template image.

[0016] Further, the step of performing motion constraints on the UAV based on the inertial measurement unit, and matching the template image with the image of the target to be measured to obtain the matching region specifically includes:

[0017] Based on the inertial measurement unit, use the error-state Kalman filter algorithm to estimate the motion state of the UAV.

[0018] Establish a constrained image of the target to be measured according to the motion state of the UAV.

[0019] Match the constrained image of the target to be measured with the template image to obtain the matching region.

[0020] Further, the step of using the error-state Kalman filter algorithm to estimate the motion state of the UAV based on the inertial measurement unit specifically includes:

[0021] Use the inertial measurement unit to perform integral update on the nominal state of the UAV.

[0022] Use the error-state Kalman filter algorithm to perform time update and measurement update on the error state of the UAV.

[0023] Combine the nominal state and the error state to obtain the motion state of the UAV.

[0024] Further, the step of matching the constrained image of the target to be measured with the template image to obtain the matching region specifically includes:

[0025] Using the Euclidean distance, calculate the Euclidean distance between any two points of the constrained target image to be measured and the template image;

[0026] According to the Euclidean distance and the data structure of the kd - tree, eliminate the error - matching points to obtain the set of homologous points;

[0027] Construct a matching region according to the set of homologous points.

[0028] Furthermore, the extended Kalman filter algorithm equation is expressed as follows:

[0029]

[0030] In the above formula, is the error covariance matrix, Q k is the covariance matrix of the process noise, T k,k-1 is the state transition matrix of the UAV from the key frame k - 1 to k, τ k,k-1 is the adjoint matrix of T k,k-1 is the initial pose estimate, is the error covariance matrix of the second - order approximation of the UAV at the key frame k, R k is the covariance matrix of the measurement noise, K k is the Kalman gain, ln(·) ∨ and exp(·^) are the operators of SE(3).

[0031] Furthermore, it also includes correcting the outputs of the on - board camera and the inertial measurement unit.

[0032] The beneficial effects of the method and system of the present invention are as follows: Based on the geographic base map and the on - board camera, by matching the on - board camera images in the UAV flight area with the geographic base map to obtain a matching region, the positioning and map measurement update of the UAV are realized. At the same time, the local pose of the UAV is estimated by the visual - inertial odometer, and the matching region and the local pose of the UAV are jointly estimated in state to obtain the real - time pose of the UAV, improving the positioning accuracy. Brief Description of the Drawings

[0033] Figure 1 is the step - flow chart of a monocular UAV absolute visual matching and positioning scheme based on a geographic base map of the present invention;

[0034] Figure 2 is the flow - schematic diagram of the geographic registration of the template image in the specific embodiment of the present invention;

[0035] Figure 3 is the schematic diagram of the image tile coordinate conversion in the specific embodiment of the present invention;

[0036] Figure 4 ​It is a schematic diagram of camera coordinate conversion in a specific embodiment of the present invention;

[0037] Figure 5 It is a schematic flow diagram of error state Kalman filtering in a specific embodiment of the present invention;

[0038] Figure 6 It is a schematic flow diagram of the process based on the geographical base map matching area in a specific embodiment of the present invention;

[0039] Figure 7 It is a schematic flow diagram of state joint estimation in a specific embodiment of the present invention;

[0040] Figure 8 It is a structural block diagram of a monocular UAV absolute vision matching and positioning system based on a geographical base map provided in a specific embodiment of the present invention. Specific Embodiments

[0041] The following further elaborates on the present invention in detail in conjunction with the accompanying drawings and specific embodiments. For the step numbers in the following embodiments, they are only set for the convenience of elaboration and explanation, and no limitation is imposed on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0042] Refer to Figure 1 , the present invention provides a monocular UAV absolute vision matching and positioning solution based on a geographical base map, and this solution includes the following steps:

[0043] S1. Refer to Figure 2 , construct a geographical base map of the UAV flight area and perform feature extraction to obtain a template image;

[0044] Among them, the UAV includes one or several of unmanned helicopters, ducted unmanned aircraft, and rotor UAVs; when using the same UAV vision matching and positioning solution and the same sensor combination, the positioning performance has consistency on different unmanned flight devices.

[0045] S1.1. According to the longitude and latitude coordinates, image zoom level, and image style of the UAV flight area, select a download source to obtain image tiles; there is no need to consume a large amount of manpower and material resources to pre - establish a template database, and it can be obtained and updated in real - time through open - source databases (Google Images and ArcGIS TM);

[0046] Specifically, the obtained image tiles are Google image tiles of the required zoom level obtained from Google Images, and the number of its image tiles will increase exponentially with the improvement of the zoom level and the expansion of the download area.

[0047] S1.2. Unify and fuse the image tile coordinates to obtain a coordinate scheme;

[0048] Specifically, since the downloaded image tiles are all in the body coordinates, it is necessary to unify the tile coordinates into the same coordinate system for the fusion of image tiles, and then through a series of appropriate coordinate conversions, select the desired coordinate scheme.

[0049] Among them, referring to Figure 3 , the specific calculation operations of the coordinate conversion are as follows:

[0050] 1) Conversion of longitude and latitude coordinates (long, lat) to tile coordinates (tileX, tileY):

[0051]

[0052] In the above formula, l is the zoom level of the tile.

[0053] 2) Conversion of longitude and latitude coordinates (long, lat) to pixel coordinates (pixelX, pixelY):

[0054]

[0055] In the above formula, l is the zoom level of the tile.

[0056] 3) Conversion of pixel coordinates (pixelX, pixelY) of the tile (tileX, tileY) to longitude and latitude coordinates (long, lat):

[0057]

[0058] In the above formula, l is the zoom level of the tile.

[0059] S1.3. Write the coordinate scheme into the world coordinates of the four corner points of the template to be processed to obtain a geographic base map, which is convenient for quickly retrieving and completing the matching positioning according to the pixel values after the matching is completed;

[0060] S1.4. Extract feature points from the geographic base map and determine the directions of the feature points to construct feature point descriptions to obtain a template image; Utilizing the feature invariance characteristics of the SIFT method, the rapid retrieval and matching of key features for UAV autonomous visual navigation based on the geographic base map are realized.

[0061] S1.4.1. Extract feature points from the geographic base map;

[0062] Specifically, first construct a Gaussian scale space to generate Gaussian blurred images of different scales; secondly, downsample the Gaussian blurred images to obtain a series of images with continuously decreasing sizes; finally, perform DOG space extreme value detection to remove some edge response points.

[0063] The Gaussian kernel is the only kernel that can generate a multi-scale space. In the input image model, the scale is continuously transformed parametrically through the Gaussian blur function, and finally a multi-scale space sequence is obtained. The specific calculation operations are as follows:

[0064]

[0065] L(x,y,σ) = G(x,y,σ) * I(x,y)

[0066] In the above formula, L(x,y,σ) is the spatial function of a certain scale in the image, G(x,y,σ) is the Gaussian function with variable parameters, I(x,y) is the original input image, σ is the scale space factor, and (x,y) are the coordinates of the feature points.

[0067] Among them, the smaller σ is, the clearer the local points are; conversely, the larger σ is, the more blurred the image is and the less it can reflect the details of the image.

[0068] S1.4.2. Determine the direction of the feature points;

[0069] Specifically, in order to achieve image rotation invariance, it is necessary to assign a direction to the feature points.

[0070] First, use the gradients of the pixels in the neighborhood of the feature points to determine their direction parameters; secondly, use the gradient histogram of the image to obtain the stable direction of the local structure of the feature points; finally, for the detected feature points, we can obtain the scale value of the feature points. Therefore, by determining this parameter, the Gaussian image at this scale can be obtained.

[0071] L(x,y) = G(x,y,σ) * I(x,y);

[0072] In the above formula, L(x,y) is the spatial function of a certain scale in the image, G(x,y,σ) is the Gaussian function with variable parameters, I(x,y) is the original input image, σ is the scale space factor, and (x,y) are the coordinates of the feature points.

[0073] Assign a direction to each extreme point through the gradient of the extreme point. The gradient magnitude is equal to the sum of the squares of the differences between the pixel values of the upper and lower points plus the sum of the squares of the differences between the pixel values of the left and right points, and the gradient direction is the quotient of the difference between the pixel values of the upper and lower points and the difference between the pixel values of the left and right points; that is:

[0074] Gradient magnitude:

[0075] Gradient direction:

[0076] S1.4.3. Construct the feature point description.

[0077] Specifically, for each feature point, there are three pieces of information: position, scale, and direction. Integrate the feature point and the feature point direction to establish a descriptor for each feature point, making it invariant to various changes, such as illumination changes and perspective changes. And the descriptor should have high uniqueness to improve the probability of correct matching of feature points.

[0078] To ensure the rotational invariance of the feature vector, it is necessary to rotate the position and direction of the image gradient in the neighborhood of the feature point by an angle θ with the feature point as the center, that is, to rotate the x-axis of the original image to the same direction as the main direction.

[0079] S1.4.4. Obtain the template image according to the feature point description.

[0080] S2. Obtain the photos taken by the airborne camera and perform feature extraction to obtain the image of the target to be measured;

[0081] Specifically, the feature extraction performed on the photos taken by the airborne camera here is the same as that in step S1.4.

[0082] S3. Based on the inertial measurement unit, perform motion constraints on the UAV, and match the template image with the image of the target to be measured to obtain the matching area;

[0083] S3.1. Refer to Figure 5 , based on the inertial measurement unit, use the error-state Kalman filter algorithm to estimate the motion state of the UAV;

[0084] S3.1.1. Use the inertial measurement unit to perform integral update on the nominal state of the UAV;

[0085] Specifically, the nominal state integral update model is as follows:

[0086]

[0087] In the above formula, p is the position, v is the velocity, a m is the acceleration measurement, a b is the acceleration bias, g is the gravitational acceleration, q is the rotation quaternion, ω m is the angular velocity measurement, ω b is the angular velocity bias, and R is the rotation matrix generated based on q.

[0088] Among them, R = R{q}, indicating the rotation from the body coordinate system of the IMU sensor to the inertial system.

[0089] It should be noted that the above equation completely does not consider noise, and the derivation of the equation is obvious. It only needs to assume that all noise terms are 0 on the basis of the motion equation of the inertial measurement unit.

[0090] S3.1.2. Use the error-state Kalman filter algorithm to perform time update and measurement update on the error state of the UAV;

[0091] Specifically, for the error-state time update, the goal of this process is to determine the linear dynamics of the error state, which is executed at each time step; for each state equation, solve the error state and simplify all second-order decimal error-state time update equations as follows:

[0092]

[0093] In the above formula, δp is the position increment, δv is the velocity increment, R is the rotation matrix, a m is the acceleration measurement, a b is the acceleration bias, δθ is the angle increment, δa b is the acceleration increment, δg is the acceleration increment, a n is the acceleration measurement noise, ω m is the angular velocity measurement, ω b is the angular velocity bias, δω b is the angular velocity increment, ω n is the angular velocity measurement noise.

[0094] For the error-state measurement update, in the measurement update, the error state is updated over time only when the measurement value changes. Since the data of the inertial measurement unit contains a large amount of noise, generally, the information of more sensors needs to be used to correct the state.

[0095] Generally, the basic form of the measurement equation is as follows:

[0096] y = h(X t ) + v;

[0097] In the above formula, h(·) is the measurement function, which is related to the on-board camera and the inertial measurement unit. v is Gaussian white noise, and v ~ N{0, V}.

[0098] The error-state measurement update equation is as follows:

[0099]

[0100] In the above formula, K is the Kalman gain, P is the error covariance matrix, H is the Jacobian matrix of the observation equation, V is the observation covariance matrix, is the state increment, ← is a mathematical relationship, representing approximation and trend.

[0101] S3.1.3. Combine the nominal state and the error state to obtain the actual state, that is, the motion state of the UAV;

[0102] Among them, the nominal state refers to the main trend of the UAV's motion state, the actual state refers to the actual situation of the UAV's motion state, and the error state refers to the difference between the actual state and the nominal state.

[0103] S3.1.4. Combine the nominal state and the error state and reset the error state.

[0104] Specifically, state combination is to combine the error state with the nominal state, that is, to correct the drift of the nominal state over time.

[0105]

[0106] In the above formula, x is the state vector, δx is the state increment, p is the position vector, δp is the position increment, v is the velocity vector, δv is the velocity increment, q is the quaternion, δθ is the angle increment, a b is the acceleration bias, δa b is the acceleration bias increment, ω b is the angular velocity bias, δω b is the angular velocity bias increment, g is the gravitational acceleration, δg is the change in gravitational acceleration, is matrix addition, is quaternion multiplication.

[0107] Among them, after adding the error to the nominal state, the error state mean is reset. This is particularly relevant to the orientation part because the new orientation error will be locally represented with respect to the orientation frame of the new nominal state. In order to complete the error state Kalman filter update, the covariance of the error needs to be updated according to this modification.

[0108] S3.2. Establish a constrained target image to be measured according to the UAV's motion state; make a relatively clear regional restriction on the target image to be measured, which improves the matching efficiency on the one hand and enhances the robustness of the positioning scheme on the other hand;

[0109] S3.3. Refer to Figure 6 , and match the constrained target image to be measured with the template image to obtain the matching area.

[0110] S3.3.1. Adopt the Euclidean distance to calculate the Euclidean distance between any two points of the constrained target image to be measured and the template image;

[0111] Specifically, first obtain the feature point descriptors of the constrained target image to be measured and the feature point descriptors of the template image, and then substitute the feature point descriptors of the target image to be measured and the feature point descriptors of the template image into the Euclidean distance calculation formula:

[0112]

[0113] In the above formula, Ri is the descriptor of the feature points of the template image, r ij is a point in R i and S i is the descriptor of the feature points of the image of the target to be measured, s ij is a point in S i and d(R i , S i ) is the Euclidean distance between any two points of the descriptor of the feature points of the template image and the descriptor of the feature points of the image of the target to be measured.

[0114] S3.3.2. Eliminate the error matching points according to the Euclidean distance and the data structure of the kd - tree to obtain the set of homologous points;

[0115] Specifically, the matching of the feature points is completed by using the data structure of the kd - tree for searching; the content of the search is based on the feature points of the image of the target to be measured, and search for the feature points of the original image that are the nearest neighbor and the second - nearest neighbor to the feature points of the image of the target to be measured; the calculation formula for eliminating the error matching points is as follows:

[0116]

[0117] In the above formula, Threshold is the image binarization.

[0118] When the above - mentioned formula conditions are met, they are the finally retained paired feature point descriptors, that is, the set of homologous points is obtained.

[0119] S3.3.3. Construct the matching area according to the set of homologous points.

[0120] S4. Construct a visual - inertial odometer and obtain the local pose of the UAV according to the output of the airborne camera and the output of the inertial measurement unit;

[0121] Specifically, let T W,k be the transformation from the UAV at the key frame k to the ENU frame.

[0122]

[0123] In the above formula, T W,k is the local pose of the k - th key frame, C W,k is the pose of the k - th frame, is the position in the ENU coordinate system.

[0124] Among them, represents the three - axis coordinates (x, y, z) of the position r.

[0125] C W,k =(φ W,k , θ W,k , ψ W,k ) and φW,k is the roll angle, θ W,k is the pitch angle, ψ W,k is the yaw angle, C W,k is the initial rotation matrix obtained through camera calibration, representing the attitude change matrix between camera frames.

[0126] The first step in the estimation is to perform visual odometry on the UAV images. The input is the rectified grayscale image and the non-static UAV-to-sensor transformation It is calculated at 10 Hz per frame. It is calculated by using the known translations between the three angles for a composite transformation and then rotated into the standard camera frame. When the yaw follows the UAV heading, the roll and pitch axes are globally stabilized in the gravity-aligned inertial frame.

[0127] For each frame of the image, features are extracted and SIFT descriptors are matched between frames to perform landmark triangulation. Features that cannot be triangulated through matching are triangulated through the motion between consecutive frames. The descriptors in the latest image are matched with the last key frame to generate 2D-3D point correspondences. The maximum likelihood estimation sample consensus (MLESAC) estimator is used to determine the pose T from the current frame to the last key frame f,k . If the translation or rotation exceeds the threshold, or the number of inliers is below the minimum value, a new key frame is added. For each new key frame, windowed refinement (bundle adjustment) is performed using simultaneous trajectory estimation and mapping (STEAM).

[0128] S5. Refer to Figure 7 , using the extended Kalman filter algorithm, the state of the matching region and the local pose of the UAV are jointly estimated to obtain the real-time pose of the UAV; using the consistency principle, through the comparison of the two, the drift of the inertial odometer is weakened, and the multi-source information fusion improves the accuracy of image matching.

[0129] Specifically, combining the uncertain relative transformation of T k,k-1 in visual odometry with the uncertain attitude measurement value of the initial pose T k,0 of image registration for state fusion. It should be noted that T W,0 is the transformation from the local coordinate system to the global coordinate system, which is constructed from the RTK pose at the first key frame. Therefore, the extended Kalman filter algorithm equation is expressed as follows:

[0130]

[0131] In the above formula, is the error covariance matrix, Q k is the covariance matrix of the process noise, T k,k-1 is the state transition matrix of the UAV from key frame k-1 to k, τ k,k-1is T k,k-1 is the adjoint matrix of is the initial pose estimate, is the error covariance matrix of the second-order approximation of the UAV at the key frame k, R k is the covariance matrix of the measurement noise, K k is the Kalman gain, ln(·) ∨ and exp(· ∧ ) are operators of SE(3).

[0132] Among them, SE(3) is rotation plus displacement, also known as Euclidean transformation and rigid body transformation. Generally, we use the matrix to represent it, where R is rotation and t is displacement. Therefore, it has 6 degrees of freedom, 3 rotations and 3 positions.

[0133] Furthermore, as a preferred embodiment of this method, it further includes correcting the outputs of the airborne camera and the inertial measurement; performing real-time data processing through the airborne data processing unit to reduce the task load of data transmission back and the remote control center, and increase the timeliness and security of the solution.

[0134] Specifically, the correction processing of the outputs of the airborne camera and the inertial measurement includes image frame distortion correction, camera coordinate conversion, data time error correction and time alignment, inertial measurement unit error correction, and data rationality selection.

[0135] 1) Image frame distortion correction

[0136] Distortion is an offset of the straight-line projection. Simply put, distortion means that a straight line cannot remain a straight line when projected onto an image. Distortion can be divided into two categories, including radial distortion and tangential distortion.

[0137] There are three effects of radial distortion. One is barrel distortion, another is pincushion distortion, and there is also a combination of the two called mustache distortion.

[0138] The correction calculation operation of radial distortion is as follows:

[0139]

[0140] The correction calculation operation of tangential distortion is as follows:

[0141]

[0142] After the above distortion correction, 5 distortion parameters D=(k1,k2,k3,p1,p2) are obtained;

[0143] In the above formula, (x corr , y corr ) is the distorted coordinate actually observed on the image plane, (xdis , y dis ), where \((x', y')\) are the coordinates after distortion correction, \(k_1\), \(k_2\), and \(k_3\) are radial distortion parameters, and \(p_1\) and \(p_2\) are tangential distortion parameters.

[0144] 2) Refer to Figure 4 , the camera coordinate transformation

[0145] In image processing and stereo vision, the world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system are often involved. Through the transformation of these four coordinate systems, we can obtain how a point is transferred from the world coordinate system to the pixel coordinate system.

[0146] The specific calculation operation from world coordinates to pixel coordinates is as follows:

[0147]

[0148] In the above formula, is the internal parameter of the camera, is the external parameter of the camera; the internal and external parameters of the camera can be obtained through Zhang Zhengyou calibration.

[0149] From the above transformation operations, we know that a coordinate point in three dimensions can find a corresponding pixel point in the image. However, conversely, finding its corresponding point in three dimensions through a point in the image is a problem because we do not know the value of \(Z\) on the left side of the equation. c value.

[0150] 3) Data time error correction and time alignment

[0151] The specific calculation formula for time interpolation alignment is as follows:

[0152]

[0153] In the above formula, \(T\) c (k) and \(\alpha(k)\) are equally spaced data sequences taken by the device, and \(t\) s (k) is the time series with an interval of 0.05 s after time alignment for each device.

[0154] Among them,

[0155] 4) Inertial measurement unit error correction

[0156] When manufacturing a multi-axis inertial measurement unit, the axes may not be perpendicular; therefore, calibration is required before use, that is, the errors of the inertial measurement unit are corrected.

[0157] Specifically, the accelerometer is calibrated using the six - face calibration method. The three axes of the accelerometer are placed horizontally, either facing up or down, for a period of time, and the data of six faces are collected to complete the calibration. When considering the cross - axis error, the relationship between the actual acceleration and the acceleration measurement value is as follows:

[0158]

[0159] In the above formula, \(l\) is the actual acceleration, \(a\) is the acceleration measurement value, \(s\) is the scale factor, \(m\) is the non - perpendicularity factor, and \(b\) is the measurement deviation. The variables in the above formula can be calculated using the least - squares method.

[0160] Different from the six - face method of the accelerometer, the true value of the gyroscope is provided by a high - precision turntable. Here, the six faces refer to the clockwise and counter - clockwise rotations of each axis. However, the calibration principle is similar. Only the inertial measurement unit needs to be placed horizontally, but usually a large amount of data needs to be measured to reduce the influence of uncertain errors, and the LM algorithm is used to find the optimal solution.

[0161] 5) Data rationality selection

[0162] For the acceleration, gyroscope, and magnetic compass measurement data of the inertial measurement unit, and the real - time frames of the monocular camera measurement device, rationality checks need to be carried out. The rationality check is mainly to eliminate the outliers in the measurement elements and prevent the measurement values containing gross errors from passing through the filter, thereby affecting the tracking accuracy. For the real - time image frames, only the matching results can be processed. Based on the consistency principle, the estimation results of the inertial measurement unit and the matching and positioning results of the previous moment are compared in real - time, and some matching results with large positioning deviations are eliminated. Here, a measurement - element - oriented selection algorithm based on the five - point linear prediction method is used.

[0163] Furthermore, as a preferred embodiment of this method, in order to implement the above - mentioned technical solution, referring to Figure 8 , the present invention provides a monocular UAV absolute vision matching and positioning system based on a geographic base map, including:

[0164] A template image construction module, used to construct a geographic base map of the UAV flight area and perform feature extraction to obtain a template image;

[0165] A to - be - measured target image construction module, used to obtain the photos of the on - board camera and perform feature extraction to obtain a to - be - measured target image;

[0166] A matching area construction module, based on the inertial measurement unit to perform motion constraints on the UAV, and match the template image with the to - be - measured target image to obtain a matching area;

[0167] A local pose estimation module, used to construct a visual - inertial odometer and obtain the local pose of the UAV according to the output of the on - board camera and the output of the inertial measurement unit;

[0168] A real-time pose estimation module, which uses the extended Kalman filter algorithm to jointly estimate the states of the matching area and the local pose of the UAV, and obtain the real-time pose of the UAV.

[0169] The content in the above method embodiments is applicable to the system embodiments of the present invention. The functions specifically implemented by the system embodiments of the present invention are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0170] The beneficial effects of the present invention specifically include:

[0171] 1) The UAV absolute vision positioning technology based on the geographical base map overcomes the error accumulation effect and time drift problem of relative vision positioning, but does not damage the positioning accuracy and efficiency of the relative positioning scheme at the same time.

[0172] 2) The vision sensor of the UAV absolute vision positioning technology based on the geographical base map is a passive perception sensor. It relies on external light to perceive the surrounding environment information through images. It has the characteristics of small volume, low cost, rich information, strong positioning autonomy and reliability, and high positioning accuracy. It is a very suitable autonomous navigation device for UAVs. In most photosensitive scenarios, the vision-based positioning technology can be used as an effective supplement to the GNSS navigation means.

[0173] 3) The UAV absolute vision positioning technology based on the geographical base map can replace the GPS scheme to obtain short-period absolute positioning information of the UAV when the GPS signal is noisy or signal rejection occurs in a complex battlefield environment and accurate absolute position cannot be obtained, or when the GPS signal is rejected or the positioning has a large deviation.

[0174] 4) The UAV absolute vision positioning technology based on the geographical base map has great search and rescue advantages and low cost. It forms a task cluster suitable for search and rescue tasks and search and rescue environments, has high flexibility, and its decentralized mode greatly reduces the difficulty of maintenance and upgrade.

[0175] The above is a specific description of the preferred embodiment of the present invention, but the present invention is not limited to the described embodiment. Those skilled in the art can make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A monocular UAV absolute vision matching and positioning method based on a geographic base map, characterized in that Including the following steps: Construct a geographic base map of the UAV flight area and perform feature extraction to obtain a template image; Obtain on-board camera photos and perform feature extraction to obtain an image of the target to be measured; Based on the inertial measurement unit, perform motion constraints on the UAV, match the template image with the image of the target to be measured, and obtain a matching area; According to the outputs of the on-board camera and the inertial measurement unit, construct a visual inertial odometer and obtain the local pose of the UAV; Using the extended Kalman filter algorithm, perform state joint estimation on the matching area and the local pose of the UAV to obtain the real-time pose of the UAV; The step of constructing a geographic base map of the UAV flight area and performing feature extraction to obtain a template image specifically includes: According to the longitude and latitude coordinates, image scaling level, and image style of the UAV flight area, select a download source and obtain image tiles; Unify and fuse the image tile coordinates to obtain a coordinate scheme; Write the coordinate scheme into the world coordinates of the four corner points of the template to be processed to obtain a geographic base map; Perform feature point extraction on the geographic base map, determine the feature point directions, construct feature point descriptions, and obtain a template image; The step of performing feature point extraction on the geographic base map, determining the feature point directions, constructing feature point descriptions, and obtaining a template image specifically includes: Construct a Gaussian scale space and perform feature point extraction on the geographic base map; Use the gradients of the pixels in the neighborhood of the feature points to determine the direction parameters of the feature points; Integrate the feature points and the direction parameters of the feature points, and establish descriptors for each feature point; Obtain a template image according to the feature point descriptions.

2. The monocular UAV absolute vision matching and positioning method based on a geographic base map according to claim 1, characterized in that The step of performing motion constraints on the UAV based on the inertial measurement unit, matching the template image with the image of the target to be measured, and obtaining a matching area specifically includes: Based on the inertial measurement unit, use the error-state Kalman filter algorithm to estimate the motion state of the UAV; Establish a constrained image of the target to be measured according to the motion state of the UAV; Match the constrained image of the target to be measured with the template image to obtain a matching area.

3. The monocular UAV absolute vision matching and positioning method based on a geographic base map according to claim 2, wherein The step of using the error-state Kalman filter algorithm to estimate the motion state of the UAV based on the inertial measurement unit specifically includes: Use the inertial measurement unit to perform integral update on the nominal state of the UAV; Use the error-state Kalman filter algorithm to perform time update and measurement update on the error state of the UAV; Combine the nominal state and the error state to obtain the motion state of the UAV.

4. The monocular UAV absolute vision matching and positioning method based on a geographic base map according to claim 3, characterized in that The step of matching the constrained image of the target to be measured with the template image to obtain a matching area specifically includes: Adopt the Euclidean distance to calculate the Euclidean distance between any two points of the constrained image of the target to be measured and the template image; According to the Euclidean distance and the data structure of the kd tree, eliminate the error matching points to obtain a set of corresponding points; Construct a matching area according to the set of corresponding points.

5. The monocular UAV absolute vision matching and positioning method based on a geographic base map according to claim 1, wherein The equation of the extended Kalman filter algorithm is expressed as follows: In the above formula, is the error covariance matrix, and Q k is the covariance matrix of the process noise, T k,k-1 is the state transition matrix of the UAV from key frame k - 1 to k, τ k,k-1 is the adjoint matrix of T k,k-1 , is the initial pose estimate, is the error covariance matrix of the second - order approximation of the UAV at key frame k, R k is the covariance matrix of the measurement noise, K k is the Kalman gain, ln(·) ∨ and exp(·^) are operators of SE(3).

6. The monocular UAV absolute vision matching and positioning method based on a geographic base map according to claim 1, wherein It also includes performing correction processing on the outputs of the on-board camera and the inertial measurement unit.

Images