Multi-uav high-precision matching positioning method

Abstract

The present application relates to a kind of multi-unmanned aerial vehicle high-precision matching positioning method, including multi-unmanned aerial vehicle data acquisition, multi-aerial image rough matching based on POS data, multi-aerial image precision matching based on the improved SIFT feature and image matching positioning etc.Four steps.The present application has been applied to POS data and image information of multi-unmanned aerial vehicle, realizes multi-aerial image correction and matching base map cutting based on POS data collinear solution, eliminates SIFT mis-matching point based on the secondary matching method of feature vector Euclidean distance, geographical distance, realizes multi-unmanned aerial vehicle aerial image feature matching, aerial fusion image and the matching of reference image, overcome the problem that positioning accuracy is directly subject to navigation system attitude measurement error in principle, and can effectively improve image matching probability and matching precision.The present application can realize the accurate positioning of multi-unmanned aerial vehicle aerial image multi-target, positioning precision is high, adaptability is good, easy to engineering implementation.

Description

TECHNICAL FIELD

[0001] The present application relates to a multi-unmanned aerial vehicle high-precision matching positioning method, belonging to the field of unmanned aerial vehicle remote sensing and surveying. BACKGROUND

[0002] Single unmanned aerial vehicle platform reconnaissance detection is limited by performance such as action radius, endurance time, practical ceiling and task load, and often has characteristics such as locality, incompleteness, uncertainty, and the like, and is difficult to perform target reconnaissance positioning tasks under complex conditions such as concealment, camouflage, time sensitivity and dispersion, and cannot meet high timeliness and high-quality target sensing requirements. Multi-unmanned aerial vehicle platforms can expand battlefield coverage and extension, realize information sharing and fusion processing, enhance the probability and efficiency of completing multiple tasks, improve the response capability to battlefield emergency situations, and improve the battlefield survival capability. Multi-unmanned aerial vehicles can obtain more timely, more comprehensive and more accurate battlefield region situation through multi-angle configuration for cooperative positioning of multiple targets.

[0003] Current multi-unmanned aerial vehicle cooperative positioning research mainly applies unmanned aerial vehicle position and attitude data, optical-electrical pan-tilt attitude and other positioning and orientation (POS) data for spatial solution, such as cross positioning and multi-point positioning methods, and realizes the improvement of multi-vehicle cooperative positioning accuracy compared with single-vehicle positioning by solving and restraining positioning errors through multi-vehicle measurement sensor data. However, there is still a lack of cooperative positioning methods that comprehensively use multi-unmanned aerial vehicle POS data and image information, and the comprehensive research of spatial solution positioning and matching positioning methods can overcome the problem that positioning accuracy is directly limited by navigation system attitude measurement errors.

[0004] Therefore, in view of the above analysis, a multi-unmanned aerial vehicle matching positioning method is proposed. Combined with the same region aerial photography sequence images obtained by multi-unmanned aerial vehicles, an image matching positioning method is applied to realize multi-unmanned aerial vehicle sequence image matching, sequence aerial photography image and reference image matching, and then the fused image is guided to a geographic information system (GIS) according to position information to obtain a region battlefield image containing target position information. The method helps to weaken the dependence on high-precision hardware measurement equipment, and effectively realizes multi-target accurate positioning and target situation awareness.

[0005] Therefore, in view of the above analysis, a multi-unmanned aerial vehicle matching positioning method is proposed. Combined with the same region aerial photography sequence images obtained by multi-unmanned aerial vehicles, an image matching positioning method is applied to realize multi-unmanned aerial vehicle sequence image matching, sequence aerial photography image and reference image matching, and then the fused image is guided to a geographic information system (GIS) according to position information to obtain a region battlefield image containing target position information. The method helps to weaken the dependence on high-precision hardware measurement equipment, and effectively realizes multi-target accurate positioning and target situation awareness. SUMMARY

[0006] In order to solve the problems in the prior art, the present application provides a multi-unmanned aerial vehicle high-precision matching positioning method and method.

[0007] A multi-unmanned aerial vehicle high-precision matching positioning method comprises the following steps:

[0008] S1, multi-UAV data acquisition, first, a digital satellite map with accurate position information or a pre-spliced image and the like is acquired as a reference matching image; then, video frame images and POS data of the multi-UAV are acquired in real time or in an offline loading manner, the POS data mainly including UAV position and attitude data, photoelectric platform attitude and the like;

[0009] S2, multi-aerial image coarse matching based on the POS data, first, position data of any point in the image is solved based on the collinear imaging equation, combined with the respective position and attitude angle of the multi-UAV, photoelectric platform attitude angle and the like, to realize correction of the multi-aerial images provided by each UAV, and to acquire the regional position of the multi-aerial images; then, based on the regional position of the multi-UAV images, the regional range of the reference image is determined, the reference satellite map or the spliced image is cropped, a small regional reference matching base map containing the reconnaissance image range of the multi-UAV is acquired, and a reference image cropping processing job based on the regional position is completed;

[0010] S3, multi-aerial image fine matching based on the improved SIFT feature, first, the matching and fusion of the multi-aerial images corrected in the S2 step are realized based on the matching method of the improved SIFT feature, to obtain a fusion image region covering a region larger than the coverage region of each aerial image of the UAV, and to suppress the residual correction error in the S2 step; then, the matching and fusion of the aerial fusion image and the reference image are realized based on the matching method of the improved SIFT feature.

[0011] S4, image matching positioning, the position information of the fusion image is guided to a geographic information system (GIS), and then the position data and regional situation information of any point on the aerial image are obtained.

[0012] Further, in the S1 step, the digital satellite map can be obtained by application or resource download and the like of a surveying and mapping department, and the pre-spliced image can be obtained by image splicing of pre-reconnaissance aerial photographs or videos.

[0013] Further, in the S2 step, when the image position is solved based on the collinear equation, the following steps are performed:

[0014] First step, coordinate conversion matrix solving, a conversion matrix of a UAV geographic coordinate system and an image space coordinate system is calculated, four coordinate systems including the image space coordinate system, the UAV body coordinate system, the UAV geographic coordinate system and the Gaussian plane rectangular coordinate system are applied, and the coordinate systems are defined as follows:

[0015] 1) image space coordinate system s, origin O s is a projection center, X s , Y sParallel to the imaging plane array frame and consistent with the image display storage direction, Z s According to the right-hand coordinate system, the camera optical axis is Z s axis;

[0016] 2) UAV body coordinate system b, coordinate origin O b is the aircraft center of mass, X b points to the right, Y b points forward, Z b points up;

[0017] 3) UAV geographic coordinate system e, coordinate origin O e is the aircraft center of mass, X e points east, Y e points north, Z e points to the sky; 4) Gauss plane rectangular coordinate system g;

[0018] At the same time, let the high angle of the aerial camera or camera holder be α, the azimuth angle be β, the heading angle of the UAV be ψ, the pitch angle be θ, and the tilt angle be γ. According to the coordinate transformation principle, the coordinate transformation matrix between different coordinate systems is obtained. Let the coordinates of the target in the UAV geographic coordinate system e be (x e , y e , z e ), and the coordinates of the target in the image space coordinate system s be (x s , y s , -f), and f be the focal length of the camera or camera. The coordinate transformation relationship between the two is represented as:

[0019]

[0020]

[0021]

[0022] Second step, solve the collinear imaging equation. Let the coordinates of the target in the Gauss plane rectangular coordinate system g be (x g , y g ), and the coordinates of the UAV in the Gauss plane rectangular coordinate system g be (x a , y a ), and H be the relative height of the UAV and the ground target point. Then, based on the UAV attitude data, the attitude data of the aerial camera or camera, and the telemetry parameters such as the internal parameters, the collinear condition equation of the imaging model can be obtained as

[0023]

[0024] Third step, coordinate position solution and image correction, according to the collinear imaging equation, the Gaussian plane rectangular coordinates of any pixel point in the aerial image can be solved. According to the pixel coordinates, Gaussian plane rectangular coordinates and coordinate transformation matrix of the aerial image, nearest neighbor interpolation resampling is applied to realize image correction. The orthographic corrected image can suppress image distortion, reduce the difference of shooting angle, etc., and has a similar view angle with the reference satellite map or pre-spliced image, thereby effectively improving the success rate and precision of image matching positioning.

[0025] Further, the reference image cropping based on the region position in the S2 step is performed according to the following steps:

[0026] First step, coordinate extreme value calculation, since the region range of the digital satellite map or pre-spliced image is much larger than the aerial image, the image matching region is reduced and the image matching time is shortened, the reference map is cropped according to the position of the four corner points of each aerial image to obtain the reference base map for image matching; wherein when the coordinate value is calculated, assuming that there are n images in total, the position of the four corner points of the i-th aerial image is (x i1 ,y i1 )、(x i2 ,y i2 )、(x i3 ,y i3 )、(x i4 ,y i4 ), and the coordinate extreme value (x min ,x max ,y min ,y max ) of each aerial image is

[0027]

[0028] Second step, reference image cropping, the reference image is cropped according to the cropped image coordinates to obtain the reference base map of a small region range, and the coarse matching of the aerial image is realized; in the coarse matching of the aerial image, the position coordinates obtained by the collinear positioning method have positioning errors; and assuming that the maximum positioning error value is l, the upper left corner coordinates of the cropped image can be determined as (x max +l,y min -l), and the lower right corner coordinates are (x min -l,y max +l); then the image is cropped according to the pixel coordinates, and the coordinate transformation between the position coordinates and the pixel coordinates is performed;

[0029] wherein when the position coordinates and the pixel coordinates are transformed, assuming that the left upper corner position coordinates of the reference image are (x0, y0), the left upper corner position coordinates of the cropped image are (x s ,y s ), and the pixel coordinates are (r s ,cs ), image pixel resolution t1 in x direction, image pixel resolution t2 in y direction, row rotation parameter t3, column rotation parameter t4, image north-up rotation parameter is 0, and the coordinate-pixel coordinate transformation relationship is represented as:

[0030]

[0031] Further, the matching method based on the improved SIFT feature in the S3 step mainly includes two links of improved SIFT feature extraction and description and feature matching based on secondary false matching elimination.

[0032] Further, the matching method based on the improved SIFT feature in the S3 step is performed in the improved SIFT feature extraction and description link according to the following steps.

[0033] Firstly, SIFT feature extraction is performed to extract SIFT feature key points in aerial correction images or between the aerial correction images and the reference cropped images; in order to improve the stability of feature point matching, the position and scale of the key points are accurately determined by a three-dimensional quadratic fitting function.

[0034] The SIFT feature extraction mainly includes two steps of scale space construction and spatial extreme point detection. The scale space construction is based on a DOG pyramid, and the scale space representation of a two-dimensional image I(x, y) at different scales is L(x, y, σ), which can be obtained by convolution of the image I(x, y) and a Gaussian kernel G(x, y, σ):

[0035]

[0036] Wherein, x and y represent the horizontal and vertical coordinates of the pixel points respectively, and σ represents the variance of the Gaussian normal distribution; a point in the scale space is the maximum or minimum value in the surrounding 8 points and the upper and lower 18 neighborhood points, so that the point is determined as an extreme point; in order to improve the stability of feature point matching, the position and scale of the key points are accurately determined by a three-dimensional quadratic fitting function.

[0037] Secondly, improved SIFT feature description is performed; the gradient direction projection of all pixel points in the neighborhood of the key point is used to determine the main direction of the key point, and the main direction of the feature is directly set to 0 when the main direction is determined; at the same time, the key point is kept rotationally invariant, 16 small neighborhoods of 4x4 pixels are selected with the key point as the center, 8 direction gradient histograms are formed in each small neighborhood, and finally a 128-dimensional feature vector is obtained.

[0038] Further, the matching method based on the improved SIFT feature in the S3 step is performed in the feature matching link based on secondary false matching elimination according to the following steps.

[0039] First step, apply the brute-force matching method to determine the initial matching point pairs between the images to be matched. Use the Euclidean distance of the feature vectors of the matching point pairs to preliminarily filter out the matching point pairs with large errors.

[0040] Calculate the Euclidean distance of the matching point pairs between the images to be registered, find the minimum and maximum distances of all matching point pairs. If the distances of the matching point pairs are less than the set threshold for the nearest and farthest distances, then retain the matching point pairs; otherwise, discard the matching point pairs.

[0041]

[0042] In the formula, x i,k is the k-th element of the feature vector of the i-th feature point in the matching image, y i,k is the k-th element of the feature vector of the j-th feature point in the reference image, n is the dimension of the feature vector, d i,j is the Euclidean distance of the matching point pair between the i-th feature point of the aerial image and the j-th feature point of the reference image, d max is the maximum Euclidean distance of the matching point pair, d min is the minimum Euclidean distance of the matching point pair, α and β are the set thresholds.

[0043] Second step, further eliminate the mismatched point pairs. Use the POS data to calculate the position coordinates and geographical distances of the matching point pairs, and use the maximum positioning error value as the threshold to eliminate the mismatched points, which can adapt to different types of input images and improve the matching probability and accuracy. Among them, define the pixel coordinates (r a , c a , c a ) and Gauss plane rectangular coordinates (x a , y a ) of the feature matching point P b in the reference image, and the pixel coordinates (r b , c b ) and Gauss plane rectangular coordinates (x b , y b ) of P a in the matching image. The geographical location distance k between P b and P

[0044]

[0045] If k < l, that is, the geographical distance k of the matching feature point pairs is less than the maximum positioning error l, then accept the matching point pairs; otherwise, discard the matching point pairs.

[0046] Third step, according to RANSAC (random sample consensus) algorithm, the transformation matrix between the images to be matched is calculated, the mis-matching points are effectively filtered out through the secondary matching method based on the Euclidean distance of feature vectors and geographical distance, and the calculation amount of RANSAC is reduced, wherein the pixel coordinates of the matching image are (r i ,c i ), the pixel coordinates of the matching control points of the reference image are (r ri ,c ri ), and the transformation matrix H is represented as

[0047]

[0048] Fourth step, the image fusion of the images to be matched is realized according to the image transformation matrix.

[0049] Compared with the conventional positioning method, the POS data and image information of multiple unmanned aerial vehicles are comprehensively applied, the POS data is applied to assist the matching positioning of the aerial images of multiple unmanned aerial vehicles, on one hand, the POS data collinear solution is used to realize the correction and matching of the multiple aerial images and the cutting of the base map, and on the other hand, the secondary matching method based on the Euclidean distance of feature vectors and geographical distance is used to remove the SIFT mis-matching points, the feature matching of the aerial images of multiple unmanned aerial vehicles and the matching between the aerial fusion images and the reference images are realized, the problem that the positioning accuracy is directly affected by the attitude measurement error of the navigation system is solved in principle, and the image matching probability and matching accuracy can be effectively improved. The present application can realize the accurate positioning of multiple targets of the aerial images of multiple unmanned aerial vehicles, has high positioning accuracy, good adaptability and is easy to realize in engineering. BRIEF DESCRIPTION OF DRAWINGS

[0050] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments;

[0051] Figure 1 It is a positioning method flowchart of the present application;

[0052] Figure 2 It is a multi-unmanned aerial vehicle high-precision matching positioning flowchart of the present application;

[0053] Figure 3 It is a matching method flowchart based on the improved SIFT feature of the present application;

[0054] Figure 4 It is a reference image cutting schematic diagram of the present application;

[0055] Figure 5 It is the first test result of the multiple aerial image matching positioning of the present application;

[0056] Figure 6 It is the second test result of the multiple aerial image matching positioning of the present application. DETAILED DESCRIPTION

[0057] In order to facilitate the construction of the technical means, the creation features, the purposes and the effects achieved by the present application, the present application is further described below in combination with specific embodiments.

[0058] As Figure 1 A multi-UAV high-precision matching positioning method, as shown in FIG. 4, comprises the following steps:

[0059] S1, multi-UAV data acquisition, first, a digital satellite map with accurate position information or a pre-spliced image and the like is acquired as a reference matching image; then, video frame images and POS data of the multi-UAV are acquired in real time or in an offline loading manner, the POS data mainly including UAV position and attitude data, photoelectric platform attitude and the like;

[0060] S2, multi-aerial image coarse matching based on POS data, first, the position data of any point in the image is solved based on the collinear imaging equation in combination with the respective position and attitude angle of the multi-UAV, photoelectric platform attitude angle and the like, so as to realize the correction of the multi-aerial images provided by each UAV and acquire the regional position of the multi-aerial images; then, based on the regional position of the multi-UAV images, the regional range of the reference image is determined, the reference satellite map or the spliced image is cropped, a small regional reference matching base map containing the reconnaissance image range of the multi-UAV is acquired, and the reference image cropping processing operation based on the regional position is completed;

[0061] S3, multi-aerial image fine matching based on improved SIFT features, first, the matching and fusion of the multi-aerial images corrected in the S2 step are realized based on the matching method of the improved SIFT features, a fusion image region with a coverage area larger than that of each aerial image of the UAV is obtained, and the residual correction error in the S2 step is suppressed; then, the matching and fusion of the aerial fusion image and the reference image are realized based on the matching method of the improved SIFT features;

[0062] Through the larger fusion image regional range, the correction error can be effectively suppressed, and the matching with the reference image is easily realized;

[0063] S4, image matching positioning, the position information of the fusion image is guided to a geographic information system (GIS), and then the position data and regional situation information of any point on the aerial image are obtained.

[0064] Further, in the S1 step, the digital satellite map can be obtained by application or resource download and the like from a surveying and mapping department, and the pre-spliced image can be obtained by image splicing of pre-reconnaissance aerial photographs or videos.

[0065] Further, in the S2 step, when the image position is solved based on the collinear equation, the following steps are performed:

[0066] The first step is to calculate the conversion matrix of the unmanned aerial vehicle geographic coordinate system and the image space coordinate system. Four coordinate systems are applied, including the image space coordinate system, the unmanned aerial vehicle body coordinate system, the unmanned aerial vehicle geographic coordinate system, and the Gauss plane rectangular coordinate system, which are defined as follows:

[0067] 1) Image space coordinate system s, with the origin O s as the projection center, X s , Y s parallel to the imaging plane array frame and consistent with the image display storage direction, and Z s determined according to the right-hand coordinate system, with the camera optical axis as the Z s axis.

[0068] 2) Unmanned aerial vehicle body coordinate system b, with the coordinate origin O b as the aircraft center of mass, X b pointing to the right, Y b pointing forward, and Z b pointing upward.

[0069] 3) Unmanned aerial vehicle geographic coordinate system e, with the coordinate origin O e as the aircraft center of mass, X e pointing east, Y e pointing north, and Z e pointing up. 4) Gauss plane rectangular coordinate system g

[0070] Meanwhile, let the high and low angles of the aerial camera or camera holder be α, the azimuth angle be β, the heading angle of the unmanned aerial vehicle be ψ, the pitch angle be θ, and the tilt angle be γ. According to the coordinate transformation principle, the coordinate transformation matrix between different coordinate systems is obtained. Let the coordinates of the target in the unmanned aerial vehicle geographic coordinate system e be (x e , y e , z e ), the coordinates of the target in the image space coordinate system s be (x s , y s , -f), and f be the focal length of the camera or camera, then the coordinate transformation relationship between the two is represented as:

[0071]

[0072]

[0073]

[0074] The second step is to solve the collinear imaging equation. Let the coordinates of the target in the Gauss plane rectangular coordinate system g be (x g , y g ), and the coordinates of the unmanned aerial vehicle in the Gauss plane rectangular coordinate system g be (x a , y a), H is the relative height of the UAV and the ground target point; then based on the telemetry parameters of the UAV attitude data, the attitude data and internal parameters of the aerial camera or video camera, the collinear condition equation of the imaging model can be obtained

[0075]

[0076] Thirdly, coordinate position calculation and image correction, according to the collinear imaging equation, the Gaussian plane rectangular coordinates of any pixel point in the aerial image can be calculated. According to the pixel coordinates of the aerial image, the Gaussian plane rectangular coordinates and the coordinate transformation matrix, the nearest neighbor interpolation resampling is applied to realize image correction. The orthographic corrected image can suppress image distortion, reduce the difference of shooting angle, etc., and has a similar view angle with the reference satellite map or the pre-spliced image, thereby effectively improving the success rate and accuracy of image matching positioning.

[0077] Further, the reference image cropping based on the region position in the S2 step is performed according to the following steps:

[0078] Firstly, coordinate extreme value calculation, since the region range of the digital satellite map or the pre-spliced image is much larger than the aerial image, the image matching region is reduced and the image matching time is shortened, the reference map is cropped according to the position of the four corner points of each aerial image to obtain the reference base map for image matching; wherein when calculating the coordinate value, assuming that there are n images, the position of the four corner points of the i-th aerial image is (x i1 ,y i1 ), (x i2 ,y i2 ), (x i3 ,y i3 ) and (x i4 ,y i4 ), then the coordinate extreme values (x min ,x max ,y min ,y max ) of each aerial image are

[0079]

[0080] Secondly, reference image cropping, the reference image is cropped according to the cropped image coordinates to obtain the reference base map of a small region range, thereby realizing the coarse matching of the aerial image; in the coarse matching of the aerial image, the position coordinates obtained by the collinear positioning method have positioning errors; assuming that the maximum positioning error value is l, then the upper left corner coordinates of the cropped image can be determined as (x max +l,y min -l), and the lower right corner coordinates are (x min -l,y max +l); then the image is cropped according to the pixel coordinates, and the coordinate transformation between the position coordinates and the pixel coordinates is performed.

[0081] wherein when the position coordinate and the pixel coordinate are transformed, the position coordinate (x0, y0) of the upper left corner of the reference image, the position coordinate (x s ,y s ) of the upper left corner of the cropped image, the pixel coordinate (r s ,c s ), the image pixel resolution t1 in the x direction, the image pixel resolution t2 in the y direction, the row rotation parameter t3, the column rotation parameter t4, and the rotation parameter of the north above the image being 0, the position coordinate and the pixel coordinate transformation relationship are represented as:

[0082]

[0083] Further, the matching method based on the improved SIFT feature in the S3 step mainly includes two links of improved SIFT feature extraction and description and feature matching based on secondary mismatch elimination.

[0084] Further, the matching method based on the improved SIFT feature in the S3 step is performed in the improved SIFT feature extraction and description link according to the following steps.

[0085] Firstly, SIFT feature extraction is performed to extract SIFT feature key points in the aerial correction images or between the aerial correction images and the reference cropped image; in order to improve the stability of the feature point matching, the position and scale of the key points are accurately determined by a three-dimensional quadratic fitting function;

[0086] wherein the SIFT feature is a kind of image scale invariant feature, the key points are extracted by using a Gaussian difference (Difference of Gaussian, DOG) operator, the position and main direction of the key points are accurately determined, and the local neighborhood descriptor of the key points is generated. The SIFT feature extraction mainly includes two steps of scale space construction and spatial extreme point detection. The basis of the scale space construction is a DOG pyramid, and the scale space representation of a two-dimensional image I(x, y) at different scales is L(x, y, σ), which can be obtained by the convolution of the image I(x, y) and the Gaussian kernel G(x, y, σ):

[0087]

[0088] wherein x and y respectively represent the horizontal and vertical coordinates of the pixel points, σ represents the variance of the Gaussian normal distribution, a certain point in the scale space is the maximum or minimum value in the surrounding 8 points and the upper and lower 18 neighborhood points, and the position and scale of the key points are accurately determined by a three-dimensional quadratic fitting function in order to improve the stability of the feature point matching;

[0089] Secondly, the SIFT feature description is improved. The main direction of the key point is determined by projecting the gradient direction of all pixel points in the neighborhood of the key point. The main direction of the feature is directly set to 0 when determining the main direction. Meanwhile, the key point is kept rotation-invariant. A 16-pixel small neighborhood is selected with the key point as the center. A gradient histogram of 8 directions is formed in each small neighborhood. Finally, a 128-dimensional feature vector is obtained.

[0090] For the application scenarios of multi-UAV aerial image feature matching, aerial correction image and reference image matching, the view angle of the image to be matched is similar, and the step of determining the main direction of the key point can be ignored, and the main direction of the feature is directly set to 0. The main direction of the key point is omitted, which can improve the algorithm speed, obtain more matching control points, and improve the matching success rate.

[0091] Further, the matching method based on the improved SIFT feature in the S3 step is designed based on the secondary matching method based on the Euclidean distance of the feature vector and the geographical distance of the matching point pair in the feature matching link based on the secondary mismatch elimination. Wherein:

[0092] The SIFT feature matching point often has a large number of error matching point pairs, which may reduce the matching positioning accuracy of the UAV, and even cause the matching positioning to fail. The traditional SIFT feature matching adopts a fixed threshold similarity measure to eliminate error feature matching points, which is difficult to adapt to the differences of different images. A secondary matching method based on the Euclidean distance of the feature vector and the geographical distance of the matching point pair is designed to fully eliminate the error matching point pairs, and the RANSAC algorithm is combined to solve the image transformation matrix. The specific steps are as follows:

[0093] Firstly, the initial matching point pairs between the images to be matched are determined by applying the brute force matching method. The Euclidean distance of the feature vector of the matching point pair is used to preliminarily filter out the matching point pairs with large errors.

[0094] The Euclidean distance of the matching point pairs between the images to be matched is calculated. The minimum and maximum distances of all matching point pairs are calculated. If the distance of the matching point pair is less than the nearest and farthest distances of the set threshold, the matching point pair is retained, otherwise the matching point pair is discarded.

[0095]

[0096] In the formula, x i,k is the kth element of the feature vector of the ith feature point in the matching image, y i,k is the kth element of the feature vector of the jth feature point in the reference image, n is the dimension of the feature vector, d i,j is the Euclidean distance of the matching point pair of the ith feature point of the aerial image and the jth feature point of the reference image, d max is the maximum Euclidean distance of the matching point pair, d minTo match the minimum Euclidean distance of the point pair, α and β are set threshold values;

[0097] In the second step, the matching point pair is further removed, the position coordinates and geographical distance of the matching point pair are calculated by using the POS data, the maximum positioning error value is taken as the threshold value to remove the mismatching point, different types of input images can be adapted, the matching probability and matching accuracy are improved, wherein the pixel coordinates (r a ,c a ) of the feature matching point P a in the reference image, the rectangular coordinates (x a ,y a ) in the Gaussian plane, the pixel coordinates (r b ,c b ) of P b in the matching image, the rectangular coordinates (x b ,y b ) in the Gaussian plane are defined. a The geographical distance k between P b and P

[0098]

[0099] If k < l, that is, the geographical distance k of the matching feature point pair is less than the maximum positioning error l, the matching point pair is accepted, otherwise the matching point pair is discarded;

[0100] In the third step, the transformation matrix between the matching images is calculated according to the RANSAC (random sample consensus) algorithm, the mismatching points are effectively filtered out by the secondary matching method based on the Euclidean distance of the feature vector and the geographical distance, the calculation amount of RANSAC is reduced, the pixel coordinates of the matching image are (r i ,c i ), the pixel coordinates of the reference image matching control point are (r ri ,c ri ), and the transformation matrix H is represented as

[0101]

[0102] In the fourth step, the image matching positioning is realized according to the image transformation matrix.

[0103] According to the position information of the fused image, the position data and regional situation information of any point on the aerial image are obtained, and the matching positioning test of the aerial image is performed.

[0104] As Figure 5As shown, based on the matching positioning method of the unmanned aerial vehicle high-precision matching positioning method, a matching positioning test software is designed, and the software interface mainly includes menu bar, GIS display area, state display area and the like. The unmanned aerial vehicle task load is all photoelectric pan-tilt, two aerial video frame images and corresponding POS data are collected, and the image resolution is 1920*1080. On the basis of collinear positioning solution, aerial image correction and reference map clipping are realized, aerial image matching fusion is carried out, and according to the matching image position data guided to the GIS system, the position data of any point can be obtained, and the matching positioning solution time is 19s. As can be seen, the aerial image is accurately matched to the reference image, the matching positioning error is small, the average positioning error is 15 meters, and the positioning precision is high.

[0105] As shown, Figure 6 The matching positioning test of multiple aerial images is carried out. The unmanned aerial vehicle task load is all photoelectric pan-tilt, three aerial video frame images and corresponding POS data are collected, and the image resolution is 1920*1080. On the basis of collinear positioning solution, aerial image correction and reference map clipping are realized, aerial image matching fusion is carried out, and according to the matching image position data guided to the GIS system, the position data of any point can be obtained, and the matching positioning solution time is 30s. As can be seen, the aerial image is accurately matched to the reference image, the matching positioning error is small, the average positioning error is 10 meters, and the positioning precision is high.

[0106] Compared with the traditional positioning method, the POS data and image information of multiple unmanned aerial vehicles are comprehensively applied, the POS data is applied to assist the matching positioning of multiple unmanned aerial vehicle aerial images, on the one hand, the collinear solution based on the POS data realizes the correction of multiple aerial images and the clipping of the matching base map, and on the other hand, the secondary matching method based on the Euclidean distance of the feature vector and the geographical distance eliminates the SIFT mismatching points, and the feature matching of multiple unmanned aerial vehicle aerial images and the matching of the aerial fusion image and the reference image are realized, which overcomes the problem that the positioning precision is directly affected by the attitude measurement error of the navigation system in principle, and can effectively improve the image matching probability and the matching precision. The present application can realize the accurate positioning of multiple targets of multiple unmanned aerial vehicle aerial images, has high positioning precision, good adaptability and is easy to implement in engineering.

[0107] The basic principles and main features of the present application and the advantages of the present application are shown and described. It should be understood by those skilled in the art that the present application is not limited by the above examples, and the above examples and descriptions in the specification are only to illustrate the principles of the present application, and various changes and improvements can be made without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A high-precision matching and positioning method for multiple unmanned aerial vehicles (UAVs), characterized in that: The multi-UAV high-precision matching and positioning method includes the following steps: S1, multi-UAV data acquisition: First, digital satellite maps or pre-stitched images with precise location information are acquired in advance; then, video frame images and POS data of multiple UAVs are acquired in real time or offline loading. The POS data mainly includes UAV position and attitude data, and electro-optical gimbal attitude. S2, coarse matching of multiple aerial images based on POS data, firstly combines the positions and attitude angles of multiple UAVs and the attitude angle POS parameters of the optoelectronic platform, and preliminarily calculates the position data of any point in the image based on the collinear imaging equation, thereby correcting the multiple aerial images provided by each UAV and obtaining the regional position of the multiple aerial images; then, based on the regional position of the multiple UAV images, the regional range of the reference image is determined, and digital satellite maps or pre-stitched images are cropped to obtain a small-area reference matching base map containing the range of the multiple UAV reconnaissance images, thus completing the reference image cropping processing based on the regional position; S3, based on improved SIFT features, performs fine matching of multiple aerial images. First, based on the improved SIFT feature matching method, the matching and fusion of each UAV aerial image after correction in step S2 is achieved, and the resulting fused image region with a coverage area larger than that of each UAV aerial image suppresses the residual correction error in step S2. Then, based on the improved SIFT feature matching method, the matching and fusion of the multi-UAV aerial image and the reference image is achieved. S4, image matching and positioning, guides the image location information after the aerial image and the reference image are fused to the geographic information system (GIS), thereby obtaining the location data of any point on the aerial image and the regional situation information.

2. The high-precision matching and positioning method for multiple unmanned aerial vehicles according to claim 1, characterized in that, In step S1, digital satellite maps can be obtained through application to surveying and mapping departments or by downloading resources, and pre-stitched images can be stitched together using aerial photographs or videos obtained from prior reconnaissance.

3. The high-precision matching and positioning method for multiple UAVs according to claim 1, characterized in that, In step S2, the image position calculation based on the collinearity equation is performed according to the following steps: The first step is to calculate the coordinate transformation matrix, specifically the transformation matrix between the UAV's geographic coordinate system and the image space coordinate system. Four coordinate systems are used: the image space coordinate system, the UAV's body coordinate system, the UAV's geographic coordinate system, and the Gaussian Cartesian coordinate system. The definitions of each coordinate system are as follows: 1) Imagine a spatial coordinate system s, with the origin O. s As the projection center, X s Y s Z is parallel to the outer frame of the imaging array and consistent with the image display and storage direction. s The camera's optical axis is determined using a right-handed coordinate system, with axis Z being the Z-axis. s axis; 2) The UAV's body coordinate system b, with the origin O. b For the center of gravity of the aircraft, X b Point to the right, Y b Z b Pointing upwards; 3) The UAV's geographic coordinate system is e, with the origin O. e For the center of gravity of the aircraft, X e Pointing east, Y e North, Z e Pointing to the sky; 4) Gaussian plane rectangular coordinate system g; Meanwhile, let the elevation angle of the aerial camera or camera gimbal be α, the azimuth angle be β, the heading angle of the UAV be ψ, the pitch angle be θ, and the tilt angle be γ. Based on the coordinate transformation principle, the coordinate transformation matrix between different coordinate systems is obtained. Let the coordinates of the target in the UAV's geographic coordinate system e be (x... e y e , z e The target's coordinates in the image space coordinate system s are (x...). s y s (-f), where f is the focal length of the camera or video camera, then the coordinate transformation relationship between the two is expressed as: The second step is to solve the collinear imaging equation. Let the coordinates of the target in the Gaussian plane rectangular coordinate system g be (x...). g y g The coordinates of the UAV in the Gaussian Cartesian coordinate system g are (x...). a y a Let H be the relative altitude between the UAV and the ground target point; then, based on the UAV attitude data, the attitude data of the aerial camera or video camera, and the intrinsic parameters and telemetry parameters, according to the collinearity condition equation of the imaging model, we can obtain... The third step is coordinate position calculation and image correction. Based on the collinear imaging equation, the Gaussian Cartesian coordinates of any pixel in the aerial image are calculated. Based on the pixel coordinates, Gaussian Cartesian coordinates and coordinate transformation matrix of the aerial image, the nearest neighbor interpolation resampling is applied to achieve image correction. Orthorectified images can suppress image distortion and reduce shooting angle differences, similar to the perspective of the reference satellite map or pre-stitched images, thereby effectively improving the success rate and accuracy of image matching and positioning.

4. The high-precision matching and positioning method for multiple UAVs according to claim 1, characterized in that, The reference image cropping based on region location in step S2 is performed according to the following steps: The first step is to calculate the extreme values ​​of the coordinates. A base map is cropped based on the corner positions of each aerial image to obtain the base map for image matching. When calculating the coordinate values, there are n images in total, and the corner positions of the i-th aerial image are (x...). i1 ,y i1 ), (x i2 ,y i2 ), (x i3 ,y i3 ), (x i4 ,y i4 If the extreme values ​​of the coordinates (x, y) of each aerial image are obtained, then the extreme values ​​of the coordinates (x min ,x max ,y min ,y max )for The second step is reference image cropping. Based on the coordinates of the cropped image, a reference base map of a small area is obtained, achieving coarse matching of the aerial image. In this coarse matching, the position coordinates obtained using the collinear positioning method have positioning errors. Assuming the maximum positioning error is l, the coordinates of the upper left corner of the cropped image can be determined as (x...). max +l,y min -l), the coordinates of the lower right corner are (x min -l,y max +l); Then, image cropping is performed based on pixel coordinates, and a coordinate transformation between position coordinates and pixel coordinates is performed; When performing the position coordinate to pixel coordinate transformation, let the position coordinates of the top left corner of the reference image be (x0, y0), and the position coordinates of the top left corner of the cropped image be (x0, y0). s ,y s ), pixel coordinates (r) s ,c s The image pixel resolution in the x-direction is t1, the image pixel resolution in the y-direction is t2, the row rotation parameter is t3, the column rotation parameter is t4, and the rotation parameter is 0 when the top of the image points north. The relationship between the set coordinates and pixel coordinates is expressed as follows: .

5. The high-precision matching and positioning method for multiple unmanned aerial vehicles according to claim 1, characterized in that, The matching method based on improved SIFT features in the step S3 mainly includes two links: improved SIFT feature extraction and description, and feature matching based on quadratic false match elimination.

6. A high-precision matching and positioning method for multiple unmanned aerial vehicles (UAVs) according to claim 1 or 5, characterized in that, For the matching method based on improved SIFT features in the step S3, in the link of improved SIFT feature extraction and description, the following steps are carried out: The first step is SIFT feature extraction, which extracts SIFT feature key points between aerial rectified images or between an aerial rectified image and a reference cropped image. To improve the stability of feature point matching, the position and scale of key points are accurately determined by a three-dimensional quadratic fitting function. Among them, SIFT feature extraction mainly includes two steps: scale space construction and spatial extreme point detection. The basis of scale space construction is the DOG pyramid. The scale space representation of a two-dimensional image I(x, y) at different scales is L(x, y, σ), which can be obtained by the convolution of the image I(x, y) and the Gaussian kernel G(x, y, σ): Among them, x and y respectively represent the horizontal and vertical coordinates of pixel points, σ represents the variance of the Gaussian normal distribution. If a point in the scale space is the maximum or minimum value among the surrounding 8 points and the 18 neighborhood points in the upper and lower layers, then this point can be determined as an extreme point. To improve the stability of feature point matching, the position and scale of key points are accurately determined by a three-dimensional quadratic fitting function. The second step is improved SIFT feature description. Using the gradient direction projection of all pixel points in the neighborhood of the key point, the main direction of the key point is determined. When determining the main direction, the feature main direction is directly set to 0; at the same time, the key point is made rotation-invariant. A 16 4×4 pixel small neighborhood is selected with the key point as the center, and an 8-direction gradient histogram is formed in each small neighborhood, and finally a 128-dimensional feature vector is obtained.

7. A high-precision matching and positioning method for multiple unmanned aerial vehicles (UAVs) according to claim 1 or 5, characterized in that, For the matching method based on improved SIFT features in the step S3, in the link of feature matching based on quadratic false match elimination, the following steps are carried out: The first step is to apply the brute-force matching method to determine the initial matching point pairs between the待匹配图像之间的初始匹配点对, and use the Euclidean distance of the feature vectors of the matching point pairs to filter out the matching point pairs whose errors do not meet the set threshold. Calculate the Euclidean distance between the matching point pairs of the待配准图像间匹配点对, find the minimum and maximum distances of all matching point pairs. If the distance of the matching point pair is less than the nearest and farthest distances of the set threshold, then keep this matching point pair, otherwise discard this matching point pair. In the formula, x i,k To match the k-th element of the feature vector of the i-th feature point in the image, y i,k Let be the k-th element of the eigenvector of the j-th feature point in the reference image, where n is the dimension of the eigenvector, and d i,j Let d be the Euclidean distance between the matching point pairs of the i-th feature point in the aerial image and the j-th feature point in the reference image. max To match the maximum Euclidean distance between point pairs, d min To find the minimum Euclidean distance between matching point pairs, α and β are set thresholds; The second step further eliminates mismatched point pairs. Using POS data, the location coordinates and geographical distance of matching point pairs are calculated. The maximum positioning error value is used as a threshold to eliminate mismatched points. This method can adapt to different types of input images, improving matching probability and accuracy. Specifically, a feature matching point P in the reference image is defined. a pixel coordinates (r) a ,c a Gaussian plane rectangular coordinates (x) a ,y a ), matching P in the image b pixel coordinates (r) b ,c b Gaussian plane rectangular coordinates (x) b ,y b ); P a and P b The geographical distance k is If k < l, that is, the geographical distance k of the matching feature point pair is less than the maximum positioning error l, then accept this matching point pair, otherwise discard this matching point pair. The third step involves calculating the transformation matrix between the images to be matched based on the RANSAC (random sample consensus) algorithm. A secondary matching method based on Euclidean distance and geographic distance of feature vectors effectively filters out mismatched points, reducing the computational cost of RANSAC. Let the pixel coordinates of the matched image be (r... i ,c i The pixel coordinates of the control points for matching the reference image are (r ri ,c ri The transformation matrix H is represented as follows: The fourth step is to realize the image fusion of the待匹配图像 according to the image transformation matrix. It should be noted that there are some unclear expressions in the original text such as "待匹配图像之间的初始匹配点对", "待配准图像间匹配点对", etc. which need to be further clarified in the original context for more accurate translation.

Images