Hybrid direct-to-ground positioning SLAM method and system oriented to rapid orthoimage generation

Abstract

The invention belongs to the technical field of unmanned aerial vehicle remote sensing images, and relates to a mixed direct to-the-ground positioning SLAM method and system for fast generation of orthoimages in a weak texture scene, the method and system are applied to the weak texture scene, and the method comprises the following steps: extracting ORB feature points for each image; performing rough matching on the feature points based on a K-nearest neighbor algorithm and a Low's algorithm; optimizing matching point pairs after rough matching based on an RANSAC (Random Sample Consensus) algorithm; removing mismatching point pairs based on a Delaunay triangulation network algorithm and reprojection constraint to obtain fine matching point pairs, estimating camera pose changes, and calculating a visual SLAM estimation pose of each frame of image; when the extremely weak texture scene condition causes the visual SLAM multi-constraint lightweight image matching failure, an orthoimage is generated by adopting direct earth positioning; and based on the exterior orientation elements of the image, correcting a pose resolving error and an accumulative error of the visual SLAM method to obtain a final orthoimage. According to the invention, flexible switching between the visual SLAM technology and the direct to-ground positioning technology in the orthoimage generation process is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle remote sensing images, and in particular relates to a hybrid direct ground positioning SLAM method and system for rapid generation of orthophotos for weak-texture scenes. Background Art

[0002] UAV low-altitude remote sensing orthophoto (Digital Orthophoto Map, DOM) is a digital map generated by processing UAV low-altitude aerial photography data. It plays an important role in geographic surveying and mapping, land use planning, environmental monitoring and other fields, and helps to accurately measure, analyze and understand geographic space. The rapid generation of orthophotos can provide instant and accurate geographic information, which is suitable for emergency surveying and mapping, navigation planning and environmental change monitoring. It provides key data support for practitioners in related industries and helps to make real-time decisions. The current methods for generating orthophotos are mainly divided into photogrammetry, direct ground positioning and SLAM-based methods.

[0003] The core of the existing photogrammetry method for generating orthophotos is to restore the camera pose and obtain the three-dimensional coordinates of the image points through the SfM (Structure from Motion) algorithm, so it can be divided into two categories: global SfM and incremental SfM. Both global SfM and incremental SfM have strong robustness, accuracy and completeness. However, due to the large amount of calculation, photogrammetry methods often take several hours or more to complete the reconstruction of a scene, which makes this method difficult to apply to emergency mapping with limited computing power.

[0004] The method of directly positioning on the ground to generate orthophotos has a simple process and a small amount of calculation, and can meet the needs of emergency mapping in terms of efficiency. And because there is no need for image matching, it avoids the problem of lack of feature points in weak texture scenes from the root. However, the direct ground positioning method has very high accuracy requirements for airborne sensors such as GNSS and IMU, which are often not met by sensors carried by consumer-grade drones and low-cost mapping-grade drones. In addition, extreme environments such as mountains, islands and polar regions may have inaccurate measurements of POS systems, making the direct ground positioning technology relatively poor in robustness in emergency mapping in extreme environments, and difficult to use as the main means of generating orthophotos.

[0005] The method of generating orthophotos based on visual SLAM has a strong timeliness due to the fast advantages of SLAM technology, and can better meet the needs of emergency mapping in terms of speed. However, compared with traditional photogrammetry methods, the method of generating orthophotos based on SLAM has two problems: (1) Visual SLAM usually selects video streams or images with high overlap and low resolution as input data. However, drone aerial survey images in the field of mapping usually have low overlap and high resolution, which may cause the failure of visual SLAM pose solution. (2) For efficiency reasons, the feature point extraction and matching algorithms selected by visual SLAM technology are usually not robust, especially for weak texture scenes. In scenes with weak texture, difficult feature points to obtain, or even extremely textureless, the method of generating orthophotos based on SLAM is likely to fail to extract feature points and match feature points incorrectly, resulting in the failure of orthophoto generation. Summary of the invention

[0006] In order to solve the problems existing in the prior art, the present invention proposes a hybrid direct ground positioning SLAM method and system for rapid generation of orthophotos, introduces direct ground positioning technology, and realizes flexible switching between visual SLAM and direct ground positioning technology during the orthophoto generation process; at the same time, the image extra-method elements obtained by the POS system are combined to reduce the influence of the cumulative error and the visual SLAM pose solution error, thereby improving the robustness of the algorithm in quickly generating high-quality orthophotos in various weak-texture scenes.

[0007] In an embodiment of the present invention, a hybrid direct-to-ground positioning SLAM method for rapid generation of orthophotos includes the following steps:

[0008] S1, extract ORB feature points for each image;

[0009] S2, performing rough matching on the extracted feature points based on K nearest neighbor algorithm and Lowe's algorithm;

[0010] S3, optimizing the matching point pairs after rough matching based on the RANSAC algorithm;

[0011] S4, based on the Delaunay triangulation algorithm and reprojection constraints, remove the wrong matching point pairs, obtain the precise matching point pairs, estimate the camera's pose change, and calculate the visual SLAM estimated pose of each frame of image;

[0012] S5. When the extremely weak texture scene causes the lightweight image matching of visual SLAM multi-constraints to fail, the direct ground positioning technology is used to generate orthophotos;

[0013] S6. Based on the external orientation elements of the image, the pose solution error and cumulative error of the visual SLAM method are corrected to obtain the final orthophoto.

[0014] Preferably, step S3 uses the RANSAC algorithm and the flatness of the ground to iteratively fit the homography matrix of the image, solves the homography matrix that achieves the relatively optimal matching effect, and optimizes the coarse matching point pairs based on the K nearest neighbor algorithm and Lowe's algorithm.

[0015] Further preferably, step S3 comprises:

[0016] Randomly select 4 matching point pairs from the coarse matching points as the internal points, i.e., the set of observation data, and the other matching points as the external points; then use the internal points to calculate the homography matrix;

[0017] Through the calculated homography matrix, a model with different numbers of inliers is established; the projection error between other matching points and the model is calculated according to the homography matrix, and a threshold t is set. If the projection error is less than the threshold t, it is considered a new inlier; if it is greater than the threshold t, it is considered an outlier; all the inliers are counted to update the model, the number of iterations k is set, and the update process of the model is iterated repeatedly, and finally the model with the most inliers is obtained as the best model output by the RANSAC algorithm;

[0018] The pose changes of the image are constrained through the optimal model, so as to retain the correct matching point pairs to the greatest extent and optimize the matching effect.

[0019] A hybrid direct ground positioning SLAM system for rapid generation of orthophotos according to an embodiment of the present invention is implemented based on the above hybrid direct ground positioning SLAM method and includes the following modules:

[0020] Feature point extraction module, extracts ORB feature points for each image;

[0021] The coarse matching module performs coarse matching on the extracted feature points based on the K nearest neighbor algorithm and Lowe's algorithm;

[0022] The optimization module optimizes the matching point pairs after rough matching based on the RANSAC algorithm;

[0023] The precise matching module eliminates mismatched point pairs based on the Delaunay triangulation algorithm and reprojection constraints, obtains precise matching point pairs, estimates the camera's pose change, and calculates the visual SLAM estimated pose of each frame of image;

[0024] Direct ground positioning module, when the extremely weak texture scene causes the lightweight image matching of visual SLAM multi-constraints to fail, the direct ground positioning technology is used to generate orthophotos;

[0025] The error correction module corrects the pose solution error and cumulative error of the visual SLAM method based on the image exterior orientation elements to obtain the final orthophoto.

[0026] Compared with the prior art, the present invention has the following advantages and effects:

[0027] 1. The present invention introduces direct ground positioning technology to achieve flexible switching between visual SLAM and direct ground positioning technology during orthophoto generation, solving the problem of SLAM solution failure caused by weak texture or extreme textureless scenes.

[0028] 2. The multi-constrained lightweight image matching technology proposed in this invention reduces the adverse effects of sparse or repeated textures in the survey area on drone image matching, and improves the applicability and robustness of feature point matching in the visual SLAM algorithm. It can not only achieve good matching results in a variety of scenarios, but also ensure the efficiency of visual SLAM technology, and improve the applicability and robustness of visual SLAM methods in generating orthophotos in different scenarios.

[0029] 3. The present invention improves the switching mechanism between visual SLAM and direct ground positioning technology, and applies constraints based on image external orientation elements to the pose estimation of visual SLAM. Compared with a single visual SLAM method, the present invention can detect the situation where the pose solution error and the cumulative error are too large in time, realize the timely switching between the visual SLAM method and the direct ground positioning technology, and effectively use the image external orientation elements to improve the integrity and accuracy of the generated orthophoto. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flowchart of a multi-constrained lightweight image matching method used in an embodiment of the present invention;

[0031] Figure 2 This is a flow chart of Delaunay triangulation matching in an embodiment of the present invention;

[0032] Figure 3 Schematic diagram of similarity of Delaunay triangulation of feature points in an embodiment of the present invention, where (a) is image A and (b) is image B;

[0033] Figure 4 A flow chart of a hybrid direct ground positioning SLAM algorithm for rapid generation of orthophotos for weak-texture scenes in an embodiment of the present invention;

[0034] Figure 5 The figure is a flow chart of error correction based on image exterior orientation elements in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0036] Example

[0037] This embodiment provides a hybrid direct ground positioning SLAM method for rapid generation of orthophotos, which is mainly used for rapid generation of low-altitude remote sensing orthophotos of low-cost, lightweight and small UAVs, including weak-texture scenes such as snow, ocean, forest and grassland. In order to improve the image matching success rate of the visual SLAM method, this embodiment considers the problem of sparse feature points or high similarity in UAV images caused by weak-texture environments, and proposes a lightweight SLAM image matching strategy based on multiple constraints, which is divided into four stages of "feature point extraction, rough matching, matching point pair optimization, and error matching elimination" for feature point matching. The specific process is as follows: Figure 1 As shown, the steps corresponding to each stage are described in detail as follows.

[0038] S1. Extract ORB feature points for each image.

[0039] In order to meet the requirements of rapid generation of orthophotos, this embodiment extracts ORB feature points from each image. ORB feature points have the characteristics of rapid extraction and high quality, and are suitable for feature point extraction of different types of images. They also have good feature point extraction effects in weak texture environments such as snow and woodland. When extracting ORB feature points, the FAST (features from accelerated segment test) algorithm is used to detect key points, and the BRIEF (Binary Robust Independent Elementary Features) algorithm is combined to calculate the descriptors of feature points, realizing real-time calculation of high-quality feature points.

[0040] S2. Perform rough matching on the extracted feature points based on K nearest neighbor algorithm and Lowe's algorithm.

[0041] First, in order to obtain as many matching point pairs as possible in a weak texture environment and ensure that the subsequent feature point matching optimization steps can work, this step uses the KNN algorithm to match the feature points. Through the KNN algorithm, K=2 most similar feature points can be found in the B image for each feature point of the A image. Afterwards, since the BRIEF descriptor of the ORB feature point uses binary coding to describe the key points, in order to select more similar matching points from the candidate points of the KNN matching, this embodiment uses Lowe's algorithm, combined with the Hamming distance of the BRIEF descriptor, to calculate the number of different values ​​at the same position in the binary string of different candidate points, and further screen with a preset threshold of r=0.8 to select a better matching point from the two candidate points of the KNN matching.

[0042] The key steps of Lowe's algorithm can be expressed by formula (1) and formula (2).

[0043]

[0044] Among them, P in formula (1) i is the description vector of a feature point in image A, P ′ i is the description vector of the candidate matching point of image B, D H It describes the Hamming distance between two points; P1 in formula (2) is a point in image A, and P2 and P3 are two candidate points in image B.

[0045] S3. Optimize the matching point pairs after rough matching based on the RANSAC algorithm.

[0046] The homography matrix is ​​a projection matrix that describes one plane to another plane. It can be used to describe the position transformation of feature points in feature point matching. The RANSAC algorithm optimizes the result of feature point matching by calculating the best homography matrix between two images. Emergency mapping is often applied to weak texture scenes such as snow and woodland, which usually have small surface height difference and flat ground, which can facilitate the application of the RANSAC algorithm and the calculation of the homography matrix.

[0047] In this step, the homography matrix of the image is iteratively fitted through the RANSAC algorithm and the assumption of flat ground, and the homography matrix that can achieve the relatively optimal matching effect is solved, and the coarse matching point pairs based on the K nearest neighbor algorithm and Lowe's algorithm are optimized.

[0048] S31. First, among the coarse matching points, four matching point pairs (non-collinear) are randomly selected as the internal points, i.e., the set of observation data, and the other matching points are external points; then, the homography matrix is ​​calculated according to formula (3) using these four pairs of internal points.

[0049]

[0050] Among them, (x, y) is the pixel coordinate of the feature point on the image, (X ′ ,Y ′ ) is the coordinate of the target point in the world coordinate system (assuming the target point is located in the plane of Z=0), and s is the scale parameter.

[0051] S32. Establish a model with different numbers of inliers through the calculated homography matrix; calculate the projection error between other matching points and the model according to the homography matrix, set a threshold t, if the projection error is less than the threshold t, it is considered to be a new inlier, if it is greater than the threshold t, it is considered to be an outlier, that is, a mismatched point; count all the inliers to update the model, set the number of iterations k, and iterate the update process of the model in a loop, and finally obtain the model with the most inliers as the best model output by the RANSAC algorithm.

[0052] S33, through the best model within the limit of the number of iterations, constrain the change of the image posture, so as to retain the correct matching point pairs to the greatest extent and optimize the matching effect. After the matching point pairs are optimized by the RANSAC algorithm, the rough matching is based on the K nearest neighbor algorithm and Lowe's algorithm.

[0053] S4. Based on the Delaunay triangulation algorithm and reprojection constraints, the incorrect matching point pairs are eliminated to obtain the precise matching point pairs, and the camera posture change is estimated to calculate the visual SLAM estimated posture of each frame of the image.

[0054] As a special graph structure, Delaunay triangulation has dynamic generation and good geometric characteristics. For natural object images taken at an orthographic angle, the Delaunay triangulation structure formed by the organization of feature points has a certain similarity, which is of great significance for feature point matching.

[0055] This embodiment uses the Delaunay triangulation to organize initial matching points, local geometric constraints based on spatial angle order, and matching point expansion steps under triangulation constraints to eliminate mismatched point pairs based on the rough matching of the K nearest neighbor algorithm and Lowe's algorithm and the optimization of matching point pairs using the RANSAC algorithm. The specific process is as follows: Figure 2 shown.

[0056] S41, construct a Delaunay triangulation network G1 for the ORB feature points of image A after rough matching and matching point pair optimization, and construct a matching graph G2 in image B based on the point pair relationship. Since the drone images used in this embodiment are all taken from an orthographic angle, the Delaunay triangulation networks of feature points of adjacent images have certain similarities, and the order of feature points around the same feature point is similar, such as (a) (b)

[0057] Figure 3 shown.

[0058] S42, based on the similarity of the feature point arrangement order, calculate the spatial angle order similarity. First, randomly select a vertex v in the triangulated network G1 1i , the vertex v 1i The connection points are arranged in order according to the polar angle And calculate the vertex v 1i The matching candidate point v in image B 2i Order Next, calculate the vertex v 1i and candidate point v 2i The spatial angle order similarity score is calculated as follows:

[0059]

[0060] Among them, d ced (·) is used to calculate the angle order distance, N represents the vertex v 1i The number of vertices connected in the same triangulation network, j is the number of feature points in image A. The threshold of the spatial angle order similarity score is set according to the image dataset used. If the matching point pair score is higher than the threshold, it is considered to be a correct matching point pair.

[0061] S43. For matching point pairs whose spatial angle order similarity scores are lower than a set threshold, use triangle local constraints to further expand the matching.

[0062] The Delaunay triangulation divides the image plane into smaller, approximately equilateral triangle patches. Thanks to the similarity of the arrangement of feature points in the images taken at an orthographic angle, the Delaunay triangulation constrains the position of the feature points on the image. First, find the feature point p in the triangle Δabc formed by a feature point of image A. i , and find the corresponding triangles Δa′b′c′ and p in image B i ′ , if p i ′ If the matching point pairs are still within Δa′b′c′, the matching point pairs that do not meet the above spatial angle order similarity score requirements will be restored to correct matches.

[0063] S44. Based on the elimination of mismatched points in the Delaunay triangulation algorithm, the reprojection constraint is used to continue eliminating mismatched point pairs.

[0064] The projection of the three-dimensional space point on the image when the camera is shooting, that is, the target point P captured by the camera in the world coordinate system is mapped to the pixel point P1 on the left image as the first projection. After the drone is equipped with a camera, the camera's position [R, t] and the coordinates of point P in the world coordinate system can be calculated through epipolar geometry and other methods, so the pixel point of the target point P on the right image can be calculated. This is the second projection, i.e. reprojection. Through feature matching, we can obtain the matching point P2 of pixel point P1 on the right image. The matching point P2 and pixel point The difference between is the reprojection error. Due to the inaccuracy of the image orientation elements and the errors in the projection matrix calculation, the reprojection error is usually not 0. Therefore, it is necessary to minimize the reprojection error to obtain the optimal camera pose and target point coordinates. The reprojection error can be written as:

[0065]

[0066] After the reprojection constraint, the mismatched point pairs can be further eliminated to obtain the final precise matching point pairs.

[0067] After the four-stage feature point matching process of "feature point extraction, rough matching, matching point pair optimization, and error matching elimination" of the multi-constrained lightweight image matching technology of this embodiment, higher quality matching point pairs can be obtained with higher efficiency in weak texture environments such as snow and woods.

[0068] S5. When the extremely weak texture scene causes the lightweight image matching of visual SLAM multi-constraints to fail, direct ground positioning technology is used to generate orthophotos.

[0069] A preferred implementation method of direct ground positioning technology is as follows:

[0070] Direct ground positioning technology can obtain the world coordinates of ground points through the point projection coefficient method. Specifically, the coordinates (X, Y, Z) of any target point in the world coordinate system can be expressed by formula group (6):

[0071]

[0072] Among them, (X1, Y1, Z1) is the coordinate of the target point in the auxiliary coordinate system of the image space, are the positions of the camera on the X, Y, and Z axes when the left and right images were taken, i.e., the exterior orientation line elements; N1 and N2 are the point projection coefficients of the left and right image points, respectively, and are calculated as follows:

[0073]

[0074] In formula group (7), B X , B Y, B Z are the three baseline components of the photographic epipolar line in the auxiliary coordinate system of the image space, which can be calculated by formula group (8):

[0075]

[0076] The coordinates of the target point in the auxiliary coordinate system of the image space can be expressed by the coordinates (x1, y1) and (x2, y2) in the pixel coordinate systems of the left and right images as follows:

[0077]

[0078] Where R1 and R2 are the orthogonal transformation matrices composed of the exterior azimuth elements of the left and right images, respectively, and f is the camera principal distance. Thus, the three-dimensional coordinates of each image point on the image in the world coordinate system can be calculated.

[0079] Although the multi-constrained lightweight image matching technology can improve the matching effect of feature points in most scenes, there are still some extreme textureless scenes that are prone to cause the visual SLAM method to fail in image matching; or weak texture scenes lead to large errors in the visual SLAM solution posture and cumulative errors. In response to the above problems, this embodiment combines the characteristics of the POS system currently carried by the UAV platform to obtain the external orientation elements of the image. On the basis of the improved visual SLAM image matching method, it mixes the direct ground positioning technology to ensure that high-quality orthophotos can still be obtained in the event of image matching failure. The specific process is as follows: Figure 4 shown.

[0080] First, input the data required for the present embodiment, including information such as drone aerial photography, camera internal and external orientation elements. Subsequently, multi-constraint lightweight image matching technology is used for fast matching of visual SLAM images. If the extremely weak texture scene situation causes the multi-constraint lightweight image matching of visual SLAM to fail, the direct ground positioning technology is directly used to generate orthophotos: read the external orientation elements of each image, and project it to the ground, obtain the corresponding three-dimensional coordinates, and then perform operations such as orthorectification and image stitching to generate orthophotos. When using the direct ground positioning method, the system continues to initialize image matching, and if the initialization is successful, the visual SLAM method is continued. If the image initialization matching is successful when the system receives input data, the system automatically selects the visual SLAM technology to process the output data: first extract the image feature points, and use the multi-constraint lightweight image matching calculation to match the feature points, obtain the matching point pairs of the two images, and then calculate the homography matrix and estimate the key frame pose.

[0081] S6. Based on the external orientation elements of the image, the pose solution error and cumulative error of the visual SLAM method are corrected to obtain the final orthophoto.

[0082] This embodiment proposes error correction based on image exterior orientation elements to ensure that the pose solution error and cumulative error of the visual SLAM method will not be too large. During the data processing process, the difference between the visual SLAM estimated position and the exterior orientation element position between each frame of the input image is calculated. If it is less than the set threshold, subsequent dense matching and three-dimensional surface reconstruction are performed, and the final orthophoto is obtained through steps such as orthorectification and image stitching; if it is greater than the threshold, the direct ground positioning method is used to generate the orthophoto. The error correction process based on image exterior orientation elements is as follows: Figure 5 As shown, it includes switching between visual SLAM method and direct ground positioning technology.

[0083] The calculation of the difference d and the error threshold T is shown in formula (10) and formula (11):

[0084]

[0085] Among them, (X V ,Y V ,Z v ) is the coordinate position of the camera on the X, Y, and Z axes in three-dimensional space calculated by the SLAM visual odometer, which is calculated by obtaining matching point pairs through multi-constrained lightweight image matching technology; (X G ,Y G ,Z G ) is the coordinate position of the camera on the X, Y, and Z axes obtained by the drone's onboard GNSS system, which is obtained by reading the exterior orientation elements of the image. Δxy is the plane accuracy of the drone itself, and Δz is the vertical accuracy of the drone. Both parameters are given by the drone manufacturer.

[0086] Finally, this embodiment obtains the final orthophoto image through orthorectification, image stitching and other operations.

[0087] Based on the same inventive concept, this embodiment also provides a hybrid direct ground positioning SLAM system for rapid generation of orthophotos. The system is implemented based on the above hybrid direct ground positioning SLAM method and specifically includes the following modules:

[0088] Feature point extraction module, extracts ORB feature points for each image;

[0089] The coarse matching module performs coarse matching on the extracted feature points based on the K nearest neighbor algorithm and Lowe's algorithm;

[0090] The optimization module optimizes the matching point pairs after rough matching based on the RANSAC algorithm;

[0091] The precise matching module eliminates mismatched point pairs based on the Delaunay triangulation algorithm and reprojection constraints, obtains precise matching point pairs, estimates the camera's pose change, and calculates the visual SLAM estimated pose of each frame of image;

[0092] Direct ground positioning module, when the extremely weak texture scene causes the lightweight image matching of visual SLAM multi-constraints to fail, the direct ground positioning technology is used to generate orthophotos;

[0093] The error correction module corrects the pose solution error and cumulative error of the visual SLAM method based on the image exterior orientation elements to obtain the final orthophoto.

[0094] This embodiment proposes a multi-constrained lightweight image matching technology. Combined with the need of this embodiment to quickly generate orthophotos in weak-texture scenes, constraints such as the K nearest neighbor algorithm, Lowe's algorithm, RANSAC algorithm, Delaunay triangulation robust matching, and reprojection error are used to reduce the adverse effects of sparse or repeated textures in the survey area on drone image matching. Not only can the successful matching of feature points be achieved in most scenes, but the lightweight feature ensures the high efficiency of the visual SLAM algorithm, and has good applicability in weak-texture scenes such as snow and oceans, improving the accuracy of visual SLAM pose solution.

[0095] In addition, this embodiment proposes a hybrid direct ground positioning visual SLAM technology. Combined with the technical advantage of the drone platform that can automatically obtain the external orientation elements of the image, the problem of SLAM solution failure caused by extreme textureless scenes is solved, and flexible switching between visual SLAM and direct ground positioning technology is achieved during the orthophoto generation process. At the same time, the external image method element constraints obtained by the drone's onboard POS system reduce the impact of the cumulative error caused by weak texture and the visual SLAM pose solution error, ensuring that high-quality orthophotos can be generated in various scenarios.

[0096] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A hybrid direct-to-ground positioning SLAM method for rapid generation of orthophotos, characterized in that: The following steps are involved: S1, extract ORB feature points for each image; S2, performing rough matching on the extracted feature points based on K nearest neighbor algorithm and Lowe's algorithm; S3, optimizing the matching point pairs after rough matching based on the RANSAC algorithm; S4, based on the Delaunay triangulation algorithm and reprojection constraints, remove the wrong matching point pairs, obtain the precise matching point pairs, estimate the camera's pose change, and calculate the visual SLAM estimated pose of each frame of image; S5. When the extremely weak texture scene causes the lightweight image matching of visual SLAM multi-constraints to fail, the direct ground positioning technology is used to generate orthophotos; S6. Based on the external orientation elements of the image, the pose solution error and cumulative error of the visual SLAM method are corrected to obtain the final orthophoto.

2. The hybrid direct-to-ground positioning SLAM method according to claim 1, characterized in that: When extracting ORB feature points in step S1, the FAST algorithm is used to detect key points, and the BRIEF algorithm is combined to calculate the descriptors of the feature points.

3. The hybrid direct-to-ground positioning SLAM method according to claim 2, characterized in that: In step S2, the KNN algorithm is used to find K=2 most similar feature points in image B for each feature point of image A; Lowe's algorithm is used in combination with the Hamming distance of the BRIEF descriptor to calculate the number of different values ​​at the same position in the binary strings of different candidate points, and further screening is performed using a preset threshold to select a better matching point from the two candidate points matched by KNN.

4. The hybrid direct-to-ground positioning SLAM method according to claim 1, characterized in that: In step S3, the homography matrix of the image is iteratively fitted by the RANSAC algorithm in combination with the flatness of the ground, the homography matrix that achieves the relatively optimal matching effect is solved, and the coarse matching point pairs based on the K nearest neighbor algorithm and Lowe's algorithm are optimized.

5. The hybrid direct-to-ground positioning SLAM method according to claim 4, characterized in that: Step S3 includes: Randomly select 4 matching point pairs from the coarse matching points as the internal points, i.e., the set of observation data, and the other matching points as the external points; then use the internal points to calculate the homography matrix; Through the calculated homography matrix, a model with different numbers of inliers is established; the projection error between other matching points and the model is calculated according to the homography matrix, and a threshold t is set. If the projection error is less than the threshold t, it is considered a new inlier; if it is greater than the threshold t, it is considered an outlier; all the inliers are counted to update the model, the number of iterations k is set, and the update process of the model is iterated repeatedly, and finally the model with the most inliers is obtained as the best model output by the RANSAC algorithm; The pose changes of the image are constrained through the optimal model, so as to retain the correct matching point pairs to the greatest extent and optimize the matching effect.

6. The hybrid direct-to-ground positioning SLAM method according to claim 1, characterized in that: Step S4 uses the Delaunay triangulation to organize initial matching points, local geometric constraints based on spatial angle order, and a matching point expansion step under triangulation constraints to eliminate mismatched point pairs.

7. The hybrid direct-to-ground positioning SLAM method according to claim 6, characterized in that: Step S4 includes: For the ORB feature points of image A after rough matching and matching point pair optimization, a Delaunay triangulation network G1 is constructed, and a matching graph G2 is constructed in image B based on the point pair relationship; Based on the similarity of the feature point arrangement order, the spatial angle order similarity is calculated; For matching point pairs whose spatial angle order similarity scores are lower than the set threshold, the triangle local constraint is used to further expand the matching; Based on the elimination of mismatched points in the Delaunay triangulation algorithm, the reprojection constraint is used to further eliminate mismatched point pairs.

8. The hybrid direct-to-ground positioning SLAM method according to claim 7, characterized in that: When calculating the spatial angle sequence similarity in step S4, a vertex v in the triangulated network G1 is first randomly selected. 1i , the vertex v 1i The connection points are arranged in order according to the polar angle And calculate the vertex v 1i The matching candidate point v in image B 2i Order Next, calculate the vertex v 1i and candidate point v 2i The spatial angle order similarity score is calculated; and the above steps are repeated for the feature points in image A; Spatial angle order similarity score geo The calculation formula for (i) is: Among them, d ced (·) is used to calculate the angle order distance, N represents the vertex v 1i The number of vertices connected in the same triangulation network, j is the number of feature points of image A; the threshold of the spatial angle sequence similarity score is set according to the image dataset used. If the matching point pair score is higher than the threshold of the spatial angle sequence similarity score, it is considered to be a correct matching point pair; When using the triangle local constraint to further expand the matching, find the feature point p in the triangle Δabc formed by a feature point of image A. i , and find the corresponding triangles Δa′b′c′ and p in image B i ′ , if p i ′ If the matching point pairs are still within Δa′b′c′, the matching point pairs that do not meet the above spatial angle order similarity score requirements will be restored to correct matches.

9. The hybrid direct ground positioning SLAM method according to claim 7 or 8, characterized in that: When the reprojection constraint is used in step S4 to continue to eliminate the mismatched point pairs, the target point P captured by the camera in the world coordinate system is mapped to the pixel point P1 on the left image as the first projection. After the drone carries the camera to move, the pixel point of the target point P on the right image is calculated. is the second projection; through feature matching, the matching point P2 of the pixel point P1 on the right image is obtained, and the matching point P2 and the pixel point The difference between them is the reprojection error; the reprojection error is minimized to obtain the optimal camera pose and target point coordinates.

10. A hybrid direct ground positioning SLAM system for rapid generation of orthophotos, implemented based on any hybrid direct ground positioning SLAM method in claims 1-9, comprising the following modules: Feature point extraction module, extracts ORB feature points for each image; The coarse matching module performs coarse matching on the extracted feature points based on the K nearest neighbor algorithm and Lowe's algorithm; The optimization module optimizes the matching point pairs after rough matching based on the RANSAC algorithm; The precise matching module eliminates mismatched point pairs based on the Delaunay triangulation algorithm and reprojection constraints, obtains precise matching point pairs, estimates the camera's pose change, and calculates the visual SLAM estimated pose of each frame of image; Direct ground positioning module, when the extremely weak texture scene causes the lightweight image matching of visual SLAM multi-constraints to fail, the direct ground positioning technology is used to generate orthophotos; The error correction module corrects the pose solution error and cumulative error of the visual SLAM method based on the image exterior orientation elements to obtain the final orthophoto.

Images