Hybrid direct georeferencing slam method and system for fast orthoimage generation

Abstract

The application belongs to the technical field of unmanned aerial vehicle remote sensing images, and relates to a hybrid direct georeferencing SLAM method and system for rapid generation of orthographic images for weak texture scenes, which is applied to weak texture scenes and comprises the following steps: extracting ORB feature points for each image; performing coarse matching on the feature points based on a K nearest neighbor algorithm and Lowe's algorithm; optimizing the matched point pairs after coarse matching based on a RANSAC algorithm; removing the false matched point pairs based on a Delaunay triangulation algorithm and a re-projection constraint to obtain accurate matched point pairs, estimating the change of the camera pose, and calculating the visual SLAM estimated pose of each image; when the visual SLAM multi-constraint lightweight image matching fails due to an extremely weak texture scene, generating an orthographic image by direct georeferencing; correcting the visual SLAM method pose solution error and cumulative error based on image exterior orientation elements to obtain the final orthographic image. The application realizes flexible switching between the visual SLAM and direct georeferencing technologies in the orthographic image generation process.

Description

Technical Field

[0001] This invention belongs to the field of UAV remote sensing image technology, specifically a hybrid direct ground positioning SLAM method and system for rapid generation of orthophotos of weak texture scenes. Background Technology

[0002] Digital orthophoto maps (DOMs) generated from UAV low-altitude aerial photographic data are digital maps that play a crucial role in geographic surveying, land use planning, and environmental monitoring, facilitating accurate measurement, analysis, and understanding of geospatial information. The rapid generation of orthophotos enables the provision of immediate and accurate geographic information, making them suitable for emergency mapping, navigation planning, and environmental change monitoring. This provides critical data support for professionals in these industries, aiding in real-time decision-making. Currently, the main methods for generating orthophotos are photogrammetry, direct geolocation, and SLAM-based methods.

[0003] The core of existing photogrammetric methods for generating orthophotos is to recover camera pose and obtain the 3D coordinates of image points using the SfM (Structure from Motion) algorithm. Therefore, these methods can be divided into two categories: global SfM and incremental SfM. Both global and incremental SfM possess strong robustness, accuracy, and completeness. However, due to the large computational load, photogrammetric methods often require several hours or more to complete the reconstruction of a scene, making them unsuitable for emergency mapping applications with limited computing power.

[0004] Direct ground positioning (DG) for generating orthophotos is a simple process with low computational cost, meeting the efficiency requirements for emergency mapping. Furthermore, since it eliminates the need for image matching, it avoids the problem of insufficient feature points in weak-texture scenes. However, DG requires extremely high accuracy from airborne sensors such as GNSS and IMUs, which are often insufficient for the sensors on consumer-grade drones and low-cost mapping drones. In addition, extreme environments such as forests, islands, and polar regions may present inaccurate POS system measurements, making DG relatively less robust for emergency mapping in extreme environments and thus unsuitable as a primary means of generating orthophotos.

[0005] The method of generating orthorectified images based on visual SLAM benefits from the speed advantage of SLAM technology and has strong timeliness, which can better meet the needs of emergency mapping in terms of speed. However, compared with traditional photogrammetry methods, the method of generating orthorectified images based on SLAM has two problems: (1) Visual SLAM usually selects video streams or images with high overlap and low resolution as input data. However, UAV aerial survey images in the field of mapping usually have low overlap and high resolution, which may lead to failure of visual SLAM pose calculation. (2) For efficiency considerations, the feature point extraction and matching algorithms selected by visual SLAM technology are usually not robust, especially for weak texture scenes. In scenes with weak texture, difficult feature point acquisition or even extreme lack of texture, the method of generating orthorectified images based on SLAM is likely to fail in feature point extraction and feature point pair matching, thus leading to failure of orthorectified image generation. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention proposes a hybrid direct ground-based localization (DGLA) SLAM method and system for rapid generation of orthophotos. It introduces DGLA technology to enable flexible switching between visual SLAM and DGLA technology during orthophoto generation. Simultaneously, it combines image extra-method elements acquired by the POS system to reduce the impact of accumulated errors and visual SLAM pose calculation errors, thereby improving the robustness of the algorithm in rapidly generating high-quality orthophotos in various weak-texture scenes.

[0007] In an embodiment of the present invention, a hybrid direct geolocation SLAM method for rapid generation of orthophotos includes the following steps:

[0008] S1. Extract ORB feature points for each image;

[0009] S2. Perform coarse matching on the extracted feature points based on the K nearest neighbor algorithm and Lowe's algorithm;

[0010] S3. Optimize the matching point pairs after coarse matching based on the RANSAC algorithm;

[0011] S4. Based on the Delaunay triangulation algorithm and reprojection constraints, mismatched point pairs are eliminated to obtain finely matched point pairs. The pose change of the camera is estimated, and the visual SLAM estimated pose of each frame of image is calculated.

[0012] S5. When extreme weak texture scene conditions cause the lightweight image matching of visual SLAM with multiple constraints to fail, direct ground positioning technology is used to generate orthophotos.

[0013] S6. Based on the exterior orientation elements of the image, the pose calculation error and cumulative error of the visual SLAM method are corrected to obtain the final orthophoto.

[0014] Preferably, step S3 uses the RANSAC algorithm, combined with the flatness of the ground, to iteratively fit the homography matrix of the image, calculate the homography matrix that achieves the relatively optimal matching effect, and optimize the coarse matching point pairs based on the K nearest neighbor algorithm and Lowe's algorithm.

[0015] More preferably, step S3 includes:

[0016] Four pairs of matching points are randomly selected from the coarse matching points as interior points, i.e., the set of observed data, and the other matching points are exterior points; then the homography matrix is ​​calculated using the interior points.

[0017] By calculating the homography matrix, models with different numbers of inliers are established. The projection error between other matching points and the model is calculated based on the homography matrix. A threshold t is set. If the projection error is less than the threshold t, it is considered a new inlier; if it is greater than the threshold t, it is considered an outlier. All inliers are counted to update the model. The number of iterations k is set, and the model update process is iterated repeatedly until the model with the most inliers is obtained, which is the best model output by the RANSAC algorithm.

[0018] By constraining the pose changes of the image using the optimal model, the correct matching point pairs are preserved to the greatest extent, thereby optimizing the matching effect.

[0019] An embodiment of the present invention provides a hybrid direct ground positioning SLAM system for rapid generation of orthophotos, which is based on the above-mentioned hybrid direct ground positioning SLAM method and includes the following modules:

[0020] The feature point extraction module extracts ORB feature points for each image;

[0021] The coarse matching module performs coarse matching on the extracted feature points based on the K nearest neighbor algorithm and Lowe's algorithm.

[0022] The optimization module optimizes the matching point pairs after coarse matching based on the RANSAC algorithm.

[0023] The fine matching module, based on the Delaunay triangulation algorithm and reprojection constraints, removes mismatched point pairs to obtain fine matching point pairs, estimates the camera pose change, and calculates the visual SLAM estimated pose for each frame of image.

[0024] The direct ground-based localization module generates orthophotos when lightweight image matching with multiple constraints in visual SLAM fails due to extremely weak texture scenes.

[0025] The error correction module corrects the pose calculation error and cumulative error of the visual SLAM method based on the image exterior orientation elements to obtain the final orthophoto.

[0026] Compared with the prior art, the present invention has the following advantages and effects:

[0027] 1. This invention introduces direct ground positioning technology to enable flexible switching between visual SLAM and direct ground positioning technology during orthophoto generation, thus solving the problem of SLAM solution failure caused by weak texture or extremely textureless scenes.

[0028] 2. The multi-constraint lightweight image matching technique proposed in this invention mitigates the adverse effects of sparse or repetitive textures in the survey area on UAV image matching, improving the applicability and robustness of feature point matching in visual SLAM algorithms. It not only achieves good matching results in various scenarios but also ensures the efficiency of visual SLAM technology, enhancing the applicability and robustness of visual SLAM methods in generating orthophotos in different scenarios.

[0029] 3. This invention improves the switching mechanism between visual SLAM and direct ground positioning technology by applying constraints based on image exterior orientation elements to pose estimation in visual SLAM. Compared to a single visual SLAM approach, this invention can promptly detect pose calculation errors and excessive accumulated errors, enabling timely switching between the visual SLAM method and direct ground positioning technology. It effectively utilizes image exterior orientation elements to improve the completeness and accuracy of the generated orthophotos. Attached Figure Description

[0030] Figure 1 This is a flowchart of the lightweight image matching process with multiple constraints used in the embodiments of the present invention;

[0031] Figure 2 This is a flowchart of the Delaunay triangulation matching process in an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram of the similarity of the Delaunay triangulation of feature points in an embodiment of the present invention, where (a) is image A and (b) is image B;

[0033] Figure 4 This is a flowchart of a hybrid direct ground localization (SLAM) algorithm for rapid generation of orthophotos of weakly textured scenes in an embodiment of the present invention.

[0034] Figure 5 This is a flowchart of error correction based on image exterior orientation elements in an embodiment of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0036] Example

[0037] This embodiment provides a hybrid direct ground-based SLAM method for rapid generation of orthophotos, primarily for the rapid generation of low-altitude remote sensing orthophotos from low-cost, lightweight UAVs, including weak-texture scenes such as snowfields, oceans, forests, and grasslands. To improve the image matching success rate of visual SLAM methods, this embodiment considers the problem of sparse feature points or excessively high similarity in UAV images caused by weak-texture environments. It proposes a lightweight SLAM image matching strategy based on multiple constraints, which performs feature point matching in four stages: feature point extraction, coarse matching, matching point pair optimization, and incorrect matching removal. The specific process is as follows: Figure 1 As shown, the steps corresponding to each stage are described in detail below.

[0038] S1. Extract ORB feature points for each image.

[0039] To meet the requirement of rapid orthophoto generation, this embodiment extracts ORB feature points from each image. ORB feature points are characterized by rapid extraction and high quality, making them suitable for feature point extraction from different types of images, and they also show good feature point extraction results in weak texture environments such as snowfields and forests. When extracting ORB feature points, the FAST (features from accelerated segment test) algorithm is used to detect key points, combined with the BRIEF (Binary Robust Independent Elementary Features) algorithm to calculate the descriptors of the feature points, achieving real-time computation of high-quality feature points.

[0040] S2. Coarse matching of the extracted feature points is performed based on the K-nearest neighbor algorithm and Lowe's algorithm.

[0041] First, to obtain as many matching point pairs as possible in a weakly textured environment and ensure the effectiveness of subsequent feature point matching optimization steps, this step uses the KNN algorithm to match feature points. The KNN algorithm can find K=2 most similar feature points in image B for each feature point in image A. Next, since the BRIEF descriptors of ORB feature points use binary encoding to describe keypoints, to select more similar matching points from the KNN matching candidate points, this embodiment uses Lowe's algorithm. Combining the Hamming distance of the BRIEF descriptors, it calculates the number of different values ​​at the same position in the binary strings of different candidate points, and further filters using a preset threshold of r=0.8, selecting a better matching point from the two candidate points matched by KNN.

[0042] The key steps of Lowe's algorithm can be expressed by formulas (1) and (2).

[0043]

[0044] Wherein, P in formula (1) i P is the description vector of a certain feature point in image A. ′ i D is the description vector of candidate matching points in image B. H It describes the Hamming distance between two points; in formula (2), P1 is a point in image A, and P2 and P3 are two candidate points in image B.

[0045] S3. Optimize the matching point pairs after coarse matching based on the RANSAC algorithm.

[0046] The homography matrix describes the projection from one plane to another. In feature point matching, it can be used to describe the pose transformation of feature points. The RANSAC algorithm optimizes the feature point matching results by calculating the optimal homography matrix between two images. Emergency mapping is often applied to weakly textured scenes such as snowfields and woodlands. These scenes typically have small elevation differences and flat terrain, which facilitates the application of the RANSAC algorithm and the calculation of the homography matrix.

[0047] This step uses the RANSAC algorithm, combined with the assumption of flat ground, to iteratively fit the homography matrix of the image, calculate the homography matrix that can achieve a relatively optimal matching effect, and optimize the coarse matching point pairs based on the K nearest neighbor algorithm and Lowe's algorithm.

[0048] S31. First, in the coarse matching points, randomly select 4 matching point pairs (non-collinear) as interior points, that is, the set of observation data, and the other matching points as exterior points; then use these 4 pairs of interior points to calculate the homography matrix according to formula (3).

[0049]

[0050] Where (x,y) are the pixel coordinates of the feature point on the image, (X... ′ ,Y ′ ) represents the coordinates of the target point in the world coordinate system (assuming the target point is located in the plane Z=0), and s is the scale parameter.

[0051] S32. Using the calculated homography matrix, establish models containing different numbers of interior points; calculate the projection error between other matching points and the model based on the homography matrix, set a threshold t, if the projection error is less than the threshold t, it is considered a new interior point, if it is greater than the threshold t, it is considered an exterior point, i.e., a mismatched point; count all interior points to update the model, set the number of iterations k, and iterate the update process of the model in a loop, finally obtaining the model with the most interior points, which is the best model output by the RANSAC algorithm.

[0052] S33. By using the optimal model within the iteration limit, the pose changes of the image are constrained, thereby preserving the correct matching point pairs to the greatest extent and optimizing the matching effect. After matching point pair optimization by the RANSAC algorithm, coarse matching is performed based on the K nearest neighbor algorithm and Lowe's algorithm.

[0053] S4. Based on the Delaunay triangulation algorithm and reprojection constraints, mismatched point pairs are eliminated to obtain finely matched point pairs. The pose change of the camera is estimated, and the visual SLAM estimated pose of each frame is calculated.

[0054] Delaunay triangulation, as a special graph structure, possesses dynamic generation and good geometric properties. For natural ground feature images taken at orthophoto angles, the Delaunay triangulation structure formed by feature points has certain similarities, which is of great significance for feature point matching.

[0055] This embodiment, based on coarse matching using the K-nearest neighbor algorithm and Lowe's algorithm, and optimization of matching point pairs using the RANSAC algorithm, uses Delaunay triangulation to organize initial matching points, local geometric constraints based on spatial angle order, and matching point expansion steps under triangulation constraints to eliminate mismatched point pairs. The specific process is as follows: Figure 2 As shown.

[0056] S41. For the ORB feature points of image A after coarse matching and matching point pair optimization, construct a Delaunay triangulation G1, and construct a matching map G2 in image B based on the point pair relationship. Since the UAV images used in this embodiment are all taken from an orthophoto angle, the Delaunay triangulations of feature points in adjacent images have a certain similarity, and the sequential arrangement of feature points around the same ground feature is similar, such as (a) and (b).

[0057] Figure 3 As shown.

[0058] S42. Based on the similarity of the arrangement order of feature points, calculate the spatial angular order similarity. First, randomly select a vertex v in the triangulation G1. 1i , will vertex v 1i The connection points are arranged in order according to the size of the polar angle. And calculate vertex v 1i At the matching candidate point v in image B 2i order Next, calculate vertex v. 1i and candidate point v 2i The spatial angular order similarity score is calculated, and the above steps are repeated for feature points in image A. The spatial angular order similarity score is calculated as shown in formula (4):

[0059]

[0060] Where, d ced (·) is used to calculate the angular order distance, where N represents the vertex v 1i The number of vertices connected within the same triangulation, where j is the number of feature points in image A. A threshold for spatial angular order similarity score is set based on the image dataset used; if a matching point pair's score is higher than this threshold, it is considered a correctly matched point pair.

[0061] S43. For matching point pairs whose spatial angular order similarity scores are lower than a set threshold, the matching is further expanded using triangular local constraints.

[0062] Delaunay triangulation divides the image plane into smaller, approximately equilateral triangular patches. Similarly, it benefits from the similarity in the arrangement of feature points in images taken at orthophoto angles, allowing Delaunay triangulation to impose triangular constraints on the positions of feature points in the image. First, it finds the feature point p within the triangle Δabc formed by a certain feature point in image A. i And find the corresponding triangles Δa′b′c′ and p in image B. i ′ If p i ′ If the points are still inside Δa′b′c′, then the matching point pairs that do not meet the above requirements for spatial angular order similarity score will be restored to correct matches.

[0063] S44. Based on the removal of mismatched points using the Delaunay triangulation algorithm, continue to remove mismatched point pairs using reprojection constraints.

[0064] When a camera captures an image, the projection of a point in 3D space onto the image, specifically the mapping of the target point P captured by the camera in the world coordinate system to pixel P1 in the left image, is the first projection. After the drone carrying the camera moves, the camera's pose [R,t] and the coordinates of point P in the world coordinate system can be calculated using methods such as epipolar geometry. Therefore, the pixel value of the target point P in the right image can be calculated. This is the second projection, also known as reprojection. Through feature matching, the matching point P2 of pixel P1 on the right image can be obtained. The matching point P2 and the pixel... The difference between them is the reprojection error. Due to insufficient accuracy of image orientation elements and errors in projection matrix calculation, the reprojection error is usually not zero. Therefore, it is necessary to minimize the reprojection error to obtain the optimal camera pose and target point coordinates. The reprojection error can be written as formula (5):

[0065]

[0066] After reprojection constraints, mismatched point pairs can be further eliminated to obtain the final finely matched point pairs.

[0067] After the four-stage feature point matching process of the multi-constraint lightweight image matching technology in this embodiment, namely "feature point extraction, coarse matching, matching point pair optimization, and erroneous matching removal", high-quality matching point pairs can be obtained with high efficiency in weak texture environments such as snow and woodland.

[0068] S5. When extreme weak texture scene conditions cause the lightweight image matching of visual SLAM with multiple constraints to fail, direct ground positioning technology is used to generate orthophotos.

[0069] A preferred implementation method for direct ground positioning technology is as follows:

[0070] Direct ground positioning technology can obtain the world coordinates of ground points through the point projection coefficient method. Specifically, the coordinates (X,Y,Z) of any target point in the world coordinate system can be expressed by the formula set (6):

[0071]

[0072] Where (X1, Y1, Z1) are the coordinates of the target point in the auxiliary coordinate system of the image space. The left and right images represent the camera's positions on the X, Y, and Z axes, respectively, i.e., the exterior orientation elements; N1 and N2 are the point projection coefficients of the left and right image points, calculated as follows:

[0073]

[0074] In formula group (7), B X B YB Z The three baseline components of the epipolar line in the auxiliary coordinate system of image space can be calculated using formula (8):

[0075]

[0076] The coordinates of the target point in the auxiliary coordinate system of the image space can be represented by the coordinates (x1, y1) and (x2, y2) in the pixel coordinate systems of the left and right images as follows:

[0077]

[0078] Where R1 and R2 are the orthogonal transformation matrices composed of the exterior azimuth elements of the left and right images, respectively, and f is the camera principal distance. Therefore, the three-dimensional coordinates of each image point on the image in the world coordinate system can be calculated.

[0079] While multi-constraint lightweight image matching techniques can improve feature point matching in most scenarios, some extreme textureless scenes can easily cause image matching failures in visual SLAM methods; or weakly textured scenes can lead to large pose and cumulative errors in visual SLAM solutions. To address these issues, this embodiment leverages the ability of current UAV platforms' POS systems to acquire exterior orientation elements of images. Based on an improved visual SLAM image matching method, it incorporates direct ground positioning technology to ensure high-quality orthophotos are still acquired even in cases of image matching failure. The specific process is as follows: Figure 4 As shown.

[0080] First, the required data for this embodiment is input, including UAV aerial images and camera interior / exterior orientation elements. Then, a multi-constraint lightweight image matching technique is used for rapid visual SLAM image matching. If extreme weak-texture scenes cause the multi-constraint lightweight image matching of visual SLAM to fail, direct ground-based localization is used to generate orthorectified images: the exterior orientation elements of each image are read and projected onto the ground to obtain the corresponding 3D coordinates, followed by orthorectification and image stitching to generate an orthorectified image. While using the direct ground-based localization method, the system continuously initializes image matching; if initialization is successful, the visual SLAM method continues. If image initialization matching is successful when the system receives input data, the system automatically selects visual SLAM to process the output data: first, image feature points are extracted, and multi-constraint lightweight image matching is used to calculate feature point matching. After obtaining matching point pairs between two images, the homography matrix is ​​calculated and the keyframe pose is estimated.

[0081] S6. Based on the exterior orientation elements of the image, the pose calculation error and cumulative error of the visual SLAM method are corrected to obtain the final orthophoto.

[0082] This embodiment proposes an error correction method based on image exterior orientation elements to ensure that the pose calculation error and cumulative error of the visual SLAM method are not too large. During data processing, the difference between the visual SLAM estimated position and the exterior orientation element position between each input image frame is maintained. If it is less than a set threshold, subsequent dense matching and 3D surface reconstruction are performed, and the final orthorectified image is obtained through orthorectification and image stitching. If it is greater than the threshold, the orthorectified image is generated using a direct ground positioning method. The error correction process based on image exterior orientation elements is as follows: Figure 5 As shown, this includes the switching between visual SLAM methods and direct ground positioning technology.

[0083] The difference d and the error threshold T are calculated as shown in formulas (10) and (11):

[0084]

[0085] Among them, (X) V ,Y V Z v (X) represents the camera's coordinates on the X, Y, and Z axes in 3D space, calculated using SLAM visual odometry. This coordinates are derived after obtaining matching point pairs through a multi-constraint lightweight image matching technique. G ,Y G Z G Δxy represents the camera's coordinates on the X, Y, and Z axes, obtained by the UAV's onboard GNSS system, derived from the exterior orientation elements of the imagery. Δxy represents the UAV's planar accuracy, and Δz represents its vertical accuracy; both parameters are provided by the UAV manufacturer.

[0086] Finally, this embodiment obtains the final orthophoto image through orthorectification, image stitching, and other processes.

[0087] Based on the same inventive concept, this embodiment also provides a hybrid direct ground-based SLAM system for rapid generation of orthophotos. This system is implemented based on the aforementioned hybrid direct ground-based SLAM method and specifically includes the following modules:

[0088] The feature point extraction module extracts ORB feature points for each image;

[0089] The coarse matching module performs coarse matching on the extracted feature points based on the K nearest neighbor algorithm and Lowe's algorithm.

[0090] The optimization module optimizes the matching point pairs after coarse matching based on the RANSAC algorithm.

[0091] The fine matching module, based on the Delaunay triangulation algorithm and reprojection constraints, removes mismatched point pairs to obtain fine matching point pairs, estimates the camera pose change, and calculates the visual SLAM estimated pose for each frame of image.

[0092] The direct ground-based localization module generates orthophotos when lightweight image matching with multiple constraints in visual SLAM fails due to extremely weak texture scenes.

[0093] The error correction module corrects the pose calculation error and cumulative error of the visual SLAM method based on the image exterior orientation elements to obtain the final orthophoto.

[0094] This embodiment proposes a lightweight image matching technique with multiple constraints. Considering the need for rapid orthorectified image generation in weakly textured scenes, this embodiment employs constraints such as the K-nearest neighbor algorithm, Lowe's algorithm, RANSAC algorithm, Delaunay triangulation robust matching, and reprojection error to mitigate the adverse effects of sparse or repetitive textures in the survey area on UAV image matching. This technique not only achieves successful feature point matching in most scenarios but also ensures the high efficiency of the visual SLAM algorithm due to its lightweight nature. It exhibits good applicability in weakly textured scenes such as snow and ocean, improving the accuracy of visual SLAM pose calculation.

[0095] Furthermore, this embodiment proposes a hybrid visual SLAM technique using direct ground-based localization. Leveraging the advantage of the UAV platform's ability to automatically acquire image exterior orientation elements, it addresses the SLAM solution failure issue caused by extreme textureless scenes by enabling flexible switching between visual SLAM and direct ground-based localization techniques during orthophoto generation. Simultaneously, the image exterior method element constraints acquired by the UAV's onboard POS system reduce the impact of cumulative errors caused by weak textures and visual SLAM pose calculation errors, ensuring the generation of high-quality orthophotos in various scenarios.

[0096] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A hybrid direct geolocation SLAM method for rapid generation of orthophotos, characterized in that, The proposed method proposes a lightweight SLAM image matching strategy based on multiple constraints, which performs feature point matching in four stages: feature point extraction, coarse matching, matching point pair optimization, and incorrect matching removal. The method includes the following steps: S1. Extract ORB feature points for each image; S2. Perform coarse matching on the extracted feature points based on the K nearest neighbor algorithm and Lowe's algorithm; S3. Optimize the matching point pairs after coarse matching based on the RANSAC algorithm; S4. Based on the Delaunay triangulation algorithm and reprojection constraints, mismatched point pairs are eliminated to obtain finely matched point pairs. The pose change of the camera is estimated, and the visual SLAM estimated pose of each frame of image is calculated. S5. When extreme weak texture scene conditions cause the lightweight image matching of visual SLAM with multiple constraints to fail, direct ground positioning technology is used to generate orthophotos. S6. Based on the exterior orientation elements of the image, the pose calculation error and cumulative error of the visual SLAM method are corrected to obtain the final orthophoto. Step S4 uses Delaunay triangulation to organize initial matching points, local geometric constraints based on spatial angle order, and matching point expansion steps under triangulation constraints to eliminate mismatched point pairs; Step S4 includes: For the ORB feature points of image A after coarse matching and matching point pair optimization, construct a Delaunay triangulation G1, and construct a matching map G2 in image B according to the point pair relationship. Based on the similarity of the arrangement order of feature points, the spatial angular order similarity is calculated; For matching point pairs whose spatial angular order similarity scores are below a set threshold, the matching is further expanded using triangular local constraints. Based on the removal of mismatched points using the Delaunay triangulation algorithm, reprojection constraints are used to further remove mismatched point pairs. In step S4, when using reprojection constraints to continue eliminating mismatched point pairs, the pixel P1 mapped from the target point P captured by the camera in the world coordinate system to the left image is the first projection. After the UAV carrying the camera moves, the pixel P1 of the target point P in the right image is calculated. For the second projection; through feature matching, the matching point P2 of pixel P1 on the right image is obtained, and the matching point P2 and pixel P1 are... The difference between them is the reprojection error; minimizing the reprojection error is to obtain the optimal camera pose and target point coordinates.

2. The hybrid direct ground positioning SLAM method according to claim 1, characterized in that, In step S1, when extracting ORB feature points, the FAST algorithm is used to detect key points, and the BRIEF algorithm is used to calculate the descriptors of the feature points.

3. The hybrid direct ground positioning SLAM method according to claim 2, characterized in that, Step S2 uses the K nearest neighbor algorithm to find K=2 most similar feature points in image B for each feature point in image A; Lowe's algorithm is used, combined with the Hamming distance of the BRIEF descriptor, to calculate the number of different values ​​at the same position in the binary strings of different candidate points, and further filter them with a preset threshold to select the better matching point from the two candidate points matched by KNN.

4. The hybrid direct ground positioning SLAM method according to claim 1, characterized in that, Step S3 uses the RANSAC algorithm, combined with the flatness of the ground, to iteratively fit the homography matrix of the image, calculate the homography matrix that achieves the relatively optimal matching effect, and optimize the coarse matching point pairs based on the K nearest neighbor algorithm and Lowe's algorithm.

5. The hybrid direct ground positioning SLAM method according to claim 4, characterized in that, Step S3 includes: Four pairs of matching points are randomly selected from the coarse matching points as interior points, i.e., the set of observed data, and the other matching points are exterior points; then the homography matrix is ​​calculated using the interior points. By calculating the homography matrix, models with different numbers of inliers are established. The projection error between other matching points and the model is calculated based on the homography matrix. A threshold t is set. If the projection error is less than the threshold t, it is considered a new inlier; if it is greater than the threshold t, it is considered an outlier. All inliers are counted to update the model. The number of iterations k is set, and the model update process is iterated repeatedly until the model with the most inliers is obtained, which is the best model output by the RANSAC algorithm. By constraining the pose changes of the image using the optimal model, the correct matching point pairs are preserved to the greatest extent, thereby optimizing the matching effect.

6. The hybrid direct ground positioning SLAM method according to claim 1, characterized in that, In step S4, when calculating the spatial angular order similarity, a vertex v in the triangulation G1 is first randomly selected. 1i , will vertex v 1i The connection points are arranged in order according to the size of the polar angle. And calculate vertex v 1i At the matching candidate point v in image B 2i order Next, calculate vertex v. 1i and candidate point v 2i The spatial angular order similarity score is calculated; and the above steps are repeated for feature points in image A. Spatial angular order similarity score geo The formula for calculating (i) is: Where, d ced (·) is used to calculate the angular order distance, where N represents the vertex v 1i The number of vertices connected within the same triangulation, where j is the number of feature points in image A; a threshold for spatial angular order similarity score is set based on the image dataset used. If the score of a matching point pair is higher than the threshold for spatial angular order similarity score, it is considered a correct matching point pair. When further expanding the matching using the triangle local constraint, find the feature point p in the triangle Δabc formed by a certain feature point in image A. i And find the corresponding triangles Δa′b′c′ and p in image B. i ′ If p i ′ If the points are still inside Δa′b′c′, then the matching point pairs that do not meet the above requirements for spatial angular order similarity score will be restored to correct matches.

7. A hybrid direct geolocation SLAM system for rapid generation of orthophotos, implemented based on any one of the hybrid direct geolocation SLAM methods of claims 1-6, comprising the following modules: The feature point extraction module extracts ORB feature points for each image; The coarse matching module performs coarse matching on the extracted feature points based on the K nearest neighbor algorithm and Lowe's algorithm. The optimization module optimizes the matching point pairs after coarse matching based on the RANSAC algorithm. The fine matching module, based on the Delaunay triangulation algorithm and reprojection constraints, removes mismatched point pairs to obtain fine matching point pairs, estimates the camera pose change, and calculates the visual SLAM estimated pose for each frame of image. The direct ground-based localization module generates orthophotos when lightweight image matching with multiple constraints in visual SLAM fails due to extremely weak texture scenes. The error correction module corrects the pose calculation error and cumulative error of the visual SLAM method based on the image exterior orientation elements to obtain the final orthophoto.

Images