An adaptive ORB visual front-end feature extraction method, device, equipment, medium and product

Abstract

The application discloses an adaptive ORB visual front-end feature extraction method, device, equipment, medium and product, relates to the field of integrated navigation, and comprises the following steps: carrying out grid division on the gray value image, determining the gray standard deviation of each grid area and the average value of the gray standard deviation of each grid area in the overall image, and determining the adaptive FAST detection threshold value of each grid area; in each grid area, corner point detection is carried out based on the adaptive FAST detection threshold value, and the normalized corner point response value of a candidate feature point is obtained; in each candidate feature point in each grid area, the spatial uniformity contribution of each candidate feature point is determined based on a feature distribution dispersion degree and a spatial homogeneity index; and the candidate feature points are comprehensively screened according to the normalized corner point response value and the uniformity contribution, so that the target feature points are obtained. The application improves the stability of feature point extraction under complex motion and complex environment.

Description

Technical Field

[0001] This application relates to the field of integrated navigation, and in particular to an adaptive ORB visual front-end feature extraction method, apparatus, device, medium and product. Background Technology

[0002] Visual navigation is a crucial localization technology in mobile robots, unmanned platforms, intelligent vehicles, and augmented reality. Its basic idea is to continuously acquire environmental images using a camera and reconstruct the motion state of the camera or platform by matching feature points in the image sequence. In visual localization systems, the visual front-end is the foundation of the entire system. The number of feature points output by the front-end, their spatial distribution, matching accuracy, and the stability of continuous frame tracking directly affect the subsequent essential matrix estimation, triangulation, PnP (Perspective-n-Point) pose solving, and local BA (Bundle Adjustment) optimization results. If the quality of the front-end features is poor, even with complex back-end optimization methods, it will be difficult to obtain stable and reliable trajectory estimation results.

[0003] The ORB (Oriented Fast and Rotated Brief) algorithm is based on FAST (Features from Accelerated Segment Test) corner points and BRIEF (Binary Robust Independent Elementary Features) descriptors. It has advantages such as fast computation speed, high binary descriptor matching efficiency, and convenient engineering deployment. Therefore, it is widely used in real-time visual odometry and visual SLAM systems.

[0004] In integrated navigation applications, mobile platforms frequently experience motion states such as startup, acceleration, deceleration, turning, jitter, and rapid rotation, resulting in significant changes in viewpoint between image frames and a high risk of feature point loss or mismatch. Simultaneously, the backend needs to process IMU pre-integration constraints and raw observations such as GNSS pseudorange and Doppler, placing high pressure on the system's real-time performance. Therefore, the visual frontend cannot employ overly complex feature algorithms such as SIFT, yet it must maintain sufficiently stable feature quality under complex motion and environments. Summary of the Invention

[0005] The purpose of this application is to provide an adaptive ORB visual front-end feature extraction method, apparatus, device, medium, and product, which can improve the stability of feature point extraction in complex motion and complex environments.

[0006] To achieve the above objectives, this application provides the following solution: Firstly, this application provides an adaptive ORB visual front-end feature extraction method, including: Obtain the grayscale image of the image to be extracted; The grayscale image is divided into grids, and the grayscale standard deviation of each grid region and the average grayscale standard deviation of each grid region in the overall image are determined. The adaptive FAST detection threshold for each grid region is determined based on the gray standard deviation of each grid region and the average gray standard deviation of each grid region in the overall image. Within each grid region, corner detection is performed based on the adaptive FAST detection threshold to obtain the normalized corner response value of the candidate feature point; For each candidate feature point within each grid region, the spatial uniformity contribution of each candidate feature point is determined based on the feature distribution dispersion and spatial homogeneity indices. The candidate feature points are comprehensively screened based on the normalized corner response value and the uniformity contribution to obtain the target feature points.

[0007] In one example, obtaining the grayscale image of the image to be extracted specifically includes: Acquire consecutive image frames and convert the consecutive image frames into grayscale images; The grayscale image is subjected to distortion correction using camera calibration parameters to obtain the grayscale image of the image to be extracted.

[0008] In one example, the expression for the adaptive FAST detection threshold is: ; in, For the first Adaptive FAST detection threshold for each grid region Based on the FAST detection threshold, This is the grayscale standard deviation adjustment coefficient. and These are the lower and upper limits of the detection threshold, respectively. This represents the truncation function. The standard deviation of gray level for each grid region, This represents the average of the standard deviations of gray levels in each grid region of the overall image.

[0009] In one example, for each candidate feature point within each grid region, the spatial uniformity contribution of each candidate feature point is determined based on the feature distribution dispersion and spatial homogeneity indices, specifically including: Within each grid region, the feature distribution dispersion is determined based on the number of selected feature points and the average number of feature points; Based on the feature distribution dispersion utilization Determine the spatial uniformity index; Based on the spatial uniformity index, utilize Determine the spatial uniformity contribution of each candidate feature point; in, As an index of spatial uniformity, The characteristic distribution dispersion, Contribute to the spatial uniformity of each candidate feature point. To add candidate feature points Spatial uniformity index after.

[0010] In one example, candidate feature points are comprehensively screened based on the normalized corner response value and the uniformity contribution to obtain target feature points, specifically including: The comprehensive score for each candidate feature point is determined based on the normalized corner response value and the uniformity contribution. Candidate feature points are sorted from high to low based on the comprehensive score; Select target feature points according to the sorting until the target feature point quantity is reached or the sorting is empty.

[0011] In one example, the adaptive ORB visual front-end feature extraction method further includes: Generate an ORB descriptor based on the target feature points; Based on the ORB descriptor, feature matching and basic mismatch removal are performed to obtain matching points; the matching points are used for essential matrix estimation, triangulation, PnP pose solving or local BA optimization.

[0012] Secondly, this application provides an adaptive ORB visual front-end feature extraction device, comprising: The acquisition module is used to acquire the grayscale image of the image to be extracted; The grid division module is used to divide the grayscale image into grids, determine the grayscale standard deviation of each grid region and the average grayscale standard deviation of each grid region in the overall image; An adaptive FAST detection threshold determination module is used to determine the adaptive FAST detection threshold for each grid region based on the gray standard deviation of each grid region and the average gray standard deviation of each grid region in the overall image. The corner detection module is used to perform corner detection based on the adaptive FAST detection threshold in each grid area to obtain the normalized corner response value of the candidate feature point; The spatial uniformity contribution determination module is used to determine the spatial uniformity contribution of each candidate feature point within each grid region based on the feature distribution dispersion and spatial homogeneity index. The comprehensive filtering module is used to comprehensively filter candidate feature points based on the normalized corner response value and the uniformity contribution to obtain target feature points.

[0013] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the adaptive ORB visual front-end feature extraction method.

[0014] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the adaptive ORB visual front-end feature extraction method.

[0015] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the adaptive ORB visual front-end feature extraction method.

[0016] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides an adaptive ORB visual front-end feature extraction method, apparatus, device, medium, and product. It determines the adaptive FAST detection threshold for each grid region by using the grayscale standard deviation of each grid region and the average grayscale standard deviation of each grid region in the overall image. Corner detection is performed using the adaptive FAST detection threshold. The method can automatically adjust the number of candidate feature points based on regional texture or grayscale changes. During comprehensive screening, a spatial uniformity contribution determined based on feature distribution dispersion and spatial homogeneity indices is introduced, thus balancing feature quality and spatial distribution, thereby improving the stability of feature point extraction. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating an adaptive ORB visual front-end feature extraction method provided in an embodiment of this application; Figure 2 A schematic diagram of the FAST corner detection principle provided in one embodiment of this application; Figure 3 A visualization of the distribution and matching results of visual front-end features is provided in one embodiment of this application. Figure 4A schematic diagram of the functional modules of an adaptive ORB visual front-end feature extraction device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] The standard ORB method still has shortcomings in complex integrated navigation scenarios. First, standard FAST corner detection typically uses a fixed threshold, determining whether a point is a corner by comparing the grayscale difference between the center pixel and its surrounding pixels. A fixed threshold can lead to a sharp decrease in the number of feature points in scenes with weak texture, shadows, local overexposure, underexposure, and motion blur; conversely, it can generate a large number of redundant corner points in areas with dense texture or high noise. Second, feature points extracted by standard ORB tend to be concentrated in texture-rich local areas of the image, such as building edges, tree branches, road markings, or vehicle outlines, while they are less distributed in areas with weak texture. This uneven distribution of feature points leads to insufficient geometric constraints on the image plane, causing subsequent essential matrix estimation, PnP pose solving, and local BA optimization to over-rely on local region features, thus reducing the stability of motion estimation. Third, standard ORB feature matching mainly relies on Hamming distance nearest neighbor matching. When there are repeated textures, dynamic targets, occlusions, or locally similar structures in the scene, incorrect matches are easily generated. If incorrect matches are not removed in time, they will directly contaminate the triangulation, PnP, and reprojection error optimization processes.

[0021] In integrated navigation applications, mobile platforms frequently experience motion states such as startup, acceleration, deceleration, turning, jitter, and rapid rotation, resulting in significant changes in viewpoint between image frames and a high risk of feature point loss or mismatch. Simultaneously, the backend needs to process IMU pre-integration constraints and raw observations such as GNSS pseudorange and Doppler, placing high pressure on system real-time performance. Therefore, the visual frontend cannot employ overly complex feature algorithms like SIFT, yet it must maintain sufficiently stable feature quality under complex motion and environments. Based on this, this application, using the standard ORB as a foundation, introduces a local grayscale statistics-driven adaptive threshold adjustment mechanism, an image partitioning homogenization feature selection mechanism, an improved neighborhood corner response model, a multi-level mismatch elimination mechanism, and a backend-optimized visual observation quality output mechanism. This method can improve feature point detection stability, spatial distribution uniformity, and inlier matching rate in various scenarios, providing the system with more reliable visual constraint input.

[0022] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] In one exemplary embodiment, such as Figure 1 As shown, an adaptive ORB visual front-end feature extraction method is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, it includes the following steps.

[0024] Step 101: Obtain the grayscale image of the image to be extracted.

[0025] Step 102: Divide the grayscale image into grids and determine the grayscale standard deviation of each grid region and the average grayscale standard deviation of each grid region in the overall image.

[0026] Step 103: Determine the adaptive FAST detection threshold for each grid region based on the grayscale standard deviation of each grid region and the average grayscale standard deviation of each grid region in the overall image.

[0027] Step 104: Within each grid area, corner detection is performed based on the adaptive FAST detection threshold to obtain the normalized corner response value of the candidate feature point.

[0028] Step 105: For each candidate feature point within each grid region, determine the spatial uniformity contribution of each candidate feature point based on the feature distribution dispersion and spatial homogeneity indices.

[0029] Step 106: Based on the normalized corner response value and the uniformity contribution, the candidate feature points are comprehensively screened to obtain the target feature points.

[0030] In an exemplary embodiment, obtaining the grayscale image of the image to be extracted specifically includes: acquiring consecutive image frames and converting the consecutive image frames into grayscale images; performing distortion correction processing on the grayscale images using camera calibration parameters to obtain the grayscale image of the image to be extracted.

[0031] In one exemplary embodiment, an adaptive FAST threshold adjustment mechanism based on local grayscale statistics is provided.

[0032] Standard FAST corner detection typically uses a fixed grayscale threshold to determine whether the grayscale difference between the center pixel and its neighboring pixels meets the corner detection criteria. While a fixed threshold performs well in scenes with stable lighting and clear textures, it can easily lead to insufficient feature points or false detections of noise points in scenes with weak textures, local overexposure, shadows, underexposure, or motion blur. To improve the adaptability of the ORB front-end under complex image conditions, this application proposes an adaptive FAST threshold adjustment mechanism based on the local grayscale standard deviation.

[0033] First, the input image is divided into several grid regions. Let the first region be... The number of pixels in each grid region is The set of feature points is For feature points Its pixel grayscale value The grayscale mean is Then the gray standard deviation of this region for: .

[0034] It represents the degree of dispersion of grayscale values ​​within that region. If A larger value indicates a more pronounced variation in light and shadow in the area, typically corresponding to areas with richer textures or more defined edge structures; if A smaller value indicates that the grayscale variation in this area is weak, and it may belong to a weak texture, blurry, or low-contrast area.

[0035] Let the average of the standard deviations of gray levels in each grid region of the entire image be... for: .

[0036] in, The total number of grid cells obtained by dividing the image. Based on the difference between the local gray-level standard deviation and the average gray-level standard deviation of the entire image, the number of grid cells can be obtained. Adaptive FAST detection threshold for each grid region.

[0037] The expression for the adaptive FAST detection threshold is: .

[0038] in, For the first Adaptive FAST detection threshold for each grid region Based on the FAST detection threshold, This is the grayscale standard deviation adjustment coefficient. and These are the lower and upper limits of the detection threshold, respectively. This represents the truncation function. The standard deviation of gray level for each grid region, This represents the average of the standard deviations of gray levels in each grid region of the overall image.

[0039] When the standard deviation of grayscale in a certain area is lower than the average level of the entire map, that is... This indicates that the texture in this area is weak or the grayscale variation is insufficient. Will be below the base threshold This lowers the corner detection threshold and increases the number of candidate feature points in weakly textured regions. Conversely, it indicates that the region has rich texture or strong local grayscale changes. This improves and suppresses redundant corner points and noise response in texture-dense regions.

[0040] In an exemplary embodiment, for each candidate feature point within each grid region, the spatial uniformity contribution of each candidate feature point is determined based on the feature distribution dispersion and spatial homogeneity indices. Specifically, this includes: within each grid region, determining the feature distribution dispersion based on the number of selected feature points and the average number of feature points; and utilizing the feature distribution dispersion... Determine the spatial uniformity index; utilize the spatial uniformity index... Determine the spatial uniformity contribution of each candidate feature point; where, As an index of spatial uniformity, The characteristic distribution dispersion, Contribute to the spatial uniformity of each candidate feature point. To add candidate feature points Spatial uniformity index after.

[0041] In practical applications, a partitioned homogenization feature selection mechanism for pose estimation stability is provided.

[0042] The standard ORB method typically relies primarily on corner response values ​​to preserve feature points, which can easily lead to feature concentration in textured regions while fewer features are present in less textured regions. For visual pose estimation, feature points not only need to be numerous but also evenly distributed across the image plane. If a large number of features are concentrated in local areas, even with sufficient feature quantity and disparity, motion estimation performance may degrade.

[0043] Therefore, this application introduces a spatial uniformity index in the candidate feature point selection stage. The image is divided into... Let the nth grid be the nth grid. The number of selected feature points within each grid is The average number of feature points is Define the discreteness of the feature distribution for: .

[0044] Further define the spatial uniformity index : .

[0045] in, The smaller The larger the value, the more uniform the distribution of feature points. For candidate feature points... If this point is added, the uniformity index will change from Become Then its spatial contribution for: .

[0046] In an exemplary embodiment, candidate feature points are comprehensively screened based on the normalized corner response value and the uniformity contribution to obtain target feature points. Specifically, this includes: determining a comprehensive score for each candidate feature point based on the normalized corner response value and the uniformity contribution; sorting the candidate feature points from high to low according to the comprehensive score; and selecting target feature points according to the sorting until the target number of feature points is reached or the sorting is empty.

[0047] This application uses corner response, spatial distance, and uniformity contribution together for candidate point selection: .

[0048] in, This is the normalized corner response value. These are the weighting coefficients. For comprehensive scoring, the system prioritizes retaining [reviews / reviews]. A higher number of feature points, thus balancing feature quality and spatial distribution.

[0049] In an exemplary embodiment, the adaptive ORB visual front-end feature extraction method further includes: generating an ORB descriptor based on the target feature points; performing feature matching and basic mismatch removal based on the ORB descriptor to obtain matching points; the matching points are used for essential matrix estimation, triangulation, PnP pose solving, or local BA optimization.

[0050] This application addresses the problems of large fluctuations in the number of features, uneven feature distribution, high mismatch rate, and insufficient front-end quality information to serve back-end optimization in complex scenarios using standard ORB. In another exemplary embodiment, a visual front-end processing method is proposed, which includes adaptive threshold adjustment, partition uniformity screening, improved neighborhood detection, multi-level matching error elimination, and back-end quality output.

[0051] (a) Acquire images and perform basic preprocessing.

[0052] The camera acquires continuous image frames. The input image is converted into a grayscale image, and distortion correction is performed on the image according to the camera calibration parameters to ensure that subsequent corner detection and feature matching are performed in a unified image coordinate system.

[0053] (ii) Calculate the local gray standard deviation of each grid region.

[0054] Divide the image into For each grid region, calculate its mean gray value and standard deviation, and calculate the average of the standard deviations of gray values ​​of all grid regions in the entire image as a global reference for judging the strength of texture in local areas.

[0055] Specifically: First, the input image is divided into several grid regions. Let the first region be... The number of pixels in each grid region is pixel grayscale value The grayscale mean is Then the standard deviation of gray levels in this region is: .

[0056] It represents the degree of dispersion of grayscale values ​​within that region. If A larger value indicates a more pronounced variation in light and shadow in the area, typically corresponding to areas with richer textures or more defined edge structures; if A smaller value indicates that the grayscale variation in this area is weak, and it may belong to a weak texture, blurry, or low-contrast area.

[0057] Let the average standard deviation of gray levels in each grid region of the entire image be: .

[0058] in, The total number of grids obtained by dividing the image.

[0059] (iii) Determine the adaptive FAST threshold for each grid region.

[0060] Based on the difference between the local grayscale standard deviation of each grid region and the average grayscale standard deviation of the entire image, the FAST detection threshold corresponding to that grid region is calculated according to the aforementioned adaptive threshold formula. In specific implementation, for an 8-bit grayscale image, the basic threshold... The threshold value can be 18 to 22; lower limit of the threshold. A value of 8 to 10 is acceptable; upper limit of the threshold. A value of 35-40 can be used; grayscale standard deviation adjustment coefficient A value of 0.20 to 0.35 is acceptable.

[0061] Specifically: Based on the difference between the local grayscale standard deviation and the overall image average grayscale standard deviation, the first... Adaptive FAST detection threshold for each grid region: .

[0062] in, Based on the FAST detection threshold, This is the grayscale standard deviation adjustment coefficient. and These are the lower and upper limits of the detection threshold, respectively. This represents a truncation function used to prevent the threshold from being too low or too high.

[0063] When the standard deviation of grayscale in a certain area is lower than the average level of the entire map, that is... This indicates that the texture in this area is weak or the grayscale variation is insufficient. Will be below the base threshold This lowers the corner detection threshold and increases the number of candidate feature points in weakly textured regions. Conversely, it indicates that the region has rich texture or strong local grayscale changes. This improves and suppresses redundant corner points and noise response in texture-dense regions.

[0064] (iv) Calculate the corner response values ​​of the candidate feature points.

[0065] like Figure 2 As shown, corner detection is performed within each grid area using the adaptive FAST threshold corresponding to that area, and the corner response value is recorded.

[0066] For the pixel to be detected, construct a Bresenham discrete circle with a radius of 3 pixels centered on it, and select 16 neighboring pixels p1...p16 on the circumference. Let the pixel grayscale value of the center point be... The adaptive FAST detection threshold of its grid is First, select several diagonal pixels on the circumference for rapid pre-screening. If the absolute value of the grayscale difference between these sampling points and the center point is less than... If the grayscale change at a point is not significant, it is considered to be excluded directly; otherwise, it is considered as a candidate feature point for further detection.

[0067] For the pre-selected candidate points, further traverse all 16 neighboring pixels on their Bresenham circle. If there are at least 9 consecutive pixels whose absolute difference from the center point's grayscale value is greater than 1, then... If so, then the center point p is determined to be a corner point.

[0068] To suppress repetitive or overly dense corner responses in adjacent regions, a non-maximum suppression strategy is further introduced. The corner response value is defined. It is the sum of the absolute values ​​of the grayscale differences between the 16 neighboring pixels on the circumference and the pixel at the center point: .

[0069] Let be the pixel grayscale value of the i-th neighboring pixel around pixel p. Only when the response value s of a candidate feature point reaches the maximum value in its local neighborhood will it be retained as the final feature point.

[0070] After the initial detection is completed, non-maximum suppression is applied to the candidate corner points to retain the candidate points with strong responses in the local neighborhood and remove duplicate response points in adjacent positions.

[0071] (v) Calculate the spatial contribution of candidate feature points.

[0072] For candidate feature points, calculate the spatial uniformity index. Obtain the contribution of each point to improving overall uniformity. (This is parallel to the previous step, and the final feature points are calculated together according to steps (iv) and (v).)

[0073] Specifically, the image is divided into Let the nth grid be the nth grid. The number of selected feature points within each grid is The average number of feature points is The feature distribution dispersion is defined as: .

[0074] Further define the spatial uniformity index: .

[0075] in, The smaller The larger the value, the more uniform the distribution of feature points. For candidate feature points... If this point is added, the uniformity index will change from Become Then its spatial contribution is: .

[0076] (vi) Select target feature points based on comprehensive scores.

[0077] The system calculates a comprehensive score for each candidate feature point based on its normalized corner response value and spatial uniformity contribution. The system sorts the candidate feature points from highest to lowest score and sequentially selects higher-scoring candidate points until the target number of feature points is reached or the candidate point set is empty.

[0078] Specifically, this application uses corner response, spatial distance, and uniformity contribution together for candidate point selection: .

[0079] in, This is the normalized corner response value. These are weighting coefficients. The system prioritizes retaining them. A higher number of feature points, thus balancing feature quality and spatial distribution.

[0080] (vii) Generate ORB descriptors.

[0081] For each feature point in the final set of target feature points retained by the comprehensive scoring and screening system, its orientation and binary descriptor are calculated according to the conventional ORB descriptor generation process.

[0082] Specifically, firstly, the gray-level moment is calculated within the image patch of the feature point's neighborhood to obtain the gray-level centroid. The direction of the line connecting the feature point's center and the gray-level centroid is taken as the principal direction of that feature point to enhance the descriptor's rotation adaptability. Subsequently, the BRIEF sampling template is rotated according to this principal direction. Several pixel pairs are selected from the rotated sampling template, and each pair of pixels is compared. , The grayscale relationship is used to obtain the binary test results. : .

[0083] Encode this result into a binary sequence to obtain the ORB descriptor corresponding to the feature point.

[0084] (viii) Perform feature matching and basic mismatch elimination.

[0085] For feature points in two images, their similarity can be represented by Hamming distance. x and y are ORB descriptors, which are n-bit binary strings. , Let x represent the value of its i-th binary bit, then the Hamming distance between x and y is... It can be represented as: .

[0086] The smaller the Hamming distance, the higher the similarity between two points. Therefore, when the Hamming distance between two feature points in two images is less than a certain threshold, they are considered to be matching feature points.

[0087] Subsequently, RANSAC geometric consistency testing can be used to eliminate obvious mismatches. For two adjacent frames of 2D matching points, an epipolar geometric model can be established using the fundamental matrix or essential matrix, and interior points that satisfy the model constraints can be screened using the random sampling consistency method. After this step, the system obtains a more reliable set of matching point pairs, which can be used for backend essential matrix estimation, triangulation, PnP pose solving, or local BA optimization.

[0088] like Figure 3 As shown, to verify the improvement effect of the proposed method on the visual front-end feature distribution and motion estimation results, a fixed camera is set to rotate around the y-axis. Comparative experiments.

[0089] During the experiment, the camera position remained unchanged, and only the camera rotation angle around the y-axis was changed. The standard ORB method and the method proposed in this application were used to extract and match features from the two frames before and after the rotation, respectively, to further estimate the relative rotation angle between the two frames. The experimental results are shown in Table 1 below.

[0090] Table 1 Experimental Results

[0091] Around the camera Axis rotation Under the experimental conditions, the standard ORB method extracted 96 effective feature points, while the method in this application extracted 104 effective feature points, with both methods keeping the number of features within the range of approximately 100. This indicates that the method in this application does not improve motion estimation performance by significantly increasing the number of feature points, but rather improves the quality and distribution of feature points while maintaining a lightweight feature size.

[0092] From the feature distribution results, the feature distribution dispersion of the standard ORB method is... The spatial uniformity index is 7.84. The value of 0.113 indicates that the feature points are unevenly distributed on the image plane and tend to concentrate in areas with rich local texture. The feature distribution dispersion of the method in this application... Reduced to 3.26, spatial uniformity index The value was increased to 0.235, indicating that after local gray-level standard deviation adaptive threshold adjustment and spatial uniformity contribution screening, the final retained feature points are more evenly distributed on the image plane, which can provide more complete image region constraints for subsequent geometric estimation.

[0093] Based on the motion estimation results, the rotation angle estimated by the standard ORB method is: Relative to the true rotation angle The error is The rotation angle estimated by the method in this application is: The rotation angle error was reduced to This indicates that, when the number of feature points is similar, the method in this application improves the spatial distribution of feature points, making the rotation estimation results closer to the actual motion state and reducing the geometric constraint bias caused by local feature concentration.

[0094] Meanwhile, the processing time per frame increased from 4.8ms to 5.5ms, an increase of only 0.7ms, indicating that the method in this application does not significantly compromise the advantage of fast ORB front-end computation speed while improving the quality of feature distribution and the accuracy of rotation estimation.

[0095] Based on the same inventive concept, this application also provides an adaptive ORB visual front-end feature extraction device for implementing the adaptive ORB visual front-end feature extraction method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more embodiments of the adaptive ORB visual front-end feature extraction device provided below can be found in the limitations of the adaptive ORB visual front-end feature extraction method described above, and will not be repeated here.

[0096] In one exemplary embodiment, such as Figure 4 As shown, an adaptive ORB visual front-end feature extraction device is provided, comprising: The acquisition module is used to acquire the grayscale image of the image to be extracted.

[0097] The grid division module is used to divide the grayscale image into grids and determine the grayscale standard deviation of each grid region and the average grayscale standard deviation of each grid region in the overall image.

[0098] The adaptive FAST detection threshold determination module is used to determine the adaptive FAST detection threshold for each grid region based on the gray standard deviation of each grid region and the average gray standard deviation of each grid region in the overall image.

[0099] The corner detection module is used to perform corner detection based on the adaptive FAST detection threshold in each grid area to obtain the normalized corner response value of the candidate feature points.

[0100] The spatial uniformity contribution determination module is used to determine the spatial uniformity contribution of each candidate feature point within each grid region based on the feature distribution dispersion and spatial homogeneity index.

[0101] The comprehensive filtering module is used to comprehensively filter candidate feature points based on the normalized corner response value and the uniformity contribution to obtain target feature points.

[0102] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 5 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores adaptive ORB visual front-end feature extraction data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an adaptive ORB visual front-end feature extraction method.

[0103] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment to which the present application is applied. Specific computer equipment may include, for example, [the following is a list of possible additional structures]. Figure 5 The embodiments show more or fewer components, combinations of certain components, or different component arrangements. In one exemplary embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the above-described method embodiments.

[0104] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the above-described method embodiments.

[0105] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method embodiments.

[0106] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0107] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0108] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0109] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An adaptive ORB visual front-end feature extraction method, characterized in that, include: Obtain the grayscale image of the image to be extracted; The grayscale image is divided into grids, and the grayscale standard deviation of each grid region and the average grayscale standard deviation of each grid region in the overall image are determined. The adaptive FAST detection threshold for each grid region is determined based on the gray standard deviation of each grid region and the average gray standard deviation of each grid region in the overall image. Within each grid region, corner detection is performed based on the adaptive FAST detection threshold to obtain the normalized corner response value of the candidate feature point; For each candidate feature point within each grid region, the spatial uniformity contribution of each candidate feature point is determined based on the feature distribution dispersion and spatial homogeneity indices. The candidate feature points are comprehensively screened based on the normalized corner response value and the uniformity contribution to obtain the target feature points.

2. The adaptive ORB visual front-end feature extraction method according to claim 1, characterized in that, Obtain the grayscale image of the image to be extracted, specifically including: Acquire consecutive image frames and convert the consecutive image frames into grayscale images; The grayscale image is subjected to distortion correction using camera calibration parameters to obtain the grayscale image of the image to be extracted.

3. The adaptive ORB visual front-end feature extraction method according to claim 1, characterized in that, The expression for the adaptive FAST detection threshold is: ； in, For the first Adaptive FAST detection threshold for each grid region Based on the FAST detection threshold, This is the grayscale standard deviation adjustment coefficient. and These are the lower and upper limits of the detection threshold, respectively. This represents the truncation function. The grayscale standard deviation for each grid region, This represents the average of the standard deviations of gray levels in each grid region of the overall image.

4. The adaptive ORB visual front-end feature extraction method according to claim 1, characterized in that, For each candidate feature point within each grid region, the spatial uniformity contribution of each candidate feature point is determined based on the feature distribution dispersion and spatial homogeneity indices, specifically including: Within each grid region, the feature distribution dispersion is determined based on the number of selected feature points and the average number of feature points; Based on the feature distribution dispersion utilization Determine the spatial uniformity index; Based on the spatial uniformity index, utilize Determine the spatial uniformity contribution of each candidate feature point; in, As an index of spatial uniformity, The characteristic distribution dispersion, Contribute to the spatial uniformity of each candidate feature point. To add candidate feature points Spatial uniformity index after.

5. The adaptive ORB visual front-end feature extraction method according to claim 1, characterized in that, Candidate feature points are comprehensively screened based on the normalized corner response value and the uniformity contribution to obtain target feature points, specifically including: The comprehensive score for each candidate feature point is determined based on the normalized corner response value and the uniformity contribution. Candidate feature points are sorted from high to low based on the comprehensive score; Select target feature points according to the sorting until the target feature point quantity is reached or the sorting is empty.

6. The adaptive ORB visual front-end feature extraction method according to claim 1, characterized in that, Also includes: Generate an ORB descriptor based on the target feature points; Based on the ORB descriptor, feature matching and basic mismatch elimination are performed to obtain matching points; The matching points are used for essential matrix estimation, triangulation, PnP pose solving, or local BA optimization.

7. An adaptive ORB visual front-end feature extraction device, characterized in that, include: The acquisition module is used to acquire the grayscale image of the image to be extracted; The grid division module is used to divide the grayscale image into grids, determine the grayscale standard deviation of each grid region and the average grayscale standard deviation of each grid region in the overall image; An adaptive FAST detection threshold determination module is used to determine the adaptive FAST detection threshold for each grid region based on the gray standard deviation of each grid region and the average gray standard deviation of each grid region in the overall image. The corner detection module is used to perform corner detection based on the adaptive FAST detection threshold in each grid area to obtain the normalized corner response value of the candidate feature point; The spatial uniformity contribution determination module is used to determine the spatial uniformity contribution of each candidate feature point within each grid region based on the feature distribution dispersion and spatial homogeneity index. The comprehensive filtering module is used to comprehensively filter candidate feature points based on the normalized corner response value and the uniformity contribution to obtain target feature points.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the adaptive ORB visual front-end feature extraction method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the adaptive ORB visual front-end feature extraction method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the adaptive ORB visual front-end feature extraction method as described in any one of claims 1-6.

Images