An image feature extraction method

Abstract

This invention provides an image feature extraction method, relating to the field of image processing technology. The image feature extraction method includes the following steps: performing texture analysis on the image to be extracted, and determining the grid size and / or the number N of feature points to be extracted within the grid based on the results of the texture analysis. cell When the grid size is determined based on the texture analysis results, the image to be extracted is divided into grid regions according to the grid size; when the number N of feature points to be extracted within the grid is determined based on the texture analysis results... cell At that time, a corner detector is used to extract feature points within the grid. The number of extracted feature points is denoted as N. It is then determined whether N is greater than N0. cell When N>N cell When N ≤ N, the extracted feature points are filtered, and the filtered feature points are used as the feature points of the grid output; cell In this process, the extracted feature points are directly used as the feature points output by the grid. This invention fully considers the distribution of texture in the image, making the extracted feature points more reflective of the image's characteristics.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image feature extraction method. Background Technology

[0002] ORB-SLAM3 combines the advantages of ORB-SLAM2 and VIORB algorithms, enabling visual, visual-inertial, and multi-map SLAM algorithms using monocular, binocular, and RGB-D cameras, with superior performance.

[0003] However, existing ORB-SLAM3 algorithms for ORB image feature extraction do not fully consider the distribution of texture in the image during feature extraction. In the feature extraction process, the image is divided into equal parts using a fixed grid size, and a fixed number of feature points are extracted from each region. Thus, in texture-rich regions, the limited number of feature points may restrict the number of selectable feature points, leading to the loss of valuable feature information; in texture-sparse regions, the limited number may result in too many extracted feature points, and forcibly extracting features may introduce unstable noise.

[0004] Therefore, it is necessary to improve the ORB image feature extraction method in the existing ORB-SLAM3 algorithm and provide a new image feature extraction method so that the extracted feature points can better reflect the characteristics of the image. Summary of the Invention

[0005] The purpose of this invention is to provide an image feature extraction method to solve the problem that the existing image feature extraction methods in ORB-SLAM3 do not fully consider the distribution of texture in the image, resulting in the extracted feature points failing to accurately reflect the image features.

[0006] To achieve the above objectives, the present invention provides an image feature extraction method. The image feature extraction method includes the following steps: performing texture analysis on the image to be extracted, and determining the grid size and / or the number of feature points to be extracted within the grid based on the results of the texture analysis. When the grid size is determined based on the texture analysis results, the image to be extracted is divided into grid regions according to the grid size; when the number of feature points to be extracted within the grid is determined based on the texture analysis results... At that time, a corner detector is used to extract feature points within the grid, and the number of extracted feature points is denoted as . ,judge Is it greater than ;when When the extracted feature points are selected, the selected feature points are used as the feature points output by the grid; when When the extracted feature points are used, they are directly used as the feature points of the grid output.

[0007] Furthermore, the method for performing texture analysis on the image to be extracted specifically includes: selecting pixel Q as the center pixel; with Q as the center, a radius of... A circular region is selected, with M sampling points (pixels). The grayscale value of each pixel is compared with the grayscale value of the center point. If the grayscale value of the pixel is greater than or equal to the grayscale value of the center point, the corresponding binary value of the pixel is set to 1; otherwise, it is set to 0. The binary codes obtained from the M pixels are arranged in order to obtain an M-bit binary number. The M-bit binary number is converted into a decimal number and used to mark the center pixel. The LBP calculation formula for each pixel is as follows:

[0008] (1)

[0009] in, Center pixel The grayscale value of the center pixel. The grayscale value of the sampling point. As a symbolic function, it can be represented as:

[0010] (2)

[0011] The rotation of LBP is determined by the following formula (3):

[0012] (3)

[0013] in, The binary operator rotates clockwise. Second-rate, Indicates to Perform rotation; calculate the rotation LBP value of each pixel in the image to be extracted using formulas (1)-(3); statistically analyze the rotation LBP feature value of each pixel to form an LBP histogram; sum the LBP histograms to obtain the total texture histogram of the image to be extracted. .

[0014] Furthermore, the method for determining the mesh size based on the texture analysis results specifically includes: dividing the sum of the texture histograms by the number of rows in the histogram to obtain the average LBP histogram value of the image to be extracted. The grid size for the image to be extracted is calculated using the following formula:

[0015] (4)

[0016] in, This refers to the grid size; The minimum value of the LBP histogram of the image to be extracted; The maximum value of the LBP histogram of the image to be extracted; Both b and are set coefficients of a linear function; The average value of the LBP histogram represents the average distribution of texture in the image to be extracted. The minimum value of the LBP histogram of the image to be extracted; The maximum value of the LBP histogram of the image to be extracted; Both b and are set coefficients of a linear function; The average value of the LBP histogram represents the average distribution of texture in the image to be extracted.

[0017] Furthermore, the number of feature points extracted within the mesh is determined based on the results of texture analysis. The specific methods include: calculating the rotation LBP value of each pixel in the mesh using formulas (1)-(3), statistically analyzing the histogram of the rotation LBP feature values ​​of all pixels, and obtaining the total texture histogram of the mesh by summing the histograms. The number of feature points extracted within the grid is calculated using the following formula. :

[0018] (5)

[0019] in, The number of feature points extracted within the grid. The number of feature points to be extracted from the image. This represents the sum of the texture histograms of the mesh. This represents the sum of texture histograms of the image to be extracted.

[0020] Furthermore, the method for filtering the extracted feature points includes the following steps executed sequentially: S1, treating the initial node of the grid image as 1; S2, splitting 1 node into 4 nodes, and counting the number of feature points in each node; when the number of feature points is 0, deleting the node; when the number of feature points is equal to 1, stopping the splitting of the node and saving the feature points of the node; when the number of feature points is greater than 1, waiting for the node to split and saving the node information; S3, determining the number of nodes to be retained relative to the number of feature points extracted from the grid. The relationship; when the number of retained nodes is less than When the number of nodes to be retained is greater than or equal to the specified number, execute S4; When S4 is executed, S5 is executed; S2 is executed for nodes waiting to be split; when there are multiple nodes waiting to be split, the nodes waiting to be split are split in descending order of the number of feature points; after each node waiting to be split is split, S3 is executed again; S5, the splitting of nodes waiting to be split is stopped, the response values ​​of the feature points inside the nodes waiting to be split are counted, the feature point with the largest response value is selected as the feature point to be saved for that node, and other feature points are deleted; S6, the feature points saved by each retained node are used as the filtered feature points.

[0021] Furthermore, when the grid size is determined solely based on the results of texture analysis, the image feature extraction method executes the following steps in sequence: S101, perform texture analysis on the image to be extracted; S102, determine the grid size based on the results of texture analysis; S103, divide the image to be extracted into grid regions according to the determined grid size; S104, select the first grid; S105, use the FAST corner detector and a high response threshold to extract feature points within the grid; S106, determine whether feature points have been found. If no feature points are found, use a low response threshold to extract feature points within the grid. Feature points are extracted and recorded. When a feature point is found, it is directly recorded. S107: Move to the next grid. S108: Determine whether all grids have been traversed. If all grids have been traversed, proceed to step S109. If not, proceed to steps S105-S108. S109: Filter the extracted feature points in the image to be extracted according to a fixed number of feature points. The filtered feature points are used as the output feature points of the image to be extracted. S110: Calculate the main direction of the output feature points of the image to be extracted.

[0022] Furthermore, when the number of feature points extracted within the mesh is determined solely based on the results of texture analysis... The image feature extraction method executes the following steps in sequence: S201, perform texture analysis on the image to be extracted; S202, divide the image to be extracted into grid regions according to a grid size set to a fixed value; S203, select the first grid; S204, determine the number of feature points to be extracted within the grid based on the texture analysis results; S205, extract feature points within the grid using a high response threshold with a FAST corner detector; S206, determine whether feature points have been found. If no feature points are found, proceed to S207; if feature points are found, proceed to S208; S207, extract feature points within the grid using a low response threshold, and then proceed to S208; S208, record the extracted feature points and record the number of extracted feature points as... ;S209, judgment Is it greater than ;when When the extracted feature points are selected, an octree is used for filtering, and the filtered feature points are used as the feature points for grid output; when When the extracted feature points are used as the feature points of the grid output, S210, move to the next grid; S211, determine whether all grids have been traversed. If all grids have been traversed, execute step S212; if all grids have not been traversed, execute steps S204~S211; S212, calculate the main direction of the feature points of the image output to be extracted.

[0023] Furthermore, when the mesh size and the number of feature points extracted within the mesh are determined based on the results of texture analysis... The image feature extraction method is executed in the following order: S301, perform texture analysis on the image to be extracted; S302, determine the grid size based on the texture analysis results; S303, divide the image to be extracted into grid regions according to the determined grid size; S304, select the first grid; S305, determine the number of feature points to be extracted within the grid based on the texture analysis results. S306. Use the FAST corner detector with a high response threshold to extract feature points within the grid; S307. Determine if feature points are found. If no feature points are found, proceed to S308; if feature points are found, proceed to S309; ​​S308. Lower the response threshold and use the lowered response threshold to extract feature points within the grid. After extraction, proceed to S309; ​​S309. Record the extracted feature points and record the number of extracted feature points as... S310, Judgment Is it greater than ;when When the extracted feature points are selected, the selected feature points are used as the feature points output by the grid; when When the extracted feature points are used as the feature points of the grid output, S311, move to the next grid; S312, determine whether all grids have been traversed. If all grids have been traversed, execute step S313; if all grids have not been traversed, execute steps S305~S312; S313, calculate the main direction of the feature points of the image output to be extracted.

[0024] Furthermore, the image feature extraction method also includes the following steps: dividing the complete image into pyramids to obtain multi-layer pyramid feature images, with each pyramid feature image serving as the image to be extracted; and sequentially extracting features from each pyramid feature image to complete the feature extraction of the complete image.

[0025] Furthermore, the image feature extraction method is based on the ORB-SLAM3 algorithm; the ORB image feature extraction method in the ORB-SLAM3 algorithm is improved to the image feature extraction method described above.

[0026] By applying the technical solution of this invention, texture analysis was performed on the image to be extracted, taking into full account the distribution of texture in the image, so that the extracted feature points can better reflect the features of the image.

[0027] Specifically, this application determines the grid size based on the results of texture analysis, thereby providing an image feature extraction method with adaptive grid size. Dynamically determining the grid size ensures more uniform feature point extraction, guarantees real-time performance, and avoids the problems of insufficient feature extraction for complex textures due to uniform grid size, and prolonged extraction time for simple textures, thus reducing system real-time performance. It also determines the number of feature points to be extracted within each grid based on the results of texture analysis, providing an image feature extraction method with adaptive feature point quantity. Dynamically determining the number of feature points to be extracted within each grid avoids introducing unstable noise from excessive feature point extraction in sparse texture areas and avoids the loss of valuable feature information from feature points in texture-rich areas, thus making the extracted feature points more reflective of the image characteristics. Finally, it determines both the grid size and the number of feature points to be extracted within each grid based on the results of texture analysis, providing a texture-adaptive image feature extraction method. Dynamically determining both the grid size and the number of feature points to be extracted within each grid makes the feature points more reflective of the image characteristics, thereby further improving positioning accuracy. Attached Figure Description

[0028] To more clearly illustrate the technical methods of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below.

[0029] Figure 1 This is a flowchart of an existing image feature extraction method;

[0030] Figure 2 A flowchart of an image feature extraction method according to an optional embodiment of the present invention is shown;

[0031] Figure 3 A flowchart of an image feature extraction method according to another optional embodiment of the present invention is shown;

[0032] Figure 4 A flowchart of an image feature extraction method according to another optional embodiment of the present invention is shown;

[0033] Figure 5 A schematic diagram of the filtering process in an optional embodiment of the present invention is shown;

[0034] Figure 6The diagram shows a comparison of feature extraction effects of the prior art and three optional embodiments of the present invention, wherein (a) is the feature extraction effect of the original ORB-SLAM3; (b) is the feature extraction effect of ORB-SLAM3 with adaptive mesh size; (c) is the feature extraction effect of ORB-SLAM3 with adaptive feature point number; and (d) is the feature extraction effect of ORB-SLAM3 with adaptive texture. Detailed Implementation

[0035] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0036] like Figures 2 to 4 As shown, this application provides an image feature extraction method, which includes the following steps: performing texture analysis on the image to be extracted, and determining the grid size and / or the number of feature points to be extracted within the grid based on the results of the texture analysis. When the grid size is determined based on the texture analysis results, the image to be extracted is divided into grid regions according to the grid size. When the number of feature points to be extracted within the grid is determined based on the texture analysis results, a corner detector is used to extract the feature points within the grid, and the number of extracted feature points is denoted as [missing information]. ,judge Is it greater than ;when When the extracted feature points are selected, the selected feature points are used as the feature points output by the grid; when In this process, the extracted feature points are directly used as the feature points output by the grid. Thus, the image feature extraction method provided in this application, by performing texture analysis on the image, reveals the texture distribution of the image, i.e., the distribution of texture-rich and texture-sparse regions in the image to be extracted. Then, based on the texture distribution of the image, the grid size and / or the number of feature points to be extracted within the grid are determined. This makes the feature extraction scheme more reasonable, helps to ensure the real-time performance of image feature extraction, and the feature points output by the grid can better reflect the features of the image.

[0037] Optionally, the method for performing texture analysis on the image to be extracted specifically includes: selecting pixel Q as the center pixel; with Q as the center, and a radius of... Select a circular area with M sampling points, i.e., pixels; compare the gray value of each pixel with the gray value of the center point. If the gray value of the pixel is greater than or equal to the gray value of the center point, set the corresponding binary number of the pixel to 1; otherwise, set it to 0; arrange the binary codes obtained from the M pixels in order to obtain an M-bit binary number; convert the M-bit binary number into a decimal number and use the decimal number to mark the center pixel;

[0038] The LBP calculation formula for each pixel is as follows:

[0039] (1)

[0040] in, Center pixel The grayscale value of the center pixel. The grayscale value of the sampling point. As a symbolic function, it can be represented as:

[0041] (2)

[0042] The rotation of LBP is determined by the following formula (3):

[0043] (3)

[0044] in, This refers to the binary LBP operator rotating clockwise. Second-rate, Indicates to Perform rotation; calculate the rotation LBP value of each pixel in the image to be extracted using formulas (1)-(3); statistically analyze the rotation LBP feature value of each pixel to form an LBP histogram; sum the LBP histograms to obtain the total texture histogram of the image to be extracted. Formula (3) is derived from formulas (1)-(2). Compared with the original LBP, the rotated LBP, i.e., compared with formula (1), has increased rotation and thus increased robustness.

[0045] Optionally, LBP features can be achieved by using different radii. To calculate, to cover different spatial resolutions; radius A larger radius means more sampling points and a higher computational cost, but it provides a more accurate reflection of the pixel's characteristics. The radius can be adjusted based on the specific situation. It balances computational efficiency with the effect of reflecting pixels.

[0046] The present invention employs the above method to achieve higher computational efficiency, better robustness, and the ability to obtain multi-scale texture features when performing texture analysis on the image to be extracted.

[0047] Optionally, the method for determining the mesh size based on the texture analysis results specifically includes: dividing the sum of the texture histograms by the number of rows in the histogram to obtain the average LBP histogram value of the image to be extracted. The grid size for the image to be extracted is calculated using the following formula:

[0048] (4)

[0049] in, This refers to the grid size; The minimum value of the LBP histogram of the image to be extracted; The maximum value of the LBP histogram of the image to be extracted; Both b and are set coefficients of a linear function; The average value of the LBP histogram represents the average distribution of texture in the image to be extracted.

[0050] The specific values ​​set for a and b can be revised based on the actual scenario.

[0051] In one specific embodiment of this application, a=0.4, b=220. =50, =500, meaning the grid size for the image to be extracted is calculated using the following formula:

[0052] .

[0053] Thus, when dynamically determining the grid size using the above method, the texture distribution of the image is taken into account, which is more conducive to making the extracted feature points reflect the features of the image.

[0054] Optionally, the number of feature points extracted within the mesh is determined based on the results of texture analysis. The specific methods include: calculating the rotation LBP value of each pixel in the mesh using formulas (1)-(3), statistically plotting the histogram of the rotation LBP feature values ​​of all pixels in the mesh, and obtaining the total texture histogram of the mesh by summing the histograms. The number of feature points extracted within the grid is calculated using the following formula. :

[0055] (5)

[0056] in, The number of feature points extracted within the grid. The number of feature points to be extracted from the image. This represents the sum of the texture histograms of the mesh. This represents the sum of texture histograms of the image to be extracted.

[0057] Optionally, This is a pre-set value.

[0058] Optionally, based on the set number of feature points to be extracted from the complete image and the set scale factor, the number of feature points to be extracted from each pyramid feature image is obtained, i.e., the number of feature points to be extracted from each image is obtained. .

[0059] For example, in one specific embodiment of this application, the number of feature points extracted from the overall image is 1000. The overall image is divided into 8 pyramid images, and the scale factor (scaling coefficient) is set to 1.2. Then, the number of feature points extracted from each pyramid image is 61, 73, 87, 105, 126, 151, 181, and 216 respectively, which is the number of feature points extracted from each layer of the image to be extracted. They are different. Then, based on the proportion of the sum of the texture histograms of the meshes to the sum of the texture histograms of the image to be extracted, the number of feature points within different meshes is determined. This allows for the extraction of more feature points in areas with rich texture and fewer feature points in areas with sparse texture, thus better reflecting the characteristics of the image.

[0060] Optionally, the method for filtering the extracted feature points includes the following steps performed sequentially: S1, treating the initial node of the grid image as 1; S2, splitting 1 node into 4 nodes, and counting the number of feature points in each node; when the number of feature points is 0, deleting the node; when the number of feature points is equal to 1, stopping the splitting of the node and saving the feature points of the node; when the number of feature points is greater than 1, waiting for the node to split and saving the node information; S3, determining the number of nodes to be retained and the number of feature points extracted from the grid. The relationship; when the number of retained nodes is less than When the number of nodes to be retained is greater than or equal to the specified number, execute S4; When the node is waiting to be split, execute S5; S4, execute S2 for the nodes waiting to be split; where, when there are multiple nodes waiting to be split, split the nodes in descending order of the number of feature points; after executing S2 for each node waiting to be split, execute S3 again; S5, stop splitting the nodes waiting to be split, count the response values ​​of the feature points inside the nodes waiting to be split, select the feature point with the largest response value as the feature point to be saved for that node, and delete other feature points; S6, the feature points saved by each retained node are the filtered feature points. In this way, a uniformly distributed feature subset can be obtained.

[0061] It should be noted that in step S4, the actual number of feature points within the grid is greater than the number of feature points extracted from the grid. Therefore, there is generally no situation where there are no nodes waiting to split, so it will not be discussed here.

[0062] Optionally, in step S2, one node can be split into two, six, or eight nodes, etc., as needed.

[0063] Optionally, the extracted feature points can be filtered using an octree feature point filtering method.

[0064] Based on the image feature extraction method provided by the present invention, and in combination with the order of execution steps, three preferred embodiments are provided, which are described in detail below.

[0065] like Figure 2 As shown, in an optional embodiment of this application, when the grid size is determined solely based on the results of texture analysis, the image feature extraction method executes the following steps in sequence: S101, perform texture analysis on the image to be extracted; S102, determine the grid size based on the results of texture analysis; S103, divide the image to be extracted into grid regions according to the determined grid size; S104, take the first grid; S105, extract feature points within the grid using a FAST corner detector with a high response threshold; S106, determine whether feature points are found; if no feature points are found, use a low response threshold. Feature points within the grid are extracted and recorded. When a feature point is found, it is directly recorded. S107: Move to the next grid. S108: Determine if all grids have been traversed. If all grids have been traversed, proceed to step S109; if not, proceed to steps S105-S108. S109: Filter the extracted feature points within the image to be extracted according to a fixed set number of feature points. The filtered feature points are used as the output feature points of the image to be extracted. S110: Calculate the principal direction of the output feature points of the image to be extracted. This embodiment provides an image feature extraction method with adaptive grid size. Dynamically determining the grid size allows for more uniform feature point extraction, ensuring real-time performance and avoiding the problems of insufficient feature extraction for complex textures due to uniform grid size, prolonged extraction time for simple textures, and reduced system real-time performance.

[0066] like Figure 3 As shown, in another optional embodiment of this application, when the number of feature points extracted within the mesh is determined solely based on the results of texture analysis... The image feature extraction method is executed in the following order: S201, perform texture analysis on the image to be extracted; S202, divide the image to be extracted into grid regions according to a grid size set to a fixed value; S203, select the first grid; S204, determine the number of feature points to be extracted within the grid based on the results of texture analysis. S205. Use the FAST corner detector with a high response threshold to extract feature points within the grid; S206. Determine if feature points are found. If no feature points are found, proceed to S207; if feature points are found, proceed to S208; S207. Use a low response threshold to extract feature points within the grid, and then proceed to S208; S208. Record the extracted feature points and record the number of extracted feature points as... ;S209, judgment Is it greater than ;when When the extracted feature points are selected, the selected feature points are used as the feature points output by the grid; when When extracting feature points, they are directly used as feature points output by the grid; S210, move to the next grid; S211, determine whether all grids have been traversed. If all grids have been traversed, proceed to step S212; if not, proceed to steps S204-S211; S212, calculate the principal direction of the feature points output by the image to be extracted. In this embodiment, an image feature extraction method with adaptive feature point quantity is provided. The method dynamically determines the number of feature points to be extracted in each grid, which can avoid introducing unstable noise due to too many feature points extracted in sparse texture areas and avoid the loss of valuable feature information in texture-rich areas, thereby making the extracted feature points more reflective of the image features.

[0067] like Figure 4 As shown, in another optional embodiment of this application, when the mesh size and the number of feature points extracted within the mesh are determined based on the results of texture analysis... The image feature extraction method is executed in the following order: S301, perform texture analysis on the image to be extracted; S302, determine the grid size based on the texture analysis results; S303, divide the image to be extracted into grid regions according to the determined grid size; S304, select the first grid; S305, determine the number of feature points to be extracted within the grid based on the texture analysis results. S306. Use the FAST corner detector with a high response threshold to extract feature points within the grid; S307. Determine if feature points are found. If no feature points are found, proceed to S308; if feature points are found, proceed to S309; ​​S308. Lower the response threshold and use the lowered response threshold to extract feature points within the grid. After extraction, proceed to S309; ​​S309. Record the extracted feature points and record the number of extracted feature points as... S310, Judgment Is it greater than ;when When the extracted feature points are selected, the selected feature points are used as the feature points output by the grid; when When extracting feature points, they are directly used as feature points in the grid output; S311, move to the next grid; S312, determine whether all grids have been traversed. If all grids have been traversed, proceed to step S313; if not, proceed to steps S305-S312; S313, calculate the principal direction of the output pyramid image feature points. In this embodiment, a texture-adaptive image feature extraction method is provided, dynamically determining the grid size and the number of feature points extracted within the grid, making the feature points better reflect the characteristics of the image, thereby further improving the positioning accuracy.

[0068] Optionally, the image feature extraction method further includes the following steps: dividing the complete image into pyramids to obtain multi-layer pyramid feature images, with each layer of pyramid feature image serving as the image to be extracted; and sequentially extracting features from each layer of pyramid feature image to complete the feature extraction of the complete image. In this way, using pyramid division to enhance multi-scale image changes results in better prediction performance.

[0069] Optionally, the coordinates of the feature points extracted from each pyramid feature image can be transformed back to the original image.

[0070] The key point of this invention lies in the subsequent method for extracting feature images of each pyramid layer. It can adaptively adjust the grid size and / or the number of feature points extracted within the grid for each pyramid layer feature image, so that the extracted feature points can better reflect the image features.

[0071] Optionally, the image feature extraction method provided by this invention is based on the ORB-SLAM3 algorithm; the ORB image feature extraction method in the ORB-SLAM3 algorithm is improved to the image feature extraction method described above or below. The ORB-SLAM3 algorithm generally has high accuracy in different field-of-view sizes and indoor and outdoor environments. This invention provides an improved ORB-SLAM3 algorithm by improving the ORB feature extraction step.

[0072] Optionally, except for the improvements made to the ORB feature extraction process, the rest of the present invention is consistent with the original ORB-SLAM3 feature extraction.

[0073] This application provides three improved ORB-SLAM3 algorithms: 1. An adaptive grid size ORB-SLAM3 algorithm, which dynamically determines the grid size based on the image texture to ensure more uniform feature point extraction and real-time performance. 2. An adaptive feature point quantity ORB-SLAM3 algorithm, which dynamically determines the number of feature points to be extracted in each grid based on the image texture. If the number of feature points to be extracted in a grid is greater than the number of feature points required for the grid, an octree filter is applied to the grid feature point selection to make the feature point distribution more uniform. This also avoids the loss of feature points in sparse texture areas, thus making the feature points better reflect the image features and improving localization accuracy. 3. A texture-adaptive ORB-SLAM3 algorithm, which dynamically determines the grid size and the number of feature points to be extracted in each grid based on the image texture, making the feature points better reflect the image features and thus improving localization accuracy.

[0074] The following is in conjunction with the appendix Figures 1 to 4 The present invention will further elaborate on the existing technology and specific embodiments.

[0075] Existing technologies such as Figure 1 As shown, image feature extraction is a crucial step in implementing SLAM in ORB-SLAM3. Image feature extraction in ORB-SLAM3 is primarily based on the ORB feature algorithm. When extracting feature points, the most important goal is to ensure that feature points are evenly distributed across all parts of the image. To achieve this, ORB-SLAM3 employs the following strategies: 1. Divide the image into multiple grid regions, and use a FAST corner detector to extract feature points in each grid region. The FAST detector uses high and low response thresholds to identify candidate feature points. If the response values ​​of feature points are generally low, the threshold is lowered and the search is repeated. 2. Since there may be too many feature points extracted in each region, an octree can be used to filter the pyramid image feature points. The octree ensures a more even distribution of feature points within each pyramid layer. The feature point extraction flowchart for each pyramid layer is shown below. Figure 1 As shown.

[0076] Although the above methods can effectively achieve uniform sampling of feature points, they still have some limitations:

[0077] 1. In the feature extraction process of ORB-SLAM3, the image is divided into equal parts, and feature points are extracted in each region. The grid size used for division is fixed at 35. However, this preset may lead to two problems: images with complex textures may extract too few features, while images with simple textures may extract too many features. For images with simple textures, the lack of variation in grid size results in prolonged feature extraction time and reduced system real-time performance.

[0078] 2. In the feature extraction process of ORB-SLAM3, the image is divided into equal parts, and a fixed number of feature points are extracted in each region. This approach may lead to two problems: First, in regions with rich texture, the limited number of feature points may restrict the number of selectable feature points, resulting in the loss of valuable feature information; second, in regions with sparse texture, the limited number may lead to the extraction of too many feature points, and forcibly extracting features may introduce noise with poor stability.

[0079] 3. Insufficient Texture Perception: ORB-SLAM3 does not fully consider the distribution of texture in the image during feature extraction. In reality, texture density often varies significantly across different regions. Texture-rich areas may contain a large number of valuable feature points; however, in sparsely textured areas, forcibly extracting features may result in unstable feature points. This approach can lead to less than ideal feature extraction results. Furthermore, when there are too many feature points in texture-rich areas, ORB-SLAM3 uses octrees for non-maximum suppression. This strategy may cause the loss of some important feature information. Simultaneously, it may also introduce unstable features in areas with less texture.

[0080] This invention proposes a mesh-size adaptive ORB-SLAM3 algorithm based on texture analysis. A specific embodiment of the image extraction method in the mesh-size adaptive ORB-SLAM3 algorithm is as follows: Figure 2 As shown, the specific steps for extracting the feature images of each pyramid layer are as follows:

[0081] 1. Texture analysis.

[0082] Texture analysis is a fundamental problem in image processing and computer vision. It typically refers to the process of obtaining texture feature parameters through image processing techniques to achieve a quantitative or qualitative description of the texture. Texture analysis methods can be categorized into three types: statistical methods, structural methods, and spectral methods. This paper uses the Local Binary Pattern (LBP) high-order statistical method to analyze image texture. The LBP method combines local structural analysis with event analysis statistics, providing excellent performance for texture analysis.

[0083] The specific steps of LBP texture analysis are as follows: 1.1 Select pixel Q as the center pixel. 1.2 With Q as the center, 1.3 Select a circular region with a radius of M, and set the region to M sampling points (pixels). Compare the grayscale value of each pixel with the grayscale value of the center point. If the grayscale value of the pixel is greater than or equal to the grayscale value of the center point, set the corresponding binary value of the pixel to 1; otherwise, set it to 0. 1.4 Arrange the binary codes obtained from the M pixels in order to obtain an M-bit binary number. 1.5 Convert the M-bit binary number to a decimal number and use this decimal number to mark the center pixel.

[0084] The LBP calculation formula for each pixel is as follows:

[0085] (1)

[0086] in, Center pixel The grayscale value of the center pixel. The grayscale value of the sampling point. As a symbolic function, it can be represented as:

[0087] (2)

[0088] Furthermore, LBP features can be improved by using different radii. To calculate, in order to cover different spatial resolutions.

[0089] LBP has the following advantages: ① Robustness: LBP is highly robust to changes in illumination because it only considers the relative brightness differences between pixels, rather than absolute brightness values. ② Computational efficiency: The calculation process of LBP is very simple, requiring only the comparison of pixel brightness values, thus resulting in high computational efficiency. ③ Multi-scale: By changing the radius and number of sampling points of LBP, multi-scale texture features can be obtained.

[0090] The LBP operator is grayscale invariant, but not rotation invariant. To improve system robustness while ensuring real-time performance, this invention employs rotation-invariant LBP to analyze image texture. A series of initially defined LBP values ​​are obtained by continuously rotating a circular neighborhood, and the minimum value is taken as the LBP value for that neighborhood. The rotation LBP is determined by the following formula:

[0091] (3)

[0092] in, binary The operator rotates clockwise Second-rate, Indicates to Rotate the image. Calculate the rotation LBP value of each pixel in the current pyramid image using formulas (1)-(3), and statistically analyze the rotation LBP feature value of each pixel to form an LBP histogram. Sum the LBP histograms to obtain the total texture histogram of the current pyramid image. Then, the average LBP histogram of the current pyramid image is obtained by dividing the sum of the texture histograms by the number of rows in the current pyramid image. .

[0093] 2. Determine the grid size. Based on the different textures of each pyramid image layer, the grid size used for each pyramid layer can be determined, specifically using the following linear formula.

[0094]

[0095] in, For grid size, The average value of the LBP histogram represents the average distribution of texture in the pyramid image.

[0096] 3. Extract FAST feature points. First, use a high response threshold to extract feature points. If no feature points are found, use a low response threshold to extract feature points again.

[0097] 4. Record the feature points and move to the next grid. Since the specific grid size was determined beforehand, dividing the pyramid image of this layer by the grid size gives the number of grids. After traversing each grid, determine whether all grids have been traversed.

[0098] 5. Octree Selection. The steps for octree feature point selection are as follows: ① First, initialize several nodes as the first layer of the octree based on the image size. These nodes are responsible for extracting feature points from different regions of the image. ② Assign all detected feature points to the corresponding nodes according to their coordinate positions. Therefore, each node will have some feature points within its assigned region. ③ For each node, if it has only one feature point, do not split the node further. If it has no feature points, delete the node. ④ Then check if the total number of nodes reaches the number of feature points to be extracted. If it does, end the process; otherwise, continue splitting nodes. ⑤ For nodes with multiple feature points, divide them evenly into four child nodes. Check if each child node also has a feature point; if so, add the child node to the node list. ⑥ Calculate the current number of nodes and three times the number of nodes to be split to see if it exceeds the number of feature points to be extracted. If it does, only split the largest nodes until the number of nodes reaches the number of feature points to be extracted. ⑦ Finally, retain the feature point with the largest response value in each node. These retained points are the feature subsets that are selected and are evenly distributed.

[0099] 6. Calculate the principal direction of each feature point. First, define a region with radius r for each feature point, and then calculate the centroid of this region using the following formula:

[0100] (6)

[0101] in, For pixels The grayscale value. Calculate the principal direction for each feature point. as follows:

[0102] (7)

[0103] In this way, localization and mapping are achieved by using the main direction information of each feature point as part of feature matching.

[0104] and Figure 1 In contrast, this embodiment dynamically determines the grid size based on the image texture, which can make feature point extraction more uniform and ensure real-time performance.

[0105] This invention, based on texture analysis methods, also proposes an adaptive ORB-SLAM3 algorithm with an adaptive number of feature points. A specific embodiment of the image extraction method in the adaptive ORB-SLAM3 algorithm is as follows: Figure 3 As shown, the specific steps for extracting the feature images of each pyramid layer are as follows:

[0106] First, texture analysis. The steps of texture analysis in this embodiment are the same as the image extraction method in the mesh size adaptive ORB-SLAM3 algorithm, and will not be repeated here. Second, determine the number of feature points to be extracted for each mesh. Calculate the rotation LBP value of each pixel in the current pyramid image within the mesh using formulas (1)-(3), and statistically analyze the histogram of the rotation LBP feature values ​​of all pixels in the mesh. The total texture histogram of the mesh is obtained by summing the histograms. The number of feature points to be extracted from the mesh is determined based on the proportion of the mesh texture histogram to the texture histogram of the pyramid image at that layer. The specific formula is as follows:

[0107] (5)

[0108] in, This represents the number of feature points to be extracted from a single grid in the pyramid image at this level. This represents the number of feature points extracted from this layer of the pyramid image. For the sum of the texture histograms of one of the meshes, This represents the sum of the texture histograms for this layer of the pyramid image.

[0109] Next, determine whether to use an octree to filter feature points within the grid. If the number of feature points extracted within the grid is greater than the number of feature points required to be extracted from the grid, then an octree is used to filter the feature points extracted within the grid to make the feature point distribution more even. If the number of feature points extracted within the grid is less than or equal to the number of feature points required to be extracted from the grid, then an octree is not used to filter the feature points, and the extracted feature points are directly retained to avoid the loss of feature points in sparse texture areas.

[0110] Alternatively, the steps of the octree feature point selection method are as follows:

[0111] 2.1 Split the mesh image into four nodes, i.e., split one root node into four nodes. Count the number of feature points in each region. If the number is 0, delete the node; if the number is 1, stop splitting the node and save its features; if the number is greater than 1, wait for the node to split and save its information.

[0112] 2.2 Determine if the number of nodes to be retained is greater than or equal to a set value. If the number of nodes is greater than or equal to the set value, then statistically analyze the response values ​​of feature points within nodes with a value greater than 1, select the feature point with the largest response value as the feature of that node, and delete the other features; if the number of nodes is less than the set value... If the number of feature points is greater than 1, then continue with step 2.1 until the number of nodes is greater than or equal to the set value. .

[0113] 2.3 When filtering, if multiple nodes contain more than 1 feature points, these nodes are sorted according to the number of feature points. The nodes are then split in descending order of the number of feature points. This process takes into full account the image texture while ensuring the uniformity of feature point distribution, thereby improving the accuracy of feature extraction.

[0114] In one specific embodiment, the process of filtering using octree feature points is as follows: Figure 5 As shown. Assume that one of the grids needs to extract 19 feature points. Feature point selection requires six steps. The specific implementation process is as follows:

[0115] The first step is to start with 1 node, such as... Figure 5 As shown in (a) in the diagram. The second step involves a node undergoing a quadratic split, generating four nodes. Since 4 is less than 19, the split continues, as shown in (a). Figure 5 As shown in (b) above. In the third step, each node splits into 4 nodes, resulting in a total of 16 nodes. Because... Figure 5In (c), there are no feature points in the first row and second column, so the actual number of nodes is 15. Since 15 is less than 19, the split continues. In the fourth step, the 15 split nodes are sorted by the number of feature points, and the node with the most feature points is further split, such as... Figure 5 As shown in the third row and fourth column of (d), one node splits into four. At this point, there are a total of 18 nodes. 18 is less than 19, so the splitting continues. In the fifth step, the 18 split nodes are sorted by the number of feature points, and the node with the most feature points is further split, as shown... Figure 5 As shown in the fourth row and third column of (e), one node splits into four. At this point, there are a total of 21 nodes. Since 21 is greater than 19, the splitting stops. The sixth step is to select the feature point with the highest response value from each node as the feature point of that node, and delete the other feature points, as shown below. Figure 5 As shown in (f) in the figure.

[0116] and Figure 1 In contrast, this embodiment dynamically determines the number of feature points to be extracted within each grid based on the image texture. If the number of feature points to be extracted within a grid exceeds the number of feature points required for the grid, an octree filter is applied to the grid feature point selection, resulting in a more uniform distribution of feature points. Simultaneously, it avoids the loss of feature points in sparsely textured areas, thereby ensuring that the feature points better reflect the image's characteristics and improving positioning accuracy.

[0117] This invention, based on texture analysis methods, also proposes a texture-adaptive ORB-SLAM3 algorithm. A specific embodiment of the image extraction method in the texture-adaptive ORB-SLAM3 algorithm is as follows: Figure 4 As shown, the specific steps for extracting feature images for each pyramid layer are as follows: First, perform rotated LBP texture analysis on the image, sum the texture histograms of that pyramid layer, and calculate the average texture of the histograms. Then, determine the grid size and the number of feature points to be extracted from each grid. Based on the number of feature points extracted from each grid, determine whether to use an octree for filtering within each grid. Finally, provide the feature point information.

[0118] The method for determining the grid size in this embodiment is the same as the scheme described in the image extraction method of the adaptive ORB-SLAM3 algorithm for grid size, and will not be repeated here; the method for determining the number of feature points to be extracted for each grid is the same as the scheme described in the image extraction method of the adaptive ORB-SLAM3 algorithm for feature point number, and will not be repeated here.

[0119] and Figure 1 In contrast, this embodiment dynamically determines the grid size and the number of feature points to be extracted in each grid based on the image texture, so that the feature points can better reflect the features of the image, thereby improving the positioning accuracy.

[0120] Visual SLAM boasts high autonomy, strong real-time performance, strong anti-interference capabilities, and the ability to acquire a large amount of environmental information, closely resembling human environmental perception. It is suitable for various scenarios and is increasingly widely used in industrial inspection. Among numerous visual SLAM algorithms, the original ORB-SLAM3 is currently a high-performing SLAM framework. However, the original ORB-SLAM3 does not fully consider the distribution of texture in the image during feature extraction. In reality, texture density often varies significantly across different regions. Texture-rich areas may contain a large number of valuable feature points; however, in sparsely textured areas, forcibly extracting features may result in unstable feature points. This leads to less than ideal feature extraction results, consequently affecting the accuracy of system pose estimation.

[0121] The improved ORB-SLAM3 algorithm with adaptive mesh size, feature point quantity, and texture provided by this invention can be applied to industrial inspection robots. It performs better in industrial environments with varying and repetitive textures, thus improving positioning accuracy. During feature extraction, the algorithm analyzes image texture to determine the mesh size and / or the number of extracted feature points, thereby improving the quality of feature point distribution and enhancing camera pose estimation accuracy. This improved ORB-SLAM3 algorithm with adaptive mesh size, feature point quantity, and texture enhances both system positioning and pose estimation accuracy while maintaining real-time performance.

[0122] Specifically, the improved adaptive mesh size ORB-SLAM3 algorithm provided by this invention first performs texture analysis on the image using Local Binary Pattern (LBP) and calculates the texture histogram. Then, the mesh size is determined based on the average value of the texture histogram. The principle adopted is that the larger the average texture value, the smaller the mesh size.

[0123] The adaptive ORB-SLAM3 algorithm for the number of feature points provided by this invention first performs texture analysis on the image using Local Binary Pattern (LBP) and calculates the texture histogram. Then, based on the proportion of texture within each grid cell to the total texture of the image to be extracted, the number of feature points to be extracted within that grid cell is determined. Finally, the number of feature points extracted within each grid cell is compared with the determined number of feature points for that grid cell to decide whether to use octree feature point filtering within the grid. When the actual number of extracted feature points is greater than the number of feature points required for that grid cell, octree filtering is performed; otherwise, the actual number of extracted feature points is determined as the feature points required for the current grid cell.

[0124] The texture-adaptive ORB-SLAM3 algorithm provided by this invention first performs texture analysis on the image using Local Binary Pattern (LBP) and calculates the texture histogram. Then, the grid size is determined based on the average value of the texture histogram, with a principle that the larger the average texture value, the smaller the grid size. Finally, the number of feature points to be extracted within each grid is determined based on the proportion of texture within each grid to the total texture of the image to be extracted. Finally, the number of feature points extracted within each grid is compared with the determined number of feature points for that grid to decide whether to use octree feature point filtering within the grid. When the actual number of extracted feature points is greater than the number of feature points required for that grid, octree filtering is performed; otherwise, the actual extracted feature points are determined as the feature points required for the current grid.

[0125] The ORB-SLAM3 algorithm with adaptive mesh size, feature point quantity, and texture provided in this invention comprises three parallel threads: a tracking thread, a local mapping thread, and a loop closure and map fusion thread. The main improvement lies in the ORB feature extraction stage within the tracking thread. The tracking thread primarily processes sensor information, calculates the pose of the current frame relative to the active map in real time, minimizes reprojection errors from matching map features, and outputs new keyframes for the local mapping thread. The specific implementation process of feature point extraction using the adaptive mesh size, feature point quantity, and texture method is as follows: Figure 2-4 As shown. First, the image texture is analyzed. Based on the image texture, the grid size and / or the number of feature points to be extracted within the grid are determined. When determining the number of feature points to be extracted within the grid, after feature extraction using ORB, it is decided whether to use an octree to filter the feature points within the grid based on the actual number of feature points extracted. If the actual number of extracted feature points is greater than the number of feature points required for that grid, octree filtering is performed; otherwise, the actual extracted feature points are determined as the feature points required for the current grid. Local mapping thread: still uses the ORB-SLAM3 method. This thread adds new keyframes and map points to the active map, removes redundancy, and updates the map using a sliding window through local BA. Under visual inertial conditions, the maximum a posteriori estimation technique is used to initialize and update the IMU parameters to reduce IMU noise accumulation. Loop closure and map fusion thread: still uses the ORB-SLAM3 method. This thread detects common areas between the active and inactive maps each time a new keyframe is added. If the common area belongs to the active map, loop closure correction is performed; if the common area belongs to different maps, they are merged to form the active map. After loop closure correction, to avoid affecting system real-time performance, a new thread is created to execute global BA to further optimize the map.

[0126] This invention conducted feature extraction experiments, selecting a frame image from the publicly available dataset TUM as input to extract features from the image. The extraction results are compared in the following figures. Figure 6 As shown. Figure 6 (a) shows the feature extraction results using the original ORB-SLAM3. Figure 6 (b) shows the feature extraction effect using adaptive grid size. The difference between this method and ORB-SLAM3 feature extraction is that the grid size changes dynamically.

[0127] pass Figure 6 (a) and Figure 6 The comparison in (b) shows that, compared to ORB-SLAM3 feature extraction, the mesh size adaptive feature extraction method can further extract key corner points from the back and front of the computer monitor, such as... Figure 6 The area shown in box (b) is shown in the image.

[0128] Figure 6 (c) shows the adaptive feature extraction effect based on the number of grid feature points. The difference between this method and the original ORB-SLAM3 feature extraction is that the number of feature points extracted within each grid varies, and the octree filtering is performed within the grid. Figure 6 (a) and Figure 6 The comparison in (c) shows that the feature point number adaptive feature extraction method extracts more and more feature points at the corners of seat armrests, arms, monitors, and the edges and texture changes of books. Specifically, as shown in [examples omitted] Figure 6 The area shown in box (c) is shown in the image.

[0129] Figure 6 In the diagram, (d) represents the texture-adaptive feature extraction effect, which is a fusion of mesh size adaptation and feature point number adaptation. Through... Figure 6 (a) and Figure 6 The comparison in (d) shows that the texture adaptive feature extraction method extracts more sufficient and more uniform features from the edges of objects or patterns such as chairs, books, characters, and computers. Figure 6 The region shown in box (d) is illustrated. Furthermore, its feature extraction performance is comparable to... Figure 6 (b) and Figure 6 (c) is also more outstanding.

[0130] Experiments have shown that the image feature extraction method provided by this invention can improve the accuracy of visual SLAM algorithms, make the feature point distribution more uniform, and thus provide good positioning accuracy in cases of sparse or complex textures.

[0131] To verify the effectiveness of the method provided in this invention, ablation comparison experiments were conducted on the EuRoc dataset. The results show that the improved ORB-SLAM3 algorithm with adaptive mesh size, feature point count, and texture is superior to the existing original ORB-SLAM3 algorithm in all aspects. Furthermore, the texture-adaptive ORB-SLAM3 algorithm of this invention achieves higher localization accuracy compared to two other separately executed mesh-adaptive and in-mesh feature point adaptive algorithms, as well as other monocular inertial SLAM algorithms. The preferred embodiment of the texture-adaptive ORB-SLAM3 algorithm, compared to the original ORB-SLAM3 algorithm, demonstrates higher camera pose estimation accuracy and mapping performance in most environments.

Claims

1. An image feature extraction method, characterized in that, An improvement is made to the ORB image feature extraction method in the existing ORB-SLAM3 algorithm. The image feature extraction method includes the following steps: Perform texture analysis on the image to be extracted, and determine the grid size and / or the number of feature points to be extracted within the grid based on the results of the texture analysis. ; When the grid size is determined based on the results of texture analysis, the image to be extracted is divided into grid regions according to the grid size. When the number of feature points extracted within the mesh is determined based on the results of texture analysis At that time, a corner detector is used to extract feature points within the grid, and the number of extracted feature points is denoted as . ,judge Is it greater than ;when When the extracted feature points are selected, the selected feature points are used as the feature points for the grid output; when At that time, the extracted feature points are directly used as the feature points of the grid output; The method for performing texture analysis on the image to be extracted specifically includes: Choose pixel Q as the center pixel; Centered on Q, with a radius of Select a circular region, which has M sampling points, i.e., pixels; The grayscale value of each pixel is compared with the grayscale value of the center point. If the grayscale value of the pixel is greater than or equal to the grayscale value of the center point, the corresponding binary value of the pixel is set to 1; otherwise, it is set to 0. Arrange the binary codes obtained from the M pixels in order to get an M-bit binary number; Convert the M-bit binary number into a decimal number and use the decimal number to mark the center pixel; The LBP calculation formula for each pixel is as follows: （1） in, Center pixel The grayscale value of the center pixel. The grayscale value of the sampling point. As a symbolic function, it can be represented as: （2） The rotation of LBP is determined by the following formula (3): （3） in, The binary operator rotates clockwise. Second-rate, Indicates to Rotate; The rotation LBP value of each pixel in the image to be extracted is calculated using formulas (1)-(3); The rotational LBP feature values ​​of each pixel are statistically analyzed to form an LBP histogram; Summing the LBP histograms yields the sum of the texture histograms of the image to be extracted. ; The method for determining the mesh size based on the results of texture analysis specifically includes: Divide the sum of the texture histograms by the number of rows in the histogram to obtain the average LBP histogram of the image to be extracted. ; The grid size for the image to be extracted is calculated using the following formula: （4） in, This refers to the grid size; The minimum value of the LBP histogram of the image to be extracted; The maximum value of the LBP histogram of the image to be extracted; and All are set linear function coefficients; The average value of the LBP histogram represents the average distribution of texture in the image to be extracted. The number of feature points extracted within the mesh is determined based on the results of texture analysis. The methods specifically include: The rotation LBP value of each pixel in the mesh is calculated using formulas (1)-(3). A histogram of the rotation LBP feature values ​​of all pixels in the mesh is plotted. The total texture histogram of the mesh is obtained by summing the histograms. ; The number of feature points extracted within the grid is calculated using the following formula. : （5） in, The number of feature points extracted within the grid. The number of feature points to be extracted from the image. This represents the sum of the texture histograms of the mesh. This represents the sum of texture histograms of the image to be extracted; When the mesh size and the number of feature points extracted within the mesh are determined based on the results of texture analysis. At that time, the image feature extraction method is executed in the following order: S301. Perform texture analysis on the image to be extracted; S302. Determine the mesh size based on the results of texture analysis; S303. Divide the image to be extracted into grid regions according to the determined grid size; S304, Take the first grid; S305. Determine the number of feature points to be extracted within the mesh based on the results of texture analysis. ; S306. Use the FAST corner detector to extract feature points within the grid using a high response threshold; S307. Determine whether a feature point has been found. If no feature point is found, execute S308; if a feature point is found, execute S309. S308. Reduce the response threshold and use the reduced low response threshold to extract feature points within the grid. After extraction, execute S309. S309. Record the extracted feature points, and record the number of extracted feature points as follows: ; S310, Judgment Is it greater than ;when When the extracted feature points are selected, an octree is used for filtering, and the filtered feature points are used as the feature points for grid output; when At that time, the extracted feature points are directly used as the feature points of the grid output; S311, Move to the next grid; S312. Determine whether all the meshes have been traversed. If all the meshes have been traversed, proceed to step S313; if all the meshes have not been traversed, proceed to steps S305~S312. S313. Calculate the principal direction of the feature points output from the image to be extracted.

2. The image feature extraction method according to claim 1, characterized in that, The method for filtering the extracted feature points includes the following steps performed in sequence: S1. Treat the initial nodes of the grid image as 1; S2. Split one node into four nodes and count the number of feature points in each node; When the number of feature points is 0, the node is deleted; When the number of feature points is equal to 1, the node stops splitting and the feature points of the node are saved; When the number of feature points is greater than 1, the node waits to split and its information is saved. S3. Determine the number of nodes to be retained and the number of feature points extracted within the grid. The relationship; when the number of retained nodes is less than When the number of nodes to be retained is greater than or equal to the specified number, execute S4; At that time, execute S5; S4. Execute S2 on nodes waiting to be split; where, when there are multiple nodes waiting to be split, split the nodes waiting to be split in descending order of the number of feature points; after executing S2 on each node waiting to be split, execute S3 again. S5. Stop splitting nodes that are waiting to be split, count the response values ​​of the feature points inside the nodes that are waiting to be split, select the feature point with the largest response value as the feature point to be saved for that node, and delete the other feature points. S6. The feature points saved by each retained node are used as the feature points obtained by the filtering.

3. The image feature extraction method according to claim 1, characterized in that, When the mesh size is determined solely based on the results of texture analysis, the image feature extraction method executes the following steps in sequence: S101. Perform texture analysis on the image to be extracted; S102. Determine the mesh size based on the results of texture analysis; S103. Divide the image to be extracted into grid regions according to the determined grid size; S104, Take the first grid; S105. Use the FAST corner detector to extract feature points within the grid using a high response threshold; S106. Determine whether a feature point has been found. If no feature point is found, use a low response threshold to extract the feature points within the grid and record the extracted feature points. If a feature point is found, directly record the extracted feature points. S107, Move to the next grid; S108. Determine whether all the meshes have been traversed. If all the meshes have been traversed, proceed to step S109; if all the meshes have not been traversed, proceed to steps S105-S108. S109. The extracted feature points in the image to be extracted are filtered according to a fixed number of feature points to be extracted, and the filtered feature points are used as the feature points output in the image to be extracted. S110. Calculate the principal direction of the feature points output from the image to be extracted.

4. The image feature extraction method according to claim 1, characterized in that, When the number of feature points extracted within the mesh is determined solely based on the results of texture analysis At that time, the image feature extraction method is executed in the following order: S201. Perform texture analysis on the image to be extracted; S202. Divide the image to be extracted into grid regions according to a grid size set to a fixed value; S203, Take the first grid; S204. Determine the number of feature points to be extracted within the mesh based on the results of texture analysis. ; S205. Use the FAST corner detector to extract feature points within the grid using a high response threshold; S206. Determine whether a feature point has been found. If no feature point is found, execute S207; if a feature point is found, execute S208. S207. Use a low response threshold to extract feature points within the grid, and then execute S208. S208. Record the extracted feature points, and record the number of extracted feature points as follows: ; S209. Judgment Is it greater than ;when When the extracted feature points are selected, an octree is used for filtering, and the filtered feature points are used as the feature points for grid output; when At that time, the extracted feature points are directly used as the feature points of the grid output; S210, Move to the next grid; S211. Determine whether the entire mesh has been traversed. If the entire mesh has been traversed, proceed to step S212; if the entire mesh has not been traversed, proceed to steps S204~S211. S212. Calculate the principal direction of the feature points output from the image to be extracted.

5. The image feature extraction method according to claim 1, characterized in that, The image feature extraction method further includes the following steps: The complete image is divided into pyramids to obtain multi-layer pyramid feature images, and each pyramid feature image is used as the image to be extracted. Feature extraction is performed on each pyramid feature image in turn to complete the feature extraction of the complete image.

6. The image feature extraction method according to any one of claims 1 to 5, characterized in that, The image feature extraction method is based on the ORB-SLAM3 algorithm; The ORB image feature extraction method in the ORB-SLAM3 algorithm is improved to the image feature extraction method described in any one of claims 1 to 5.

Images