Image feature extraction method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure disclose an image feature extraction method and device, electronic equipment and a storage medium. The method comprises: performing edge extraction on an initial image to obtain an edge image, and then determining non-edge pixel points in the initial image, performing feature point extraction on the non-edge pixel points to obtain to-be-divided feature points; taking the initial image as a to-be-divided grid, performing N-ary tree division on the to-be-divided grid to obtain N grids and types of the grids; determining whether a stop division condition is met according to a remaining expected feature point quantity, a quantity of dividable grids, a preset pixel quantity and a preset minimum feature point quantity, to stop the division to obtain an unfinished grid or continue the division; taking to-be-divided feature points in each valid grid as first feature points, determining second feature points according to a quantity of the first feature points, an expected feature point quantity and each unfinished grid, and then determining target feature points, to improve consistency of feature point distribution and image space distribution.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more particularly to an image feature extraction method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the field of autonomous driving, sensors such as cameras are needed to collect images to perform functions such as mapping and localization of the surrounding environment. For the collected images, image feature extraction is required, and the spatial uniformity of the extracted point features has a significant impact on the accuracy and robustness of the various related functions.

[0003] Currently, the feature point homogenization strategy used when extracting point features from images overemphasizes the uniform distribution of feature points on the image and ignores the actual feature distribution in the image space. This results in inconsistencies between the extracted feature point distribution and the actual image space distribution, as well as the suppression of local features. This can further affect the implementation of other related functions and even the implementation of autonomous driving functions. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, embodiments of this disclosure provide an image feature extraction method, apparatus, electronic device, and storage medium to improve feature point extraction efficiency and, while ensuring the uniformity of the distribution of feature points extracted in the image, improve the consistency between the feature point distribution and the spatial distribution of the image.

[0005] In a first aspect, embodiments of this disclosure provide an image feature extraction method, the method comprising:

[0006] Edge extraction is performed on the initial image to obtain an edge image corresponding to the initial image. Based on the initial image and the edge image, non-edge pixels in the initial image are determined, and feature points are extracted from the non-edge pixels to obtain feature points to be segmented.

[0007] The initial image is used as the grid to be divided, and the grid to be divided is divided into N-ary trees to obtain N grids. The type of each grid is determined; wherein, the type is valid, invalid, or divisible; a valid grid means that there is one and only one feature point to be divided in the grid; an invalid grid means that there is no feature point to be divided in the grid; a divisible grid means that there are at least two feature points to be divided in the grid.

[0008] Based on the remaining expected feature points, the number of divisible grids, the preset number of pixels, and the preset minimum number of feature points, it is determined whether each divisible grid meets the stopping condition. If so, the divisible grids that meet the stopping condition are identified as incomplete grids. If not, the divisible grids that do not meet the stopping condition are identified as grids to be divided, and the process of performing N-ary tree partitioning on the grids to be divided is returned to obtain N grids, and the type of each grid is determined, until the stopping condition is met. Here, the remaining expected feature points are the difference between the expected feature points corresponding to the initial image and the number of valid grids.

[0009] The feature points to be divided within each valid grid are taken as the first feature points, and the second feature points are determined based on the number of the first feature points, the expected number of feature points, and each incomplete grid. The first feature points and the second feature points are then determined as the target feature points.

[0010] Secondly, embodiments of this disclosure also provide an image feature extraction apparatus, the apparatus comprising:

[0011] The feature point extraction module is used to extract edges from the initial image to obtain an edge image corresponding to the initial image, and to determine non-edge pixels in the initial image based on the initial image and the edge image, and to extract feature points from the non-edge pixels to obtain the feature points to be divided.

[0012] The preliminary grid division module is used to take the initial image as the grid to be divided, and to perform N-ary tree partitioning on the grid to be divided to obtain N grids, and to determine the type of each grid; wherein, the type is valid, invalid, or divisible; a valid grid indicates that there is one and only one feature point to be divided in the grid; an invalid grid indicates that there is no feature point to be divided in the grid; a divisible grid indicates that there are at least two feature points to be divided in the grid.

[0013] The iterative grid partitioning module is used to determine whether each partitionable grid meets the stopping condition based on the remaining expected feature points, the number of partitionable grids, the preset number of pixels, and the preset minimum number of feature points. If yes, the partitionable grids that meet the stopping condition are identified as incomplete grids; otherwise, the partitionable grids that do not meet the stopping condition are identified as grids to be partitioned, and the process of performing N-ary tree partitioning on the grids to be partitioned is returned to obtain N grids, and the type of each grid is determined, until the stopping condition is met. The remaining expected feature points are the difference between the expected feature points corresponding to the initial image and the number of valid grids.

[0014] The target feature point determination module is used to take the feature points to be divided in each valid grid as the first feature points, and determine the second feature points according to the number of the first feature points, the expected number of feature points and each incomplete grid, and determine the first feature points and the second feature points as target feature points.

[0015] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising: one or more processors; a storage device for storing one or more programs; and when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the image feature extraction method as described above.

[0016] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the image feature extraction method described above.

[0017] The image feature extraction method provided in this embodiment extracts edges from an initial image to obtain an edge image corresponding to the initial image. Based on the initial image and the edge image, non-edge pixels in the initial image are identified, and feature points are extracted from these non-edge pixels to obtain feature points to be divided. This method extracts feature points from non-edge pixels, reducing computational load. Furthermore, the initial image is used as a grid to be divided, and this grid is divided into N-ary trees to obtain N grids. The type of each grid is determined. Based on the remaining expected feature points, the number of divisible grids, the preset number of pixels, and the preset minimum number of feature points, it is determined whether each divisible grid meets the stopping condition. If so, the divisible grids that meet the stopping condition are identified as incomplete grids. If not, the divisible grids that do not meet the stopping conditions are taken as grids to be divided, and the process is returned to perform N-ary tree partitioning on the grids to be divided, resulting in N grids. The type of each grid is determined until the stopping conditions are met. Various stopping conditions are used to improve the N-ary tree partitioning strategy and enhance the consistency between the feature point distribution and the image spatial distribution. Furthermore, the feature points to be divided in each valid grid are taken as the first feature points, and the second feature points are determined based on the number of first feature points, the expected number of feature points, and each incomplete grid. The first and second feature points are determined as the target feature points, which improves the efficiency of feature point extraction and enhances the consistency between the feature point distribution and the image spatial distribution while ensuring the uniformity of the feature point distribution extracted in the image. Attached Figure Description

[0018] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0019] Figure 1 This is a flowchart of an image feature extraction method according to an embodiment of the present disclosure;

[0020] Figure 2 This is a schematic diagram of a judgment process in one embodiment of the present disclosure;

[0021] Figure 3 This is a flowchart of a point and line feature extraction method in the prior art;

[0022] Figure 4 This is a flowchart of a point and line feature extraction method according to an embodiment of this disclosure;

[0023] Figure 5(a) is a schematic diagram of the first process of an image feature extraction method in an embodiment of this disclosure;

[0024] Figure 5(b) is a schematic diagram of the second process of an image feature extraction method in an embodiment of this disclosure;

[0025] Figure 5(c) is a schematic diagram of the third process of an image feature extraction method in an embodiment of this disclosure;

[0026] Figure 5(d) is a schematic diagram of the fourth process of an image feature extraction method in an embodiment of this disclosure;

[0027] Figure 5(e) is a schematic diagram of the fifth process of an image feature extraction method in an embodiment of this disclosure;

[0028] Figure 5(f) is a schematic diagram of the sixth process of an image feature extraction method in an embodiment of this disclosure;

[0029] Figure 6 This is a schematic diagram of the structure of an image feature extraction device according to an embodiment of the present disclosure;

[0030] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0031] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0032] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0033] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0034] Common image feature point extraction techniques typically involve first extracting point features from the image, and then performing quadtree uniformization on the extracted feature points. This results in good global uniformity of feature points in the image. However, this often does not match the actual spatial distribution in the image, leading to the loss of local feature points, which seriously affects the accuracy and robustness of subsequent mapping and tracking.

[0035] To address the aforementioned issues, this disclosure provides an image feature extraction method that, while ensuring the uniformity of the distribution of feature points extracted from the image, improves the consistency between the feature point distribution and the spatial distribution of the image.

[0036] Figure 1 This is a flowchart illustrating an image feature extraction method according to an embodiment of this disclosure. The method can be executed by an image feature extraction device, which can be implemented in software and / or hardware, and can be configured in an electronic device. Figure 1 As shown, the method may specifically include the following steps:

[0037] S110. Perform edge extraction on the initial image to obtain an edge image corresponding to the initial image. Based on the initial image and the edge image, determine the non-edge pixels in the initial image and extract feature points from the non-edge pixels to obtain the feature points to be divided.

[0038] The initial image is the image to be used for feature extraction. The edge image is the initial image after edge extraction. Non-edge pixels are the pixels in the initial image other than the edge pixels in the edge image. Feature points to be segmented are the feature points extracted from the non-edge pixels, used for subsequent filtering.

[0039] Specifically, edge extraction is performed on the initial image. For example, operators such as Sobel, Prewitt, Scharr, Robot, and Canny can be used to extract edges, resulting in an edge image. Then, edge pixels in the edge image are removed from the initial image, becoming non-edge pixels. Feature point extraction is then performed on these non-edge pixels. Traditional feature point extraction methods such as SURF (Speeded Up RobustFeatures), SIFT (Scale-Invariant Feature Transform), and ORB (Oriented FAST and Rotated BRIEF) or deep learning methods like SuperPoint can be used for feature point extraction. These extracted feature points are then used as the feature points for segmentation.

[0040] S120. Take the initial image as the grid to be divided, and perform N-ary tree partitioning on the grid to be divided to obtain N grids, and determine the type of each grid.

[0041] The grid to be divided is the grid for subsequent N-ary tree partitioning. N-ary tree partitioning means dividing the image into... The grid is defined by N, where N is a positive integer, and each grid cell is the same size. For example, a quadtree divides an image into a 2×2 grid with four identical grid cells. The type is valid, invalid, or divisible; valid indicates that the grid contains exactly one feature point to be divided; invalid indicates that the grid does not contain any feature points to be divided; divisible indicates that the grid contains at least two feature points to be divided.

[0042] Specifically, the initial image is used as a grid to be divided and subjected to N-ary tree partitioning, resulting in N grids of equal size. Then, the type of each grid is determined based on the number of feature points to be divided within each grid.

[0043] S130. Based on the remaining expected feature points, the number of divisible grids, the preset number of pixels, and the preset minimum number of feature points, determine whether each divisible grid meets the stopping condition. If yes, the divisible grids that meet the stopping condition are identified as incomplete grids. If no, the divisible grids that do not meet the stopping condition are identified as grids to be divided. Return to the process of dividing the grids to be divided into N-ary trees to obtain N grids and determine the type of each grid until the stopping condition is met.

[0044] The preset pixel count is the minimum number of pixels in each grid to avoid losing local repeatability. The preset minimum feature point count is the minimum number of feature points in each grid to ensure that each grid has a certain number of obvious feature points and ensure local features. The stopping condition is used to determine whether a separable grid needs to be further subdivided. An incomplete grid is a separable grid that meets the stopping condition and will not be further subdivided. The remaining expected feature point count is the difference between the expected feature point count in the initial image and the number of valid grids. Each valid grid contains one and only one feature point to be subdivided; therefore, the difference between the expected feature point count and the number of valid grids can be considered as the number of remaining expected feature points that still need to be determined. The remaining expected feature points represent the remaining expected feature points that can be determined within the separable grids. The expected feature point count is the preset number of feature points that need to be determined in the initial image.

[0045] Specifically, for each divisible grid, the following criteria are used to determine whether the stopping condition for dividing the grid is met: the number of remaining expected feature points, the number of divisible grids, the preset number of pixels, and the preset minimum number of feature points. If the condition is met, it means that no further dividing is needed, and the divisible grid is designated as an incomplete grid. If the condition is not met, it means that further dividing is needed. Therefore, the divisible grid can be designated as the grid to be divided, and an N-ary tree partitioning is performed on the grid to be divided to obtain N grids. The type of each grid is determined until the stopping condition for dividing is met.

[0046] Based on the above example, the following methods can be used to determine whether each divisible mesh meets the stopping condition for mesh division:

[0047] If the number of remaining expected feature points is greater than the number of divisible grids, then the number of pixels in each divisible grid is compared with the preset number of pixels.

[0048] If the number of pixels in each divisible grid is not greater than the preset number of pixels, then the product of the preset minimum number of feature points and the number of divisible grids is compared with the remaining expected number of feature points.

[0049] If the product of the preset minimum number of feature points and the number of divisible grids is less than the remaining expected number of feature points, then each divisible grid is determined not to meet the stopping condition. The divisible grids that do not meet the stopping condition are taken as grids to be divided, and the process is returned to perform N-ary tree partitioning on the grids to be divided, resulting in N grids. The type of each grid is determined until the stopping condition is met.

[0050] If the product of the preset minimum number of feature points and the number of divisible grids is not less than the remaining expected number of feature points, then each divisible grid is determined to meet the stopping condition, and the divisible grids that meet the stopping condition are determined to be incomplete grids.

[0051] Specifically, the process involves comparing the remaining expected feature points with the number of divisible grids (the first judgment). If the first judgment indicates that the remaining expected feature points are greater than the number of divisible grids, then there are enough remaining expected feature points for allocation. Therefore, each divisible grid can continue to be judged, specifically by comparing the number of pixels in each divisible grid with a preset number of pixels (the second judgment). If the second judgment indicates that the number of pixels in each divisible grid is not greater than the preset number of pixels, then the divisible grids are small enough, and further judgment is needed to determine whether further division is necessary. This involves comparing the product of the preset minimum number of feature points and the number of divisible grids with the remaining expected feature points (the third judgment). If the third judgment indicates that the product of the preset minimum number of feature points and the number of divisible grids is less than the remaining expected feature points, then the current number of remaining expected feature points is sufficient for division within each divisible grid. Therefore, further division can continue, with the divisible grids treated as grids to be divided. The process then returns to perform an N-ary tree partitioning of these grids, resulting in N grids, and determining the type of each grid, until the stopping condition is met. If the result of the third judgment is that the product of the preset minimum number of feature points and the number of divisible grids is not less than the remaining expected number of feature points, it indicates that the remaining expected number of feature points is too small to be effectively distributed within each divisible grid. Therefore, these divisible grids are determined as incomplete grids.

[0052] It should be noted that the order of dividing each divisible grid is as follows: first, according to the grid area from largest to smallest; if the grid areas are the same, then according to the number of feature points to be divided in the grid from largest to smallest; after each division, it is determined whether the dividing stop condition is met, and the dividing is stopped in time.

[0053] Based on the above example, the following explanation addresses another scenario regarding the outcome of the first judgment:

[0054] If the number of remaining expected feature points is not greater than the number of divisible grids, then each divisible grid is determined to meet the stopping condition, and the divisible grids that meet the stopping condition are determined to be incomplete grids.

[0055] Specifically, if the first judgment result is that the number of remaining expected feature points is not greater than the number of divisible grids, it indicates that in the case of equal division, the number of feature points to be divided in each divisible grid is less than 1. Therefore, it is determined that each divisible grid meets the stopping condition, the division is stopped, and these divisible grids are determined as incomplete grids.

[0056] Based on the above example, the following explanation addresses another scenario regarding the outcome of the second judgment:

[0057] If the number of pixels in each divisible grid is greater than the preset number of pixels, then each divisible grid does not meet the stopping condition. The divisible grids that do not meet the stopping condition are taken as grids to be divided, and the process is returned to perform N-ary tree partitioning on the grids to be divided, resulting in N grids. The type of each grid is then determined until the stopping condition is met.

[0058] Specifically, if the result of the second judgment is that the number of pixels in each divisible grid is greater than the preset number of pixels, it means that the number of pixels in the divisible grid is sufficient to support further division. Therefore, these divisible grids can be used as new grids to be divided and aligned for division. That is, the grids to be divided are divided into N-ary trees to obtain N grids, and the type of each grid is determined until the condition for stopping division is met.

[0059] For example, it can be done through Figure 2 The following schematic diagram illustrates the judgment process. The first judgment condition is that the number of remaining expected feature points is greater than the number of divisible grids; the second judgment condition is that the number of pixels in each divisible grid is not greater than a preset number of pixels; and the third judgment condition is that the product of the preset minimum number of feature points and the number of divisible grids is less than the number of remaining expected feature points.

[0060] Specifically, if the first condition is met, the process continues to check if the second condition is met. If the first condition is not met, the process stops partitioning, meaning each divisible mesh meets the stopping condition and is designated as an incomplete mesh. If the second condition is met, the process continues to check if the third condition is met. If the second condition is not met, partitioning continues, meaning the divisible meshes that do not meet the stopping condition are designated as unpartitioned meshes, and the process returns to perform N-ary tree partitioning on the unpartitioned meshes, resulting in N meshes, and determining the type of each mesh. If the third condition is met, partitioning continues; if the third condition is not met, the process stops partitioning.

[0061] S140. Take the feature points to be divided in each valid grid as the first feature points, and determine the second feature points according to the number of first feature points, the expected number of feature points and each unfinished grid. Then, determine the first feature points and the second feature points as the target feature points.

[0062] In this system, the first feature point is the feature point to be divided within the already divided, valid grid. It can be understood that the number of the first feature points is the same as the number of valid grids. The second feature point is the feature point selected from the remaining feature points within the incomplete grid. The target feature point is the feature point obtained after feature point extraction and selection from the initial image; the number of target feature points is the desired number of feature points.

[0063] Specifically, the feature points to be divided within each valid grid are designated as the first feature points. Then, the difference between the expected number of feature points and the number of the first feature points is used as the number of second feature points within each incomplete grid. Next, within each incomplete grid, the importance of each feature point to be divided is determined, and the feature points with the highest number of second feature points in terms of importance are designated as the second feature points. Finally, the first and second feature points are combined to form the target feature point.

[0064] Based on the above example, the second feature point can be determined according to the number of the first feature points, the expected number of feature points, and each incomplete grid cell in the following way:

[0065] The difference between the expected number of feature points and the number of the first feature points is determined as the total number of the second feature points;

[0066] The ratio of the total number of incomplete feature points in each incomplete grid to the total number of second feature points is determined as the target ratio coefficient;

[0067] For each incomplete grid, the number of second feature points in the incomplete grid is determined based on the number of incomplete feature points in the incomplete grid and the target ratio coefficient. Then, based on the number of second feature points, the second feature points in the incomplete grid are determined from all the incomplete feature points in the incomplete grid.

[0068] The total number of second feature points is the difference between the expected number of feature points and the number of first feature points. The total number of incomplete feature points is the total number of feature points to be divided within each incomplete grid. The target scaling factor is the ratio of the total number of incomplete feature points to the total number of second feature points within each incomplete grid, indicating how many incomplete feature points can be extracted to form a second feature point. The number of second feature points is the number of second feature points that should be extracted within each incomplete grid.

[0069] Specifically, the first feature point can be considered a subset of target feature points. Therefore, the difference between the expected number of feature points and the number of the first feature points is determined as the total number of the second feature points, i.e., the number of the other set of target feature points. This can be understood as determining how many feature points remain to be considered as target feature points. Furthermore, the feature points to be divided within each incomplete grid are defined as incomplete feature points. The ratio of the total number of incomplete feature points to the total number of second feature points is determined as the target proportion coefficient, i.e., how many incomplete feature points correspond to one second feature point. Then, each incomplete grid is analyzed separately, and the product of the number of incomplete feature points within that incomplete grid and the target proportion coefficient is taken as the number of second feature points to be determined within that incomplete grid. Finally, within each incomplete grid, the incomplete feature points are ranked according to their importance, and the incomplete feature points with the highest number of second feature points are designated as the second feature points.

[0070] For example, the target scaling factor can be determined using the following formula:

[0071]

[0072] Where β is the target scaling factor, Q n P represents the total number of incomplete feature points. n This represents the total number of the second feature points.

[0073] The number of second feature points within each incomplete grid is determined using the following formula:

[0074]

[0075] in, Let be the number of second feature points within the i-th incomplete grid. Let be the number of incomplete feature points within the i-th incomplete grid. This is for rounding up.

[0076] Based on the above example, the second feature points within the unfinished grid can be determined from among the unfinished feature points within the unfinished grid according to the number of second feature points:

[0077] Determine the corner response value corresponding to each incomplete feature point within the incomplete grid, arrange the corner response values ​​from largest to smallest, and determine the incomplete feature point corresponding to the corner response value of the second-largest number of feature points as the second feature point within the incomplete grid.

[0078] The corner response value is the value obtained by Harris corner detection.

[0079] Specifically, corner response values ​​are calculated for each incomplete feature point within an incomplete grid, yielding the corner response value for each incomplete grid. Then, the incomplete feature points within the incomplete grid are arranged in descending order of their corner response values, and the incomplete feature point whose corner response value corresponds to the second-highest number of feature points after the arrangement is determined as the second feature point within that incomplete grid.

[0080] Based on the above example, after obtaining the edge image corresponding to the initial image, the target feature lines in the initial image can be further extracted:

[0081] Linear features are extracted from the edge image to obtain the target feature line corresponding to the initial image, and the target feature points and target feature lines are determined as the point and line feature results corresponding to the initial image.

[0082] The target feature line is the line feature in the initial image. The point-line feature result is the feature in the initial image obtained by combining the target feature points and the target feature line.

[0083] Specifically, straight line features are extracted from edge pixels in the edge image using algorithms such as HougLines, LSD (Line Segment Detector), Edge Draw Line, and FLD (Fast Line Detector). These algorithms obtain feature line segments, which are then validated. The validated feature line segments are used as target feature lines. Furthermore, the combination of target feature points and target feature lines is determined as the point-line feature result corresponding to the initial image.

[0084] For point and line feature extraction, Figure 3 This is a flowchart of a point and line feature extraction method in the prior art. Figure 4 This is a flowchart of a point and line feature extraction method according to an embodiment of this disclosure.

[0085] like Figure 3 As shown, the parallel extraction of feature points and feature lines does not sufficiently integrate the two methods and does not consider the correlation between the two feature extraction steps, leading to increased processing time. Furthermore, the quadtree uniformity strategy used in feature point extraction overemphasizes uniform distribution on the image, ignoring the actual feature distribution in space. This results in inconsistencies between the image distribution and the actual distribution, as well as suppression of local features, ultimately causing a decrease in system accuracy or operational abnormalities.

[0086] like Figure 4As shown, by using a deep fusion point and line feature extraction step, feature points are extracted from non-edge pixels, effectively reducing the computational load. Furthermore, an improved quadtree optimization method is used to achieve feature point homogenization while fully considering their true spatial distribution, preserving effective local features, and improving subsequent tracking accuracy and robustness.

[0087] For example, suppose the preset pixel count is 160x90 and the preset minimum feature point count is 4. The initial image yields 46 feature points to be divided and 34 expected feature points, with a pixel count of 1280x720. First, the initial image in Figure 5(a) is divided into regions for the first time, resulting in Figure 5(b). At this point, the number of valid grids is 0, the number of invalid grids is 0, the number of divisible grids is 4, the number of feature points to be divided within the divisible grids is 46, and the number of remaining expected feature points within the divisible grids is 34. The first judgment condition is met, but the second judgment condition is not met. The four divisible grids are then divided, resulting in Figure 5(c). At this point, the number of valid grids is 9, the number of invalid grids is 2, the number of divisible grids is 5, the number of feature points to be divided within the divisible grids is 37, and the number of remaining expected feature points within the divisible grids is 25. Currently, the first condition is met, but the second condition is not. The five divisible grids are then divided, resulting in Figure 5(d). At this point, there are 19 valid grids, 9 invalid grids, 3 divisible grids, 27 undivided feature points within the divisible grids, and 15 remaining expected feature points within the divisible grids. Currently, both the first and second conditions are met, with grid 1 satisfying the third condition. After dividing grid 1 in Figure 5(d), the third condition is not met, so the division of grids 2 and 3 in Figure 5(d) is stopped, resulting in Figure 5(e). At this point, there are 19 valid grids, 9 invalid grids, 6 divisible grids, 27 undivided feature points within the divisible grids, and 15 remaining expected feature points within the divisible grids. The number of feature points in the undivided grids is calculated, and the target scaling factor is 1.8. Furthermore, the number of second feature points in incomplete grids 1 to 6 in Figure 5(e) was calculated to be 4, 3, 3, 2, 2, and 1, respectively. Based on the number of second feature points in each incomplete grid, the incomplete feature points in each completed grid were sorted and selected according to their corner response values, resulting in Figure 5(f).

[0088] The image feature extraction method provided in this embodiment extracts edges from an initial image to obtain an edge image corresponding to the initial image. Based on the initial image and the edge image, non-edge pixels in the initial image are identified, and feature points are extracted from these non-edge pixels to obtain feature points to be divided. This method extracts feature points from non-edge pixels, reducing computational load. Furthermore, the initial image is used as a grid to be divided, and this grid is partitioned into N-ary trees to obtain N grids. The type of each grid is determined. Based on the remaining expected feature points, the number of divisible grids, the preset number of pixels, and the preset minimum number of feature points, it is determined whether each divisible grid meets the stopping condition. If so, the divisible grids that meet the stopping condition are identified as incomplete grids. If not, the divisible grids that do not meet the stopping conditions are taken as grids to be divided, and the process is returned to perform N-ary tree partitioning on the grids to be divided, resulting in N grids. The type of each grid is determined until the stopping conditions are met. Various stopping conditions are used to improve the N-ary tree partitioning strategy and enhance the consistency between the feature point distribution and the image spatial distribution. Furthermore, the feature points to be divided in each valid grid are taken as the first feature points, and the second feature points are determined based on the number of first feature points, the expected number of feature points, and each incomplete grid. The first and second feature points are then determined as the target feature points, which improves the efficiency of feature point extraction and enhances the consistency between the feature point distribution and the image spatial distribution while ensuring the uniformity of the feature point distribution extracted in the image.

[0089] Figure 6 This is a schematic diagram of the structure of an image feature extraction device according to an embodiment of this disclosure. Figure 6 As shown: The device includes: a feature point extraction module 610, a preliminary mesh division module 620, an iterative mesh division module 630, and a target feature point determination module 640.

[0090] The module 610 for extracting feature points to be divided is used to extract edges from the initial image to obtain an edge image corresponding to the initial image, and to determine non-edge pixels in the initial image based on the initial image and the edge image, and to extract feature points from the non-edge pixels to obtain feature points to be divided; the module 620 for preliminary grid division is used to take the initial image as a grid to be divided, and to perform N-ary tree division on the grid to be divided to obtain N grids, and to determine the type of each grid; wherein, the type is valid, invalid, or divisible; a valid grid indicates that there is one and only one feature point to be divided in the grid; an invalid grid indicates that there is no feature point to be divided in the grid; a divisible grid indicates that there are at least two feature points to be divided in the grid; the iterative grid division module 630 is used to divide the grid according to the number of remaining expected feature points, the number of divisible grids, and a preset pixel value. The system calculates the number of feature points and a preset minimum number of feature points, determines whether each divisible grid meets the stopping condition, and if so, identifies the divisible grids that meet the stopping condition as incomplete grids; otherwise, it identifies the divisible grids that do not meet the stopping condition as grids to be divided, and returns to perform N-ary tree partitioning on the grids to be divided to obtain N grids and determine the type of each grid until the stopping condition is met; wherein, the remaining expected feature point number is the difference between the expected feature point number corresponding to the initial image and the number of valid grids; the target feature point determination module 640 is used to take the feature points to be divided in each valid grid as the first feature point, and determine the second feature point according to the number of the first feature point, the expected feature point number, and each incomplete grid, and determine the first feature point and the second feature point as the target feature point.

[0091] Based on the above example, optionally, after obtaining the edge image corresponding to the initial image, the method further includes: a point-line feature fusion module, used to extract straight line features from the edge image to obtain a target feature line corresponding to the initial image, and to determine the target feature points and the target feature line as point-line feature results corresponding to the initial image.

[0092] Based on the above example, optionally, the iterative mesh partitioning module 630 is further configured to: if the number of remaining expected feature points is greater than the number of subdivisible meshes, compare the number of pixels in each subdivisible mesh with the preset number of pixels; if the number of pixels in each subdivisible mesh is not greater than the preset number of pixels, compare the product of the preset minimum number of feature points and the number of subdivisible meshes with the number of remaining expected feature points; if the product of the preset minimum number of feature points and the number of subdivisible meshes is less than the number of remaining expected feature points, determine that each subdivisible mesh does not meet the stopping partitioning condition, and designate the subdivisible meshes that do not meet the stopping partitioning condition as meshes to be partitioned, and return to perform N-ary tree partitioning on the meshes to be partitioned to obtain N meshes, and determine the type of each mesh, until the stopping partitioning condition is met; if the product of the preset minimum number of feature points and the number of subdivisible meshes is not less than the number of remaining expected feature points, determine that each subdivisible mesh meets the stopping partitioning condition, and designate the subdivisible meshes that meet the stopping partitioning condition as incomplete meshes.

[0093] Based on the above example, optionally, the iterative mesh division module 630 is further configured to determine that each divisible mesh satisfies the stopping division condition if the number of remaining expected feature points is not greater than the number of divisible meshes, and to determine the divisible meshes that satisfy the stopping division condition as incomplete meshes.

[0094] Based on the above example, optionally, the iterative grid division module 630 is further configured to determine that each divisible grid does not meet the stopping division condition if the number of pixels of each divisible grid is greater than the preset number of pixels, and to take the divisible grid that does not meet the stopping division condition as the grid to be divided, and return to perform the N-ary tree division on the grid to be divided to obtain N grids, and determine the type of each grid, until the stopping division condition is met.

[0095] Based on the above example, optionally, the target feature point determination module 640 is further configured to determine the difference between the expected number of feature points and the number of the first feature points as the total number of the second feature points; determine the ratio of the total number of unfinished feature points in each unfinished grid to the total number of the second feature points as a target ratio coefficient; for each unfinished grid, determine the number of the second feature points in the unfinished grid according to the number of unfinished feature points in the unfinished grid and the target ratio coefficient, and determine the second feature points in the unfinished grid from each unfinished feature point in the unfinished grid according to the number of the second feature points.

[0096] Based on the above example, optionally, the target feature point determination module 640 is further configured to determine the corner response value corresponding to each unfinished feature point in the unfinished grid, arrange the corner response values ​​from largest to smallest, and determine the unfinished feature point corresponding to the corner response value of the first second number of feature points as the second feature point in the unfinished grid.

[0097] The image feature extraction apparatus provided in this disclosure embodiment can execute the steps in the image feature extraction method provided in this disclosure method embodiment, and has the execution steps and beneficial effects, which will not be repeated here.

[0098] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 7 It shows a schematic diagram of the structure suitable for implementing the electronic device 700 in the embodiments of this disclosure. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0099] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes to implement the methods of the embodiments described herein, based on a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing device 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0100] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the image feature extraction method as described above. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this disclosure.

[0101] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0102] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0103] Edge extraction is performed on the initial image to obtain an edge image corresponding to the initial image. Based on the initial image and the edge image, non-edge pixels in the initial image are determined, and feature points are extracted from the non-edge pixels to obtain feature points to be segmented.

[0104] The initial image is used as the grid to be divided, and the grid to be divided is divided into N-ary trees to obtain N grids. The type of each grid is determined; wherein, the type is valid, invalid, or divisible; a valid grid means that there is one and only one feature point to be divided in the grid; an invalid grid means that there is no feature point to be divided in the grid; a divisible grid means that there are at least two feature points to be divided in the grid.

[0105] Based on the remaining expected feature points, the number of divisible grids, the preset number of pixels, and the preset minimum number of feature points, it is determined whether each divisible grid meets the stopping condition. If so, the divisible grids that meet the stopping condition are identified as incomplete grids. If not, the divisible grids that do not meet the stopping condition are identified as grids to be divided, and the process of performing N-ary tree partitioning on the grids to be divided is returned to obtain N grids, and the type of each grid is determined, until the stopping condition is met. Here, the remaining expected feature points are the difference between the expected feature points corresponding to the initial image and the number of valid grids.

[0106] The feature points to be divided within each valid grid are taken as the first feature points, and the second feature points are determined based on the number of the first feature points, the expected number of feature points, and each incomplete grid. The first feature points and the second feature points are then determined as the target feature points.

[0107] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0108] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0109] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. An image feature extraction method, characterized in that, The method includes: Edge extraction is performed on the initial image to obtain an edge image corresponding to the initial image. Based on the initial image and the edge image, non-edge pixels in the initial image are determined, and feature points are extracted from the non-edge pixels to obtain feature points to be segmented. The initial image is used as the grid to be divided, and the grid to be divided is divided into N-ary trees to obtain N grids. The type of each grid is determined. The type is valid, invalid, or divisible grid. Valid means that there is one and only one feature point to be divided in the grid. Invalid means that there is no feature point to be divided in the grid. Divisible means that there are at least two feature points to be divided in the grid. If the number of remaining expected feature points is greater than the number of divisible grids, the number of pixels in each divisible grid is compared with a preset number of pixels. If the number of pixels in each divisible grid is not greater than the preset number of pixels, the product of a preset minimum number of feature points and the number of divisible grids is compared with the number of remaining expected feature points. If the product of the preset minimum number of feature points and the number of divisible grids is less than the number of remaining expected feature points, it is determined that each divisible grid does not meet the stopping condition. The divisible grids that do not meet the stopping condition are designated as grids to be divided, and the process of performing N-ary tree partitioning on the grids to be divided is repeated to obtain N grids, and the type of each grid is determined, until the stopping condition is met. If the product of the preset minimum number of feature points and the number of divisible grids is not less than the number of remaining expected feature points, it is determined that each divisible grid meets the stopping condition. The divisible grids that meet the stopping condition are designated as incomplete grids. The number of remaining expected feature points is the difference between the number of expected feature points corresponding to the initial image and the number of valid grids. The feature points to be divided within each valid grid are taken as the first feature points, and the second feature points are determined based on the number of the first feature points, the expected number of feature points, and each incomplete grid. The first feature points and the second feature points are then determined as the target feature points.

2. The method according to claim 1, characterized in that, After obtaining the edge image corresponding to the initial image, the method further includes: Linear features are extracted from the edge image to obtain target feature lines corresponding to the initial image, and the target feature points and the target feature lines are determined as point-line feature results corresponding to the initial image.

3. The method according to claim 1, characterized in that, Also includes: If the number of remaining expected feature points is not greater than the number of divisible grids, then each divisible grid is determined to meet the stopping condition, and the divisible grids that meet the stopping condition are determined to be incomplete grids.

4. The method according to claim 1, characterized in that, Also includes: If the number of pixels in each divisible grid is greater than the preset number of pixels, then each divisible grid does not meet the stopping condition. The divisible grid that does not meet the stopping condition is taken as a grid to be divided, and the process of performing N-ary tree partitioning on the grid to be divided is returned to obtain N grids, and the type of each grid is determined, until the stopping condition is met.

5. The method according to claim 1, characterized in that, The step of determining the second feature point based on the number of the first feature points, the expected number of feature points, and each incomplete grid includes: The difference between the expected number of feature points and the number of the first feature points is determined as the total number of the second feature points; The ratio of the total number of incomplete feature points in each incomplete grid to the total number of the second feature points is determined as the target ratio coefficient; For each incomplete grid, the number of second feature points in the incomplete grid is determined based on the number of incomplete feature points in the incomplete grid and the target ratio coefficient, and the second feature points in the incomplete grid are determined from each incomplete feature point in the incomplete grid based on the number of second feature points.

6. The method according to claim 5, characterized in that, The step of determining the second feature point within the incomplete grid from among the incomplete feature points within the incomplete grid based on the number of the second feature points includes: Determine the corner response value corresponding to each incomplete feature point within the incomplete grid, arrange the corner response values ​​from largest to smallest, and determine the incomplete feature point corresponding to the corner response value of the first second number of feature points as the second feature point within the incomplete grid.

7. An image feature extraction device, characterized in that, include: The feature point extraction module is used to extract edges from the initial image to obtain an edge image corresponding to the initial image, and to determine non-edge pixels in the initial image based on the initial image and the edge image, and to extract feature points from the non-edge pixels to obtain the feature points to be divided. The preliminary grid division module is used to take the initial image as the grid to be divided, and to perform N-ary tree partitioning on the grid to be divided to obtain N grids, and to determine the type of each grid; wherein, the type is a valid, invalid, or divisible grid; a valid grid indicates that there is one and only one feature point to be divided in the grid; an invalid grid indicates that there is no feature point to be divided in the grid; a divisible grid indicates that there are at least two feature points to be divided in the grid. An iterative grid partitioning module is used to: if the number of remaining expected feature points is greater than the number of grids that can be partitioned, compare the number of pixels in each partitionable grid with a preset number of pixels; if the number of pixels in each partitionable grid is not greater than the preset number of pixels, compare the product of a preset minimum number of feature points and the number of partitionable grids with the number of remaining expected feature points; if the product of the preset minimum number of feature points and the number of partitionable grids is less than the number of remaining expected feature points, determine that each partitionable grid does not meet the stopping partitioning condition, designate the partitionable grids that do not meet the stopping partitioning condition as grids to be partitioned, and return to perform N-ary tree partitioning on the grids to be partitioned to obtain N grids and determine the type of each grid, until the stopping partitioning condition is met; if the product of the preset minimum number of feature points and the number of partitionable grids is not less than the number of remaining expected feature points, determine that each partitionable grid meets the stopping partitioning condition, and designate the partitionable grids that meet the stopping partitioning condition as incomplete grids; wherein, the number of remaining expected feature points is the difference between the number of expected feature points corresponding to the initial image and the number of valid grids. The target feature point determination module is used to take the feature points to be divided in each valid grid as the first feature points, and determine the second feature points according to the number of the first feature points, the expected number of feature points and each incomplete grid, and determine the first feature points and the second feature points as target feature points.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image feature extraction method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the image feature extraction method as described in any one of claims 1-6.

Images