Feature point extraction method, image reconstruction method and device

Abstract

The present application provides a method for extracting feature points, a method for reconstructing an image, and a device. The method for extracting feature points includes: determining the actual grid size based on the size of an input image and the expected number of feature points n; dividing the input image into grids based on the actual grid size to obtain n grids, where n is an integer greater than 1; extracting feature points from each of the n grids based on an initial threshold to obtain g feature points for the grid; when g is greater than K, determining K feature points with the highest response values ​​among the g feature points as feature points of the grid, where K is the number of feature points to be extracted for each grid. The technical solution of the present application can obtain uniformly distributed high-quality feature points.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for extracting feature points, and a method and device for reconstructing an image. Background Art

[0002] In computer vision, the image processing process is an extremely important link, and in the image processing process, the extraction of feature points is a crucial step. Since feature points can represent the content of the image, feature point extraction is widely used in the fields of moving target tracking, object recognition, image registration, panoramic image stitching, 3D reconstruction, etc. The commonly used feature point extraction methods are prone to uneven distribution of feature points and low quality of feature points, and the distribution and quality of feature points will directly affect the accuracy of subsequent image processing results. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a method for extracting feature points, a method for reconstructing an image, and a device, which can obtain uniformly distributed high-quality feature points.

[0004] In a first aspect, an embodiment of the present application provides a method for extracting feature points, comprising: determining an actual grid size based on the size of an input image and an expected number of feature points n; gridding the input image based on the actual grid size to obtain n grids, where n is an integer greater than 1; extracting feature points from each of the n grids based on an initial threshold to obtain g feature points for the grid; when g is greater than K, determining K feature points with higher response values ​​among the g feature points as feature points of the grid, where K is the number of feature points that need to be extracted for each grid.

[0005] In certain embodiments of the present application, feature points are extracted from each of n grids based on an initial threshold to obtain g feature points for the grid, including: extracting feature points from the grid based on the initial threshold to obtain f feature points; when the number of f feature points is greater than a threshold value, obtaining a first update threshold based on a first rule and the initial threshold, wherein the first update threshold is greater than the initial threshold; and re-extracting feature points from the grid based on the first update threshold to obtain g feature points.

[0006] In certain embodiments of the present application, feature points are extracted from each of n grids based on an initial threshold to obtain g feature points for the grid, and the method also includes: when the number of f feature points is 0, a second update threshold is obtained based on a second rule and the initial threshold, wherein the second update threshold is smaller than the initial threshold; and feature points are re-extracted from the grid based on the second update threshold to obtain g feature points.

[0007] In some embodiments of the present application, the first update threshold and / or the second update threshold is within a preset threshold range.

[0008] In certain embodiments of the present application, the feature point extraction method also includes: determining the number of feature points K that need to be extracted from the grid based on the actual grid size, the size of the input image, and the expected number of feature points n; when g is less than or equal to K, determining g feature points as the feature points of the grid.

[0009] In certain embodiments of the present application, the feature point extraction method further includes: determining the total number of feature points of n grids, and when the total number is greater than the expected number of feature points n, determining n feature points with the highest response values ​​among the feature points of the n grids as the final feature points.

[0010] In second aspect, an embodiment of the present application provides a feature matching method, comprising: extracting feature points in a first image using the feature point extraction method described in the first aspect, and calculating a first feature descriptor corresponding to the feature points in the first image; extracting feature points in a second image using the feature point extraction method described in the first aspect, and calculating a second feature descriptor corresponding to the feature points in the second image; performing feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result.

[0011] In a third aspect, an embodiment of the present application provides an image reconstruction method, comprising: extracting feature points in a first image using the feature point extraction method described in the first aspect, and calculating a first feature descriptor corresponding to the feature points in the first image; extracting feature points in a second image using the feature point extraction method described in the first aspect, and calculating a second feature descriptor corresponding to the feature points in the second image; performing feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result; and reconstructing the object to be reconstructed based on the matching result.

[0012] In certain embodiments of the present application, feature matching is performed on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result, including: matching the first feature descriptor and the second feature descriptor using an approximate nearest neighbor ratio method to obtain a plurality of initial matching pairs; calculating a motion model based on at least one matching pair among the plurality of initial matching pairs; and taking the matching pairs among the plurality of initial matching pairs that satisfy the motion model as the matching result.

[0013] In a fourth aspect, an embodiment of the present application provides a feature point extraction device, comprising: a first determination module, used to determine the actual grid size based on the size of the input image and the expected number of feature points n; a division module, used to divide the input image into grids based on the actual grid size to obtain n grids, wherein n is an integer greater than 1; an extraction module, used to extract feature points from each of the n grids based on an initial threshold to obtain g feature points for the grid; a second determination module, used to determine, when g is greater than K, K feature points with higher response values ​​among the g feature points as feature points of the grid, wherein K is the number of feature points to be extracted for each grid.

[0014] In a fifth aspect, an embodiment of the present application provides a feature matching device, comprising: a first extraction module, used to extract feature points in a first image using the feature point extraction method described in the first aspect, and calculate a first feature descriptor corresponding to the feature points in the first image; a second extraction module, used to extract feature points in a second image using the feature point extraction method described in the first aspect, and calculate a second feature descriptor corresponding to the feature points in the second image; a matching module, used to perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result.

[0015] In a sixth aspect, an embodiment of the present application provides an image reconstruction device, comprising: a first extraction module, used to extract feature points in a first image using the feature point extraction method described in the first aspect, and calculate a first feature descriptor corresponding to the feature points in the first image; a second extraction module, used to extract feature points in a second image using the feature point extraction method described in the first aspect, and calculate a second feature descriptor corresponding to the feature points in the second image; a matching module, used to perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result; and a reconstruction module, used to reconstruct the object to be reconstructed based on the matching result.

[0016] In the seventh aspect, an embodiment of the present application provides an unmanned device, comprising: a processor; a memory for storing processor executable instructions, wherein the processor is used to execute the feature point extraction method described in the first aspect, or execute the feature matching method described in the second aspect, or execute the image reconstruction method described in the third aspect.

[0017] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the feature point extraction method described in the first aspect, or execute the feature matching method described in the second aspect, or execute the image reconstruction method described in the third aspect.

[0018] The embodiment of the present application provides a method for extracting feature points, a method for reconstructing an image, and a device, which can meet the needs of different application scenarios by using an adaptive grid size instead of a fixed grid size in a traditional method. In addition, the adaptive grid size can ensure that the number of grids is consistent with the number of expected feature points, and combined with the process of screening feature points based on the response value, it is convenient to extract K (for example, one or two) high-quality feature points for each grid, avoiding the deletion of too many high-quality points for the same area in the traditional method, thereby improving the quality of the feature points while ensuring that the feature points are evenly distributed. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Shown is a schematic diagram of the system architecture of a feature point extraction system provided by an exemplary embodiment of the present application.

[0020] Figure 2 Shown is a flow chart of a method for extracting feature points provided by an exemplary embodiment of the present application.

[0021] Figure 3 Shown is a flow chart of a feature point extraction method provided by another exemplary embodiment of the present application.

[0022] Figure 4 Shown is a flow chart of a feature matching method provided by an exemplary embodiment of the present application.

[0023] Figure 5 Shown is a flowchart of an image reconstruction method provided by an exemplary embodiment of the present application.

[0024] Figure 6a The figure shows a schematic diagram of an extraction result obtained by using a traditional feature point extraction method.

[0025] Figure 6b Shown is a schematic diagram of an extraction result obtained by using a feature point extraction method provided by an embodiment of the present application.

[0026] Figure 7a Shown is a schematic diagram of another extraction result obtained by using a traditional feature point extraction method.

[0027] Figure 7b Shown is a schematic diagram of another extraction result obtained by using the feature point extraction method provided by an embodiment of the present application.

[0028] Figure 8 Shown is a schematic structural diagram of a feature point extraction device provided by an exemplary embodiment of the present application.

[0029] Fig. 9 Shown is a schematic structural diagram of a feature matching device provided by an exemplary embodiment of the present application.

[0030] Fig.10 Shown is a schematic structural diagram of an image reconstruction device provided by an exemplary embodiment of the present application.

[0031] Fig.11 Shown is a block diagram of an unmanned device for executing a feature point extraction method provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] Application Overview

[0034] Farmland mapping and reconstruction generally involves extracting feature points from input images, generating data associations between multiple frames based on the feature points, estimating relative motion between frames (images), and finally using Bundle Adjustment to optimize camera posture and three-dimensional scene structure.

[0035] Assuming that the expected number of feature points is n, the commonly used feature point extraction method is to use a unified threshold to extract feature points from the entire image, then sort all the extracted feature points from large to small according to the response value, and finally extract the first n feature points from the sorting results, and calculate the feature descriptors of the first n feature points as the final feature output.

[0036] The quadtree-based grid feature point extraction method divides the image into multiple grids of fixed size, extracts feature points from each grid, and performs quadtree division on all extracted feature points according to the pixel positions of the feature points until the final number of nodes is greater than or equal to the expected number of feature points n, at which point the division is terminated. For nodes with a final number of feature points greater than 1, the feature points are sorted according to the size of the response value in the node, and only the feature point with the highest response value is taken, and the remaining feature points in the node are eliminated. The feature descriptor is calculated for the n feature points finally selected as the final feature output.

[0037] For scenes with uneven texture distribution, traditional feature point extraction methods can extract a large number of feature points in areas with rich textures, but very few feature points in areas with weak textures or repeated textures. At this time, when the expected number of feature points is selected according to the response value, the problem of feature point clustering will occur, which will lead to the final feature matching result being clustered. The quadtree-based grid feature point extraction method is to divide the image into multiple grids according to a fixed grid size (the number of grids is much larger than the expected number of feature points), extract feature points for each grid, and then use the quadtree distribution method to select the expected number of feature points, so that the selected feature points can be evenly distributed on the image. However, the quadtree can only ensure that the distribution of feature points is as uniform as possible, but it cannot ensure that the selected feature points are all high-quality feature points. Specifically, in the quadtree-based grid feature point extraction method, the fixed grid size is pre-set and is determined by the computational complexity and uniformity requirements. The number of grids obtained based on the fixed grid size division is much larger than the expected number of feature points, and the quadtree structure will cause high-quality feature points in multiple grids (in the same area) to be divided into the same node, so that most of the high-quality feature points in the node will be deleted in the final screening stage, and only one feature point will be retained, resulting in the problem of low quality of the extracted feature points.

[0038] That is, for scenes with uneven texture distribution, it is difficult for traditional feature point extraction methods to achieve a balance between feature point quality and feature point distribution uniformity, and it is difficult to improve the quality of extracted feature points while satisfying distribution uniformity.

[0039] In order to solve the above technical problems, the present application proposes the following feature point extraction system.

[0040] Exemplary Systems

[0041] Figure 1 FIG. 1 is a schematic diagram of a system architecture of a feature point extraction system 100 provided by an exemplary embodiment of the present application, which illustrates an application scenario of extracting feature points from an image. The feature point extraction system 100 includes a computer device 110 and an image acquisition device 120 .

[0042] The image acquisition device 120 can capture a specific scene to obtain one or more images. Here, the specific scene can be a farmland, a road, a building, etc.

[0043] The computer device 110 may acquire one or more images from the image acquisition device 120 and extract feature points from the acquired images. The specific extraction method may be found in the description below.

[0044] The computer device 110 may be a general-purpose computer or a computer device composed of a dedicated integrated circuit, etc., which is not limited in the embodiments of the present application. For example, the computer device 110 may be a mobile terminal device such as a tablet computer, or may also be a personal computer (PC), such as a laptop computer and a desktop computer, etc. Those skilled in the art will appreciate that the number of the above-mentioned computer devices 110 may be one or more, and their types may be the same or different. The embodiments of the present application do not limit the number and device type of the computer devices 110.

[0045] The image acquisition device 120 may be a camera, for example, a monocular camera, a binocular camera, or a trinocular camera, etc. In one embodiment, the image acquisition device 120 may be a camera installed on a drone.

[0046] In some embodiments, the computer device 110 and the image acquisition device 120 may be two independent devices, and the computer device 110 may be communicatively connected with one or more image acquisition devices 120. In some other embodiments, the image acquisition device 120 may be integrated into the computer device 110, and the image acquisition device 120 integrated into the computer device 110 may be one or more.

[0047] It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the embodiments of the present application are not limited thereto. On the contrary, the embodiments of the present application can be applied to any scenario that may be applicable.

[0048] Exemplary Methods

[0049] Figure 2 Shown is a flow chart of a method for extracting feature points provided by an exemplary embodiment of the present application. Figure 2 The method can be performed by a computer device. Figure 2 As shown, the feature point extraction method includes the following contents.

[0050] 210: Determine the actual grid size based on the size of the input image and the expected number of feature points n.

[0051] In this embodiment, the actual grid size (grid size, such as the side length or area of ​​the grid) is determined based on the size of the input image (the size of the input image) and the expected number of feature points. When the size of the input image and / or the expected number of feature points changes, the actual grid size can be adaptively adjusted to achieve grid size adaptation.

[0052] For example, if the number of rows and columns of the input image is r, then the number of grids in each row, rGrid, is r / W, the number of grids in each column, cGrid, is c / W, and the total number of grids, nGrid, is rGrid×cGrid, where W is the side length of the grid. For example, each grid size can be W×W, The theoretical grid size is equal to the size of the input image divided by the expected number of feature points, in the expectation of extracting one feature point from each grid. Assuming that the size of the input image is 10×10 (the unit can be, for example, pixels, the unit is omitted here), and the expected number of feature points is 3, the theoretical grid size is 100 / 3. Since the theoretical grid size is an infinite decimal, in order to facilitate grid division, in actual operation, the theoretical grid size can be rounded down, that is, the actual grid size is 33, that is, W×W=33. At this time, the total number of grids nGrid is 100 / 33, then the average number of feature points S to be extracted from each grid is nGrid / n, that is, S=100 / (33×3), which is approximately equal to 1.01, which is equivalent to the minimum number of feature points to be extracted from each grid is 1.01. Since the actual number of feature points extracted is an integer, S can be rounded up to obtain the actual number of feature points K to be extracted from each grid as 2, to ensure K×nGrid≥n.

[0053] In this embodiment, the number of grids actually divided is equal to the expected number of feature points n, wherein the average number of feature points S to be extracted from each grid is usually floating point data. When the average number of feature points S to be extracted from each grid is an integer, the actual grid size is equal to the theoretical grid size, so that the number of feature points to be extracted from each grid is 1 (K=1). By dividing the input image into grids and extracting a feature point from each grid, it can be ensured that all feature points finally extracted are evenly distributed in the image or scene.

[0054] When the average number of feature points S to be extracted from each grid is a decimal, the actual grid size is smaller than the theoretical grid size, which is equivalent to discarding some places on the edge of the input image when the input image is gridded. At this time, since the actual grid is smaller, the number of feature points shared by each grid increases, that is, the number of feature points that need to be extracted from each grid is 2 (K=2). By gridding the input image and extracting two feature points from each grid, it can be ensured that all the feature points finally extracted are evenly distributed in the image or scene.

[0055] Since the actual grid size in this embodiment is adaptively adjusted based on different input data (the size of the input image and / or the expected number of feature points), it can also be called an adaptive grid size.

[0056] In another embodiment, in order to facilitate grid division, in actual operation, the actual grid side length W can be made an integer, and then the total number of grids nGrid, the average number of feature points S to be extracted from each grid, and the actual number of feature points K to be extracted from each grid are determined based on the actual grid side length W.

[0057] 220: Grid-divide the input image based on the actual grid size to obtain n grids, where n is an integer greater than 1.

[0058] Specifically, the shape of the grid can be a square, and the size of the grid can be represented by the side length of the square. Square grids with equal side lengths can achieve complete and uniform division of the image, thereby improving the uniformity of distribution of feature points obtained subsequently.

[0059] In some embodiments, the shape of the grid may be a polygon such as a rectangle, a rhombus, or a triangle. In this case, the grid may be divided according to a certain repetition rule to completely divide the entire image into n uniform grids.

[0060] In other embodiments, the shape of the grid may also be other regular or irregular shapes. The specific grid shape may be set according to actual needs, and the embodiments of the present application are not limited to this.

[0061] 230: Extract feature points from each of the n grids based on the initial threshold to obtain g feature points for the grid.

[0062] The feature points in the embodiments of the present application may be scale invariant feature transform (SIFT) feature points, speeded up robust feature (SURF) feature points, ORB (Oriented FAST and Rotated BRIEF) feature points, or other types of feature points.

[0063] The initial threshold (which can be expressed as T0) is pre-set. Feature points are extracted for each grid according to the initial threshold. The initial threshold can be used to measure the difference between a certain pixel point and the surrounding pixels on the image, such as the pixel value difference (grayscale or brightness difference). Specifically, if the difference between the pixel values ​​of a certain pixel point and the surrounding pixels is greater than the initial threshold, the pixel point can be extracted as a feature point.

[0064] Of course, in other embodiments, the specific meaning of the initial threshold and the specific method of extracting the feature points can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0065] In the process of extracting feature points, the number of feature points extracted for a certain grid can be recorded as g.

[0066] 240: When g is greater than K, determine K feature points with top response values ​​among the g feature points as feature points of the grid, where K is the number of feature points to be extracted for each grid.

[0067] Specifically, K may be determined based on the actual grid size, the size of the input image, and the expected number of feature points n.

[0068] Optionally, as another embodiment, K may also be a preset fixed value.

[0069] When K=1, if the number of feature points extracted from a certain grid is 1 (g=1), the extracted feature points are used as the feature points of the grid. If the number of feature points extracted from a certain grid is greater than 1 (g>1), one feature point is selected from the multiple feature points extracted as the feature point of the grid. Specifically, the multiple feature points extracted from the grid can be sorted according to the size of the response value, and the feature point with the largest response value is used as the feature point of the grid. The larger the response value, the higher the quality of the corresponding feature point.

[0070] Here, the response value of the feature point may be the response value of the corner point in the feature extraction and detection technology, for example, Harris response value, Shi-Tomasi response value or other response values.

[0071] Similarly, when K=2, if the number of feature points extracted from a certain grid is K (g=2), the extracted feature points are used as the feature points of the grid. If the number of feature points extracted from a certain grid is greater than K (g>2), two feature points are selected from the multiple feature points extracted as the feature points of the grid. Specifically, the multiple feature points extracted from the grid can be sorted according to the size of the response value, and the two feature points with the top response values ​​are used as the feature points of the grid. The larger the response value, the higher the quality of the corresponding feature point.

[0072] In the quadtree-based grid feature point extraction method, although the extracted feature points have a high distribution uniformity, the number of grids is much larger than the expected number of feature points, resulting in high-quality feature points in multiple grids (in the same area) being divided into the same node, causing most of the high-quality feature points in the node to be deleted in the final screening stage, resulting in the problem of low quality of the extracted feature points. In other words, the traditional method is difficult to strike a balance between the quality of feature points and the uniformity of feature point distribution, and it is difficult to improve the quality and distribution uniformity of the extracted feature points.

[0073] The embodiment of the present application provides a method for extracting feature points, which can meet the needs of different application scenarios by using an adaptive grid size instead of a fixed grid size in a traditional method. In addition, the adaptive grid size can ensure that the number of grids is consistent with the expected number of feature points, and combined with the process of screening feature points based on the response value, it is convenient to extract K (for example, one or two) high-quality feature points for each grid, avoiding the deletion of too many high-quality points for the same area in the quadtree-based method, thereby improving the quality of the feature points while ensuring that the feature points are evenly distributed.

[0074] According to an embodiment of the present application, feature points are extracted from each of n grids based on an initial threshold to obtain g feature points for the grid, including: extracting feature points from the grid based on the initial threshold to obtain f feature points; when the number of f feature points is greater than the threshold value, obtaining a first update threshold based on a first rule and the initial threshold, wherein the first update threshold is greater than the initial threshold; and re-extracting feature points from the grid based on the first update threshold to obtain g feature points.

[0075] In this embodiment, the number of feature points initially extracted from a certain grid according to the initial threshold value T0 is recorded as f. The threshold value can be preset according to actual needs, for example, it can be a value of 5, 10, etc. Optionally, the threshold value can be determined according to the value of K, and specifically can be a multiple of K, such as 5K, 6K, 10K, etc.

[0076] If f is greater than the threshold value, it indicates that the current grid is in a scene with rich textures, such as a city or a scene with many types of crops in farmland. At this time, the threshold needs to be increased to prevent the subsequent feature extraction from taking too much time. That is, the first updated threshold T1 is obtained based on the first rule and the initial threshold T0. Specifically, the first rule can be to take twice the existing threshold as the new threshold, that is, T1 = 2×T0.

[0077] If the number of feature points extracted based on threshold T1 is still greater than the threshold value, a new threshold T2 is obtained based on the first rule and threshold T1. This process is repeated until the number of feature points extracted for the current grid is less than or equal to the threshold value, at which point the number of feature points extracted is recorded as g.

[0078] In this embodiment, the first rule may be other content as long as it can ensure that the first update threshold is greater than the initial threshold.

[0079] By setting the first rule and the threshold value, this embodiment can appropriately increase the threshold value for scenes with richer textures to avoid extracting a large number of feature points, thereby saving computing time and improving computing efficiency.

[0080] Furthermore, feature points are extracted from each of the n grids based on the initial threshold to obtain g feature points for the grid, and the method also includes: when the number of f feature points is 0, a second update threshold is obtained based on a second rule and the initial threshold, wherein the second update threshold is smaller than the initial threshold; and feature points are re-extracted from the grid based on the second update threshold to obtain g feature points.

[0081] If f is 0, it indicates that the current grid is in a scene with weak texture or severe repeated texture, such as wheat fields, corn fields and other farmland scenes. At this time, it is necessary to lower the threshold and relax the extraction conditions to extract feature points in the current grid to ensure the uniformity of the distribution of the feature points of the n grids finally extracted (for example, at least one feature point is distributed in each grid). That is, the second updated threshold T1' is obtained based on the second rule and the initial threshold T0. Specifically, the second rule can be to take half of the existing threshold as the new threshold, that is, T1'=T0 / 2.

[0082] If the number of feature points extracted based on threshold T1' is still 0, a new threshold T2' is obtained based on the second rule and threshold T1'. This process is repeated until the number of feature points extracted for the current grid is greater than 0, at which point the number of feature points extracted is recorded as g.

[0083] In this embodiment, the second rule may be other content as long as it can ensure that the second update threshold is less than the initial threshold.

[0084] By setting the second rule, this embodiment can appropriately lower the threshold for scenes with weak textures or severe repeated textures, so as to extract feature points in the current grid and ensure uniform distribution of the n feature points finally extracted.

[0085] Compared with the traditional method of extracting feature points based on a fixed threshold, the embodiment of the present application can use an adaptive dynamic threshold to extract feature points for each grid, thereby providing sufficient preparation for subsequently obtaining high-quality and evenly distributed feature points.

[0086] According to an embodiment of the present application, the first update threshold and / or the second update threshold is within a preset threshold range.

[0087] Specifically, in the process of adjusting the threshold to extract an appropriate number of feature points, the threshold interval can be set in advance, such as [TA, TB]. This can control the threshold within a reasonable range to avoid the problem of too low quality of extracted feature points due to the threshold being too large or too small, and then causing a low matching success rate in the subsequent feature matching process.

[0088] For example, based on the first rule and the initial threshold T0 (TA<T0<TB), the first updated threshold T1 is obtained. The number of feature points extracted based on T1 is less than or equal to the threshold value, that is, the number of feature points meets the requirements at this time, but T1 is greater than TB. In order to improve the success rate of subsequent feature matching, the threshold can be further adjusted based on T1 at this time, so that the adjusted threshold is within the threshold interval. For example, the middle value of T0 and T1 can be taken as the adjusted threshold (also called the updated threshold).

[0089] For another example, based on the second rule and the initial threshold T0 (TA<T0<TB), the second updated threshold T1' is obtained. The number of feature points extracted based on T1' is greater than 0, that is, the number of feature points meets the requirements at this time, but T1' is less than TA. In order to ensure the quality of feature points and improve the success rate of subsequent feature matching, the threshold can be further adjusted based on T1' so that the adjusted threshold is within the threshold range. For example, the middle value of T1' and T0 can be taken as the adjusted threshold (also called the updated threshold).

[0090] This embodiment uses an adaptive dynamic threshold with a constraint interval (threshold interval) to extract feature points from each grid, thereby obtaining high-quality and evenly distributed feature points.

[0091] In one embodiment, due to setting a threshold interval, it may happen that no feature points can be extracted from a certain grid, which is mainly related to the specific range of the threshold interval.

[0092] In certain embodiments of the present application, the feature point extraction method also includes: determining the number of feature points K that need to be extracted from the grid based on the actual grid size, the size of the input image, and the expected number of feature points n; when g is less than or equal to K, determining g feature points as the feature points of the grid.

[0093] Specifically, if the number of feature points extracted from a certain grid is less than or equal to K, these feature points can be directly used as feature points of the grid for subsequent steps, that is, the feature point screening step can be omitted because these feature points are already high-quality feature points in the grid.

[0094] This embodiment can ensure that the number of feature points extracted from each grid is at most K, so that the distribution uniformity of the feature points finally extracted can be guaranteed as much as possible, while ensuring the quality of the feature points.

[0095] According to an embodiment of the present application, the feature point extraction method also includes: determining the total number of feature points of n grids, and when the total number is greater than the expected number of feature points n, determining n feature points with higher response values ​​among the feature points of the n grids as the final feature points.

[0096] Specifically, when the total number of feature points of n grids is greater than the expected number of feature points n, these feature points can be sorted from large to small according to the response value, and the top n feature points in the sorting result are taken as the final feature points. When the total number of feature points of n grids is less than or equal to the expected number of feature points n, these feature points can be directly taken as the final feature points.

[0097] Here, the response value of the feature point may be a response value of a corner point in feature extraction and detection technology, for example, a Harris response value, a Shi-Tomasi response value or other response values.

[0098] Figure 3 Shown is a flow chart of a feature point extraction method provided by another exemplary embodiment of the present application. Figure 3 The embodiment is Figure 2 In order to avoid repetition, the same parts of the examples in the embodiments will not be described again. Figure 3 As shown, the feature point extraction method includes the following contents.

[0099] 310: Determine the actual grid size based on the size of the input image and the expected number of feature points n.

[0100] 320: Grid-dividing the input image based on the actual grid size to obtain n grids, where n is an integer greater than 1.

[0101] 330: Extract feature points from each grid based on the initial threshold to obtain f feature points corresponding to the grid.

[0102] 340: When the number of feature points is greater than the threshold value, an updated threshold value is obtained based on the first rule, the initial threshold value, and the threshold value interval.

[0103] In this step, the updated threshold is greater than the initial threshold. The meaning of the first rule, the threshold interval, and the process of obtaining the updated threshold can refer to the description in the above embodiment.

[0104] 350: When the number of feature points is 0, an updated threshold is obtained based on the second rule, the initial threshold and the threshold interval.

[0105] In this step, the updated threshold is less than the initial threshold. The meaning of the second rule, the threshold interval, and the process of obtaining the updated threshold can refer to the description in the above embodiment.

[0106] 360: Re-extract feature points from the grid based on the updated threshold to obtain g feature points.

[0107] 370: Determine the number of feature points K that need to be extracted from the grid based on the actual grid size, the size of the input image, and the expected number of feature points n. When g is greater than K, determine the K feature points with the highest response values ​​among the g feature points as the feature points of the grid. When g is less than or equal to K, determine g feature points as the feature points of the grid.

[0108] The process of determining the actual grid size and the number K of feature points to be extracted from each grid may refer to the description in the above embodiment.

[0109] 380: Determine the total number of feature points of n grids. When the total number is greater than the expected number of feature points n, determine n feature points with the highest response values ​​among the feature points of n grids as the final feature points. When the total number of feature points of n grids is less than or equal to the expected number of feature points n, directly use these feature points as the final feature points.

[0110] For the final feature points extracted, the feature descriptor corresponding to each feature point can be calculated and the result can be output.

[0111] The feature point extraction method provided in the embodiment of the present application is a high-quality grid feature point extraction method. For scenes with uneven texture distribution, the image is divided by using an adaptive grid size instead of a fixed grid size in the traditional method, so that the uniform distribution of feature points can be guaranteed as much as possible. In addition, by using an adaptive dynamic threshold with a constraint interval (threshold interval) to extract feature points for each grid, and screening feature points based on the response value, the K feature points with the largest response value are retained in each grid, so the quality of the feature points is guaranteed. That is, the feature point extraction method provided in the embodiment of the present application can obtain uniformly distributed high-quality feature points.

[0112] Figure 4 FIG. 1 is a flow chart of a feature matching method provided by an exemplary embodiment of the present application. Figure 4 As shown, the feature matching method includes the following contents.

[0113] 410: Extract feature points in the first image, and calculate first feature descriptors corresponding to the feature points in the first image.

[0114] Specifically, the feature point extraction method provided in the above embodiment may be used to extract the feature points in the first image.

[0115] 420: Extract feature points in the second image, and calculate second feature descriptors corresponding to the feature points in the second image.

[0116] Specifically, the feature point extraction method provided in the above embodiment may be used to extract the feature points in the second image.

[0117] 430: Perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result.

[0118] Specifically, the approximate nearest neighbor ratio method can be used to select the initial matching pairs, and the basic matrix F or the essential matrix E can be calculated, and RANSAC is performed to filter out the matching pairs that do not meet the geometric constraints to obtain the final matching results.

[0119] The embodiment of the present application provides a feature matching method, which can meet the needs of different application scenarios by using a self-adaptive grid size instead of a fixed grid size in a traditional method. In addition, the self-adaptive grid size can ensure that the number of grids is consistent with the number of expected feature points, and combined with the process of screening feature points based on the response value, it is convenient to extract K (for example, one or two) high-quality feature points for each grid, avoiding the deletion of too many high-quality points for the same area in the traditional method, thereby improving the quality of the feature points while ensuring that the feature points are evenly distributed, thereby improving the success rate of feature matching.

[0120] Figure 5 FIG. 1 is a flow chart of an image reconstruction method provided by an exemplary embodiment of the present application. Figure 5 As shown, the image reconstruction method includes the following contents.

[0121] 510: Extract feature points in the first image, and calculate first feature descriptors corresponding to the feature points in the first image.

[0122] Specifically, the feature point extraction method provided in the above embodiment may be used to extract the feature points in the first image.

[0123] 520: Extract feature points in the second image, and calculate second feature descriptors corresponding to the feature points in the second image.

[0124] Specifically, the feature point extraction method provided in the above embodiment may be used to extract the feature points in the second image.

[0125] 530: Perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result.

[0126] Specifically, the approximate nearest neighbor ratio method can be used to select the initial matching pairs, and the basic matrix F or the essential matrix E can be calculated, and RANSAC is performed to filter out the matching pairs that do not meet the geometric constraints to obtain the final matching results.

[0127] 540: Reconstruct the object to be reconstructed according to the matching result.

[0128] The objects to be reconstructed can be farmland, roads, buildings, etc.

[0129] The embodiment of the present application provides an image reconstruction method, which can meet the needs of different application scenarios by using an adaptive grid size instead of a fixed grid size in a traditional method. In addition, the adaptive grid size can ensure that the number of grids is consistent with the number of expected feature points, and combined with the process of selecting feature points based on the response value, it is convenient to extract K (for example, one or two) high-quality feature points for each grid, avoiding the deletion of too many high-quality points for the same area in the traditional method, thereby improving the quality of the feature points while ensuring that the feature points are evenly distributed, thereby improving the success rate of feature matching.

[0130] According to an embodiment of the present application, feature matching is performed on a first image and a second image based on a first feature descriptor and a second feature descriptor to obtain a matching result, including: matching the first feature descriptor and the second feature descriptor using an approximate nearest neighbor ratio method to obtain a plurality of initial matching pairs; calculating a motion model based on at least one matching pair among the plurality of initial matching pairs; and taking a matching pair among the plurality of initial matching pairs that satisfies the motion model as a matching result.

[0131] Specifically, the first image and the second image may be images of the same scene taken at different angles using a camera. Each feature point in the first image corresponds to a first feature descriptor, and each feature point in the second image corresponds to a second feature descriptor. Feature descriptors may be vectors that describe corresponding feature points. The similarity distances between multiple second feature descriptors and the first feature descriptor may be calculated respectively, and the second feature descriptor with the shortest similarity distance to the first feature descriptor may be selected from the multiple second feature descriptors to match with the first feature descriptor. For example, the matching of the first feature descriptor and the second feature descriptor may be implemented based on the approximate nearest neighbor ratio method, the Euclidean distance method, etc., to obtain multiple initial matching pairs.

[0132] At least one matching pair is selected from multiple initial matching pairs to calculate a motion model, and the motion model may be a basic matrix E or an essential matrix F. It is determined whether any matching pair among the multiple initial matching pairs satisfies the motion model. If so, it is retained, and if not, it is filtered out. In this way, the retained matching pairs can be used as matching results. For example, the basic matrix E is calculated based on the random sampling consensus (RANdomSAmpleConsensus, RANSAC) algorithm and the matching pairs that do not meet the geometric constraints are filtered out. That is, at least one matching pair can be randomly selected from multiple initial matching pairs, the basic matrix E is calculated based on the at least one matching pair, the distance between any matching pair among the multiple initial matching pairs and the basic matrix E is calculated, the matching pairs whose distance is less than or equal to a certain threshold are retained, and the matching pairs whose distance is greater than a certain threshold are filtered out. Since the basic matrix E is determined by randomly selecting a certain number of matching pairs, different matching pairs can be selected to calculate different basic matrices E. For different basic matrices E, matching pair filtering operations are performed respectively, so that the basic matrix E with the most matching pairs retained is used as the final basic matrix E, and the matching pairs corresponding to the final basic matrix E are used as matching results.

[0133] This embodiment can improve the similarity of matching pairs by using the approximate nearest neighbor ratio method, and can filter out matching pairs with large errors by building a motion model, thereby improving the accuracy of the final reconstructed image.

[0134] The feature point extraction method provided in the embodiment of the present application can be used not only in the field of three-dimensional reconstruction, but also in fields that require feature point extraction, such as moving target tracking, object recognition, image registration, and panoramic image stitching.

[0135] Figure 6a The figure shows a schematic diagram of an extraction result obtained by using a traditional feature point extraction method. Figure 6b Shown is a schematic diagram of an extraction result obtained by using a feature point extraction method provided by an embodiment of the present application. Figure 7a Shown is a schematic diagram of another extraction result obtained by using a traditional feature point extraction method. Figure 7b FIG. 2 is a schematic diagram showing another extraction result obtained by using the feature point extraction method provided by an embodiment of the present application. Figure 6a and Figure 6b ,as well as Figure 7a and Figure 7b It can be seen that the feature point extraction method provided in the embodiment of the present application can obtain uniformly distributed high-quality feature points. Compared with the traditional SIFT, the matching results obtained based on the feature points are more evenly distributed, and the number of correct matching pairs is close to 1.5 times that of the traditional SIFT.

[0136] Exemplary Devices

[0137] Figure 8 FIG. 8 is a schematic diagram of a structure of a feature point extraction device 800 provided by an exemplary embodiment of the present application. Figure 8 As shown, the extraction device 800 includes: a first determination module 810 , a division module 820 , an extraction module 830 and a second determination module 840 .

[0138] The first determination module 810 is used to determine the actual grid size based on the size of the input image and the expected number of feature points n. The division module 820 is used to divide the input image into grids based on the actual grid size to obtain n grids, where n is an integer greater than 1. The extraction module 830 is used to extract feature points from each of the n grids based on an initial threshold to obtain g feature points for the grid. The second determination module 840 is used to determine the K feature points with the highest response values ​​among the g feature points as the feature points of the grid when g is greater than K, where K is the number of feature points to be extracted for each grid.

[0139] The embodiment of the present application provides a feature point extraction device, which can meet the needs of different application scenarios by using an adaptive grid size instead of a fixed grid size in a traditional method. In addition, the adaptive grid size can ensure that the number of grids is consistent with the expected number of feature points, and combined with the process of selecting feature points based on the response value, it is convenient to extract K (for example, one or two) high-quality feature points for each grid, avoiding the deletion of too many high-quality points for the same area in the traditional method, thereby improving the quality of the feature points while ensuring that the feature points are evenly distributed.

[0140] According to one embodiment of the present application, the extraction module 830 is used to: extract feature points of the grid based on an initial threshold to obtain f feature points; when the number of f feature points is greater than the threshold value, obtain a first update threshold based on a first rule and the initial threshold, wherein the first update threshold is greater than the initial threshold; re-extract feature points of the grid based on the first update threshold to obtain g feature points.

[0141] According to one embodiment of the present application, the extraction module 830 is used to: when the number of f feature points is 0, obtain a second update threshold based on a second rule and an initial threshold, wherein the second update threshold is smaller than the initial threshold; and re-extract feature points from the grid based on the second update threshold to obtain g feature points.

[0142] According to an embodiment of the present application, the first update threshold and / or the second update threshold is within a preset threshold range.

[0143] According to one embodiment of the present application, the second determination module 840 is also used to: determine the number of feature points K that need to be extracted from the grid based on the actual grid size, the size of the input image and the expected number of feature points n; when g is less than or equal to K, determine g feature points as the feature points of the grid.

[0144] According to an embodiment of the present application, the second determination module 840 is also used to determine the total number of feature points of n grids. When the total number is greater than the expected number of feature points n, n feature points with higher response values ​​among the feature points of the n grids are determined as the final feature points.

[0145] It should be understood that the operations and functions of the first determination module 810, the division module 820, the extraction module 830, and the second determination module 840 in the above embodiment can refer to the above Figure 2 or Figure 3 The description of the feature point extraction method provided in the embodiment will not be repeated here to avoid repetition.

[0146] Fig. 9 FIG. 9 is a schematic diagram of the structure of a feature matching device 900 provided by an exemplary embodiment of the present application. Fig. 9 As shown, the feature matching device 900 includes: a first extraction module 910 , a second extraction module 920 and a matching module 930 .

[0147] The first extraction module 910 is used to extract feature points in the first image using the feature point extraction method provided in the above embodiment, and calculate the first feature descriptor corresponding to the feature points in the first image. The second extraction module 920 is used to extract feature points in the second image using the feature point extraction method provided in the above embodiment, and calculate the second feature descriptor corresponding to the feature points in the second image. The matching module 930 is used to perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result.

[0148] The embodiment of the present application provides a feature matching device, which can meet the needs of different application scenarios by using an adaptive grid size instead of a fixed grid size in a traditional method. In addition, the adaptive grid size can ensure that the number of grids is consistent with the number of expected feature points, and combined with the process of screening feature points based on the response value, it is convenient to extract K (for example, one or two) high-quality feature points for each grid, avoiding the deletion of too many high-quality points for the same area in the traditional method, thereby improving the quality of the feature points while ensuring that the feature points are evenly distributed, thereby improving the success rate of feature matching.

[0149] It should be understood that the operations and functions of the first extraction module 910, the second extraction module 920 and the matching module 930 in the above embodiment can refer to the above Figure 4The description of the feature matching method provided in the embodiment will not be repeated here to avoid repetition.

[0150] Fig.10 FIG. 1 is a schematic diagram of the structure of an image reconstruction device 1000 provided by an exemplary embodiment of the present application. Fig.10 As shown, the image reconstruction device 1000 includes: a first extraction module 1010 , a second extraction module 1020 , a matching module 1030 and a reconstruction module 1040 .

[0151] The first extraction module 1010 is used to extract feature points in the first image using the feature point extraction method provided in the above embodiment, and calculate the first feature descriptor corresponding to the feature points in the first image. The second extraction module 1020 is used to extract feature points in the second image using the feature point extraction method provided in the above embodiment, and calculate the second feature descriptor corresponding to the feature points in the second image. The matching module 1030 is used to perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result. The reconstruction module 1040 is used to reconstruct the object to be reconstructed according to the matching result.

[0152] The embodiment of the present application provides an image reconstruction device, which can meet the needs of different application scenarios by using an adaptive grid size instead of a fixed grid size in a traditional method. In addition, the adaptive grid size can ensure that the number of grids is consistent with the number of expected feature points, and combined with the process of selecting feature points based on the response value, it is convenient to extract K (for example, one or two) high-quality feature points for each grid, avoiding the deletion of too many high-quality points for the same area in the traditional method, thereby improving the quality of feature points while ensuring uniform distribution of feature points, thereby improving the success rate of feature matching.

[0153] According to one embodiment of the present application, the matching module 1030 is used to: match the first feature descriptor and the second feature descriptor using the approximate nearest neighbor ratio method to obtain multiple initial matching pairs; calculate the motion model based on at least one matching pair among the multiple initial matching pairs; and use the matching pairs among the multiple initial matching pairs that satisfy the motion model as the matching results.

[0154] It should be understood that the operations and functions of the first extraction module 1010, the second extraction module 1020, the matching module 1030, and the reconstruction module 1040 in the above embodiment can refer to the above Figure 5 In order to avoid repetition, the description of the image reconstruction method provided in the embodiment will not be repeated here.

[0155] Fig.11 FIG. 1 is a block diagram of an unmanned device 1100 for executing a feature point extraction method provided by an exemplary embodiment of the present application.

[0156] Reference Fig.11 , the unmanned device 1100 includes a processing component 1110, which further includes one or more processors, and a memory resource represented by a memory 1120 for storing instructions executable by the processing component 1110, such as an application. The application stored in the memory 1120 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1110 is configured to execute instructions to perform the above-mentioned feature point extraction method, feature matching method, or image reconstruction method. In one embodiment, the unmanned device may be a drone.

[0157] The unmanned device 1100 may also include a power supply component configured to perform power management of the unmanned device 1100, a wired or wireless network interface configured to connect the unmanned device 1100 to a network, and an input / output (I / O) interface. The unmanned device 1100 may be operated based on an operating system stored in the memory 1120, such as Windows Server 2008. TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0158] A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the unmanned device 1100, the unmanned device 1100 can perform a feature point extraction method, a feature matching method, or an image reconstruction method. The feature point extraction method includes: determining the actual grid size based on the size of the input image and the expected number of feature points n; gridding the input image based on the actual grid size to obtain n grids, where n is an integer greater than 1; extracting feature points from each of the n grids based on an initial threshold to obtain g feature points for the grid; when g is greater than K, determining the K feature points with the highest response values ​​among the g feature points as the feature points of the grid, where K is the number of feature points to be extracted for each grid. The feature matching method includes: extracting feature points in the first image using the feature point extraction method provided in the above embodiment, and calculating the first feature descriptor corresponding to the feature points in the first image; extracting feature points in the second image using the feature point extraction method provided in the above embodiment, and calculating the second feature descriptor corresponding to the feature points in the second image; performing feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result. The image reconstruction method includes: extracting feature points in a first image using the feature point extraction method provided in the above embodiment, and calculating a first feature descriptor corresponding to the feature points in the first image; extracting feature points in a second image using the feature point extraction method provided in the above embodiment, and calculating a second feature descriptor corresponding to the feature points in the second image; performing feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result; and reconstructing an object to be reconstructed according to the matching result.

[0159] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, and will not be described one by one here.

[0160] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0161] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0162] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0163] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0164] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0165] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program check codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0166] It should be noted that, in the description of this application, the terms "first", "second", "third", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.

[0167] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for extracting feature points, characterized in that: include: Determine the actual grid size based on the size of the input image and the expected number of feature points n; Gridding the input image based on the actual grid size to obtain n grids, where n is an integer greater than 1; Extracting feature points from each of the n grids based on an initial threshold value to obtain g feature points for the grid; When g is greater than K, K feature points with the highest response values ​​among the g feature points are determined as feature points of the grid, wherein K is the number of feature points to be extracted for each grid, and the value of K is determined by the actual grid size and the theoretical grid size, wherein the theoretical grid size is the quotient of the size of the input image and the expected number of feature points n. When the actual grid size is equal to the theoretical grid size, the value of K is 1, and when the actual grid size is smaller than the theoretical grid size, the value of K is 2.

2. The method for extracting feature points according to claim 1, characterized in that: The extracting feature points of each of the n grids based on the initial threshold to obtain g feature points for the grid includes: Extracting feature points from the grid based on the initial threshold to obtain f feature points; When the number of the f feature points is greater than a threshold value, obtaining a first update threshold based on a first rule and the initial threshold, wherein the first update threshold is greater than the initial threshold; Re-extracting feature points from the grid based on the first update threshold to obtain the g feature points.

3. The method for extracting feature points according to claim 2, characterized in that: The extracting feature points of each of the n grids based on the initial threshold to obtain g feature points for the grid also includes: When the number of the f feature points is 0, obtaining a second update threshold based on a second rule and the initial threshold, wherein the second update threshold is smaller than the initial threshold; Feature points of the grid are extracted again based on the second update threshold to obtain the g feature points.

4. The method for extracting feature points according to claim 3, characterized in that: The first update threshold and / or the second update threshold is within a preset threshold range.

5. The method for extracting feature points according to any one of claims 1 to 4, characterized in that: Also includes: Determine the number K of feature points that need to be extracted from the grid based on the actual grid size, the size of the input image, and the expected number n of feature points; When g is less than or equal to K, the g feature points are determined as feature points of the grid.

6. The method for extracting feature points according to any one of claims 1 to 4, characterized in that: Also includes: The total number of feature points of the n grids is determined, and when the total number is greater than the expected number of feature points n, n feature points with top response values ​​among the feature points of the n grids are determined as final feature points.

7. A method for reconstructing an image, characterized in that: include: Extracting feature points in the first image using the feature point extraction method described in any one of claims 1 to 6, and calculating first feature descriptors corresponding to the feature points in the first image; Extracting feature points in the second image using the feature point extraction method described in any one of claims 1 to 6, and calculating second feature descriptors corresponding to the feature points in the second image; Performing feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result; The object to be reconstructed is reconstructed according to the matching result.

8. The image reconstruction method according to claim 7, characterized in that: The performing feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result includes: Matching the first feature descriptor and the second feature descriptor using an approximate nearest neighbor ratio method to obtain a plurality of initial matching pairs; calculating a motion model based on at least one matching pair of the plurality of initial matching pairs; The matching pairs that satisfy the motion model among the multiple initial matching pairs are taken as the matching results.

9. A feature point extraction device, characterized in that: include: A first determination module is used to determine the actual grid size based on the size of the input image and the expected number of feature points n; A division module, configured to divide the input image into grids based on the actual grid size to obtain n grids, wherein n is an integer greater than 1; An extraction module, used for extracting feature points from each of the n grids based on an initial threshold value to obtain g feature points for the grid; The second determination module is used to determine, when g is greater than K, K feature points with top response values ​​among the g feature points as feature points of the grid, wherein K is the number of feature points to be extracted for each grid, and the value of K is determined by the actual grid size and the theoretical grid size, and the theoretical grid size is the quotient of the size of the input image and the expected number of feature points n. When the actual grid size is equal to the theoretical grid size, the value of K is 1, and when the actual grid size is smaller than the theoretical grid size, the value of K is 2.

10. An image reconstruction device, characterized in that: include: A first extraction module, configured to extract feature points in a first image by using the feature point extraction method according to any one of claims 1 to 6, and calculate a first feature descriptor corresponding to the feature points in the first image; A second extraction module, configured to extract feature points in a second image by using the feature point extraction method according to any one of claims 1 to 6, and calculate a second feature descriptor corresponding to the feature points in the second image; a matching module, configured to perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result; A reconstruction module is used to reconstruct the object to be reconstructed according to the matching result.

11. An unmanned device, characterized in that: include: processor; a memory for storing instructions executable by the processor, The processor is used to execute the feature point extraction method described in any one of claims 1 to 6, or the image reconstruction method described in claim 7 or 8.

12. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the feature point extraction method described in any one of claims 1 to 6, or the image reconstruction method described in claim 7 or 8.

Images