Feature point extraction method, image reconstruction method and device

Abstract

The present application provides a method for extracting feature points, a method for reconstructing an image, and a device. The method for extracting feature points includes: determining the number of multiple feature points initially extracted from an input image; when the number of multiple feature points is greater than the expected number of feature points, performing a first grid division on the input image based on a space division method to obtain K first grids, wherein K is greater than or equal to the expected number of feature points; determining the number of feature points of each first grid in the K first grids, and when the number of feature points of the first grid is greater than 1, determining the statistical confidence score of each feature point based on the distance of each feature point in the first grid from the center of the first grid; screening the feature points of the first grid based on the statistical confidence scores of each feature point to obtain the feature point with the highest statistical confidence score in the first grid. The technical solution of the present application can improve the uniformity of the distribution of feature points.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for extracting feature points, and a method and device for reconstructing an image. Background Art

[0002] In computer vision, the image processing process is an extremely important link, and in the image processing process, the extraction of feature points is a crucial step. Since feature points can represent the content of the image, feature point extraction is widely used in the fields of moving target tracking, object recognition, image registration, panoramic image stitching, 3D reconstruction, etc. The commonly used feature point extraction methods are prone to the problem of feature point clustering and uneven distribution, especially for scenes with repeated textures. The uneven distribution of feature points will directly affect the accuracy of subsequent image processing results. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a method for extracting feature points, a method for reconstructing an image, and a device, which can improve the distribution uniformity of feature points.

[0004] In a first aspect, an embodiment of the present application provides a method for extracting feature points, comprising: determining the number of multiple feature points preliminarily extracted from an input image; when the number of multiple feature points is greater than the expected number of feature points, performing a first grid division on the input image based on a spatial division method to obtain K first grids, wherein K is greater than or equal to the expected number of feature points; determining the number of feature points of each first grid in the K first grids, and when the number of feature points in the first grid is greater than 1, determining the statistical confidence score of each feature point in the first grid based on the distance of each feature point from the center of the first grid; screening the feature points of the first grid based on the statistical confidence scores of each feature point to obtain the feature point with the highest statistical confidence score in the first grid.

[0005] In certain embodiments of the present application, the statistical confidence score of each feature point in the first grid is determined based on the distance of each feature point from the center of the first grid, including: determining the statistical confidence score of each feature point based on the distance of each feature point in the first grid from the center of the first grid and the response value of each feature point.

[0006] In certain embodiments of the present application, the statistical confidence score of each feature point is determined based on the distance of each feature point in the first grid from the center of the first grid and the response value of each feature point, including: determining the statistical confidence score of each feature point by weighted summing the inverse of the distance of each feature point in the first grid from the center of the first grid and the response value of each feature point.

[0007] In some embodiments of the present application, the first grid division is performed based on a binary tree, quadtree or octree partitioning method.

[0008] In certain embodiments of the present application, the feature point extraction method of the first aspect also includes: dividing the input image into a second grid based on a preset grid size to obtain T second grids, where T is an integer greater than 1; performing preliminary extraction of feature points for each of the T second grids to obtain multiple feature points.

[0009] In certain embodiments of the present application, the feature point extraction method of the first aspect also includes: constructing an image pyramid for the input image, wherein preliminary extraction of feature points is performed for each of the T second grids to obtain multiple feature points, including: extracting extreme points of the second grid based on the image pyramid and the initial threshold; when the number of extreme points is 0, obtaining a first update threshold based on the first rule and the initial threshold, wherein the first update threshold is less than the initial threshold; re-extracting extreme points of the second grid based on the image pyramid and the first update threshold, wherein the extreme points extracted from the second grid are used as feature points.

[0010] In certain embodiments of the present application, preliminary extraction of feature points is performed on each of the T second grids to obtain multiple feature points, and the method further includes: when the number of extreme points is greater than a threshold value, obtaining a second update threshold based on a second rule and an initial threshold, wherein the second update threshold is greater than the initial threshold; and re-extracting extreme points of the second grid based on the image pyramid and the second update threshold.

[0011] In certain embodiments of the present application, the feature point extraction method of the first aspect also includes: performing a scale pyramid pooling operation on the input image to obtain multiple images in different scale spaces; and calculating the feature descriptor corresponding to the feature point with the highest statistical confidence score in the first grid in the multiple images in different scale spaces.

[0012] In a second aspect, an embodiment of the present application provides an image reconstruction method, comprising: extracting feature points in a first image using the feature point extraction method described in the first aspect, and calculating a first feature descriptor corresponding to the feature points in the first image; extracting feature points in a second image using the feature point extraction method described in the first aspect, and calculating a second feature descriptor corresponding to the feature points in the second image; performing feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result; and reconstructing the object to be reconstructed based on the matching result.

[0013] In a third aspect, an embodiment of the present application provides a feature point extraction device, comprising: a first determination module, used to determine the number of multiple feature points preliminarily extracted from an input image; a first division module, used to perform a first grid division on the multiple feature points based on a spatial division method to obtain K first grids when the number of multiple feature points is greater than the expected number of feature points, wherein K is greater than or equal to the expected number of feature points; a second determination module, used to determine the number of feature points of each first grid in the K first grids, and when the number of feature points in the first grid is greater than 1, determine the statistical confidence score of each feature point in the first grid based on the distance of each feature point from the center of the first grid; a screening module, used to screen the feature points of the first grid based on the statistical confidence scores of each feature point to obtain the feature point with the highest statistical confidence score in the first grid.

[0014] In a fourth aspect, an embodiment of the present application provides an image reconstruction device, comprising: a first extraction module, used to extract feature points in a first image using the feature point extraction method described in the first aspect, and calculate a first feature descriptor corresponding to the feature points in the first image; a second extraction module, used to extract feature points in a second image using the feature point extraction method described in the first aspect, and calculate a second feature descriptor corresponding to the feature points in the second image; a matching module, used to perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result; and a reconstruction module, used to reconstruct the object to be reconstructed according to the matching result.

[0015] In a fifth aspect, an embodiment of the present application provides an unmanned device, comprising: a processor; a memory for storing processor executable instructions, wherein the processor is used to execute the feature point extraction method described in the first aspect above, or to execute the image reconstruction method described in the second aspect.

[0016] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the feature point extraction method described in the first aspect, or to execute the image reconstruction method described in the second aspect.

[0017] The embodiment of the present application provides a method for extracting feature points, a method for reconstructing an image, and a device, by performing a first grid division on an input image based on a spatial division method to obtain a plurality of first grids, and screening the feature points in each first grid according to the statistical confidence score to retain the feature points with the highest statistical confidence score in each first grid. Since the feature points with higher statistical confidence scores are closer to the center of the first grid, the method for extracting feature points provided by the embodiment of the present application can improve the uniformity of the distribution of all feature points finally obtained, avoid the situation where the feature points cannot be extracted from the repeated texture area in the input image, and effectively prevent the feature points from clumping. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Shown is a schematic diagram of the system architecture of a feature point extraction system provided by an exemplary embodiment of the present application.

[0019] Figure 2 Shown is a flow chart of a method for extracting feature points provided by an exemplary embodiment of the present application.

[0020] Figure 3 Shown is a flow chart of a feature point extraction method provided by another exemplary embodiment of the present application.

[0021] Figure 4 Shown is a flow chart of a feature matching method provided by an exemplary embodiment of the present application.

[0022] Figure 5 Shown is a flowchart of an image reconstruction method provided by an exemplary embodiment of the present application.

[0023] Figure 6a The figure shows a schematic diagram of an extraction result obtained by using a traditional feature point extraction method.

[0024] Figure 6b Shown is a schematic diagram of an extraction result obtained by using a feature point extraction method provided by an embodiment of the present application.

[0025] Figure 7a Shown is a schematic diagram of another extraction result obtained by using a traditional feature point extraction method.

[0026] Figure 7b Shown is a schematic diagram of another extraction result obtained by using the feature point extraction method provided by an embodiment of the present application.

[0027] Figure 8 Shown is a schematic structural diagram of a feature point extraction device provided by an exemplary embodiment of the present application.

[0028] Fig. 9Shown is a schematic structural diagram of a feature matching device provided by an exemplary embodiment of the present application.

[0029] Fig.10 Shown is a schematic structural diagram of an image reconstruction device provided by an exemplary embodiment of the present application.

[0030] Fig.11 Shown is a block diagram of an unmanned device for executing a feature point extraction method provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] Application Overview

[0033] Farmland mapping and reconstruction generally involves extracting feature points from input images, generating data associations between multiple frames based on the feature points, estimating relative motion between frames (images), and finally using Bundle Adjustment to optimize camera posture and three-dimensional scene structure.

[0034] Under the condition that the expected number of feature points is M, the commonly used feature point extraction method is to use a unified threshold to extract feature points from the entire image, then sort all the extracted feature points from large to small according to the response value, and finally extract the first M feature points from the sorting results, and calculate the feature descriptors of the first M feature points as the final feature output.

[0035] For scenes with repeated textures, such as banana groves, wheat fields, and corn fields in farmland scenes, it is usually difficult to extract feature points using a unified threshold. This is because the neighborhood information in repeated texture scenes is very close and the distinction is relatively low. When the threshold condition is more stringent, feature points cannot be extracted; when the threshold condition is more relaxed, the quality of the extracted feature points is low.

[0036] In addition, for scenes composed of repeated textures and roads, most of the extracted feature points are concentrated on both sides of the road, while only very few feature points can be extracted from farmland with repeated textures (see Figure 7a ), that is, the feature points are clustered.

[0037] In summary, when using traditional feature point extraction methods to extract feature points from scenes containing repeated textures, problems such as feature point clustering and uneven distribution are likely to occur.

[0038] Exemplary Systems

[0039] Figure 1 FIG. 1 is a schematic diagram of a system architecture of a feature point extraction system 100 provided by an exemplary embodiment of the present application, which illustrates an application scenario of extracting feature points from an image. The feature point extraction system 100 includes a computer device 110 and an image acquisition device 120 .

[0040] The image acquisition device 120 can capture a specific scene to obtain one or more images. Here, the specific scene can be a farmland, a road, a building, etc.

[0041] The computer device 110 may acquire one or more images from the image acquisition device 120 and extract feature points from the acquired images. The specific extraction method may be found in the description below.

[0042] The computer device 110 may be a general-purpose computer or a computer device composed of a dedicated integrated circuit, etc., which is not limited in the embodiments of the present application. For example, the computer device 110 may be a mobile terminal device such as a tablet computer, or may also be a personal computer (PC), such as a laptop computer and a desktop computer, etc. Those skilled in the art will appreciate that the number of the above-mentioned computer devices 110 may be one or more, and their types may be the same or different. The embodiments of the present application do not limit the number and device type of the computer devices 110.

[0043] The image acquisition device 120 may be a camera, for example, a monocular camera, a binocular camera, or a trinocular camera, etc. In one embodiment, the image acquisition device 120 may be a camera installed on a drone.

[0044] In some embodiments, the computer device 110 and the image acquisition device 120 may be two independent devices, and the computer device 110 may be communicatively connected with one or more image acquisition devices 120. In some other embodiments, the image acquisition device 120 may be integrated into the computer device 110, and the image acquisition device 120 integrated into the computer device 110 may be one or more.

[0045] It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the embodiments of the present application are not limited thereto. On the contrary, the embodiments of the present application can be applied to any scenario that may be applicable.

[0046] Exemplary Methods

[0047] Figure 2 Shown is a flow chart of a method for extracting feature points provided by an exemplary embodiment of the present application. Figure 2 The method can be performed by a computer device. Figure 2 As shown, the feature point extraction method includes the following contents.

[0048] 210: Determine the number of the plurality of feature points initially extracted from the input image.

[0049] Specifically, feature points may be extracted from the entire image to obtain multiple feature points. Alternatively, the entire image may be divided into multiple regions, feature points may be extracted from each region, and multiple feature points corresponding to the entire image may be obtained.

[0050] In one embodiment, a reasonable threshold may be set to obtain a plurality of feature points distributed in different areas of the input image.

[0051] The feature points in the embodiments of the present application may be scale invariant feature transform (SIFT) feature points, speeded up robust feature (SURF) feature points, ORB (Oriented FAST and Rotated BRIEF) feature points, or other types of feature points.

[0052] 220: When the number of the plurality of feature points is greater than the expected number of feature points, perform first grid division on the input image based on a space division method to obtain K first grids, wherein K is greater than or equal to the expected number of feature points.

[0053] The expected number of feature points is M. When the number of the extracted feature points is greater than the expected number of feature points M, M feature points need to be screened out from the multiple feature points. In order to ensure that the screened M feature points are evenly distributed on the input image, the input image can be divided into a first grid, and the extracted multiple feature points are divided into multiple first grids. For example, the feature points can be divided into corresponding first grids according to their coordinates. Each first grid can have 0, 1 or more feature points.

[0054] Specifically, the shape of the grid can be a square, and the size of the grid can be represented by the side length of the square. Square grids with equal side lengths can achieve uniform division of multiple feature points and improve the uniformity of distribution of feature points obtained subsequently. In some embodiments, the shape of the grid can be a rectangle, a rhombus, a triangle or other polygon, or other regular or irregular shapes. The specific grid shape can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0055] In one embodiment, as many feature points as possible can be extracted so that the number of extracted feature points is greater than the expected number of feature points M. This ensures that feature points are extracted in each area of ​​the input image, especially for repeated texture areas, thereby facilitating subsequent feature point matching and three-dimensional reconstruction processes.

[0056] 230: Determine the number of feature points of each of the K first grids, and when the number of feature points of the first grid is greater than 1, determine a statistical confidence score of each feature point based on a distance of each feature point in the first grid from a center of the first grid.

[0057] Specifically, the number of feature points in each first grid may be counted, and when the number of feature points in the first grid is greater than 1, a statistical confidence score S of each feature point in the first grid is determined. The statistical confidence score S may represent a distance d between the feature point and the center of the first grid. For example, a higher statistical confidence score S indicates that the feature point is closer to the center of the first grid.

[0058] 240: Screening the feature points of the first grid based on the statistical confidence scores of the respective feature points to obtain the feature points with the highest statistical confidence scores in the first grid.

[0059] The feature points in the first grid are sorted according to the statistical confidence score S of each feature point in the first grid, the feature point with the largest statistical confidence score S is retained, and the other feature points are deleted.

[0060] The feature points of the K first grids may be screened respectively according to the statistical confidence score S, so as to select the feature points closer to the center of the first grid from each first grid, thereby ensuring that all the feature points finally obtained are evenly distributed in the input image.

[0061] The embodiment of the present application provides a method for extracting feature points, by performing a first grid division on the input image based on a space division method to obtain multiple first grids, and screening the feature points in each first grid according to the statistical confidence score to retain the feature points with the highest statistical confidence score in each first grid. Since the feature points with higher statistical confidence scores are closer to the center of the first grid, the method for extracting feature points provided by the embodiment of the present application can improve the uniformity of the distribution of all feature points finally obtained, avoid the situation where the feature points cannot be extracted from the repeated texture area in the input image, and effectively prevent the feature points from clumping.

[0062] According to an embodiment of the present application, the statistical confidence score of each feature point is determined based on the distance of each feature point in the first grid from the center of the first grid, including: determining the statistical confidence score of each feature point based on the distance of each feature point in the first grid from the center of the first grid and the response value of each feature point.

[0063] Specifically, the larger the response value, the higher the quality of the corresponding feature point. The response value of the feature point can be the response value of the corner point in the feature extraction and detection technology, for example, Harris response value, Shi-Tomasi response value or other response value.

[0064] When screening multiple feature points in the first grid, the distances of the multiple feature points from the center of the first grid may be different, and the response values ​​of the multiple feature points may also be different. In addition, there is no relationship between the distance of the feature point from the center of the first grid and the response value of the feature point, wherein the distance of the feature point from the center of the first grid will affect the distribution uniformity of the feature points obtained in the end, and the response value of the feature point will affect the overall quality of the feature points obtained in the end.

[0065] Therefore, a trade-off needs to be made between the distance of the feature point from the center of the first grid and the response value of the feature point, so as to improve the overall quality of the feature point as much as possible while improving the distribution uniformity of the feature points.

[0066] This embodiment determines the statistical confidence score of the feature point based on the distance of the feature point from the center of the first grid and the response value of the feature point. It can comprehensively consider the factors affecting the distribution uniformity of the feature points and the factors affecting the overall quality of the feature points, and then screen out feature points that are relatively close to the center of the first grid and have higher quality from the first grid, so that more robust feature points can be obtained, and the effect of improving the distribution uniformity and overall quality of the final feature points can be achieved.

[0067] According to an embodiment of the present application, based on the distance of each feature point in the first grid from the center of the first grid and the response value of each feature point, the statistical confidence score of each feature point is determined, including: determining the statistical confidence score of each feature point by taking a weighted sum of the inverse of the distance of each feature point in the first grid from the center of the first grid and the response value of each feature point.

[0068] Specifically, the calculation formula of the statistical confidence score S can be: S = w1*1 / d+w2*f, where d is the distance between the feature point and the center of the first grid, f is the response value of the feature point, w1 is the first weight value, and w2 is the second weight value. w1 and w2 can be set according to actual needs.

[0069] In other embodiments, the statistical confidence score S may be obtained according to other calculation formulas, as long as it can comprehensively consider the distance of the feature point from the center of the first grid and the response value of the feature point.

[0070] According to an embodiment of the present application, the first grid division is performed based on a binary tree, quadtree or octree partitioning method.

[0071] Specifically, the binary tree partitioning method can obtain 2 t nodes, the quadtree partitioning method can get 4 nodes each time. t Nodes, the octree partitioning method can get 8 nodes each time. t nodes, where t is the number of segmentation times. Regardless of the binary tree, quadtree or octree segmentation method, there is a node before segmentation, and all of the multiple feature points (mentioned in step 210) initially extracted are located in the node, and this node corresponds to the entire input image. As the number of segmentation times increases, multiple nodes can be obtained, and the multiple nodes can correspond one-to-one to the multiple first grids.

[0072] Based on the quadtree segmentation method, a suitable number of segmentation results (K first grids) can be obtained at a reasonable segmentation speed, so the quadtree segmentation method can be used in some embodiments.

[0073] For example, the quadtree partitioning method is used to perform the first grid partitioning. When the number of nodes obtained by the partitioning is 4 t When the number of feature points is greater than or equal to the expected number M, the division is stopped to obtain K nodes (corresponding to K first grids). If the number of feature points in the first grid is greater than 1, the feature points of the first grid are screened based on the statistical confidence score.

[0074] According to one embodiment of the present application, the feature point extraction method also includes: dividing the input image into a second grid based on a preset grid size to obtain T second grids, where T is an integer greater than 1; performing preliminary extraction of feature points for each of the T second grids to obtain multiple feature points.

[0075] Specifically, the entire input image can be divided into T second grids through second grid division, feature points can be extracted from each second grid to obtain multiple feature points, and then the entire input image can be divided into K first grids through first grid division, and the feature points of each first grid can be screened based on the statistical confidence score to obtain the feature points with the highest statistical confidence score in the first grid.

[0076] The shape of the second grid is similar to that of the first grid, and will not be described in detail here. The grid size can be the side length or area of ​​the grid.

[0077] In one embodiment, the shape of the second grid is a square, and the preset grid size may refer to the side length W of the square.

[0078] For example, the preset grid size is W, the number of rows and columns of the input image is r, then the number of grids in each row rGrid is r / W, the number of grids in each column cGrid is c / W, the total number of grids nGrid is rGrid×cGrid, and the size of each grid is W×W.

[0079] W can be specifically determined based on the amount of calculation and the need for homogenization, that is, it can be pre-set according to the needs. For example, the larger W is, the larger the size of the second grid is, and the fewer the number of second grids is. In this way, when extracting feature points for a second grid, it is necessary to extract more feature points from the second grid based on a certain threshold for subsequent screening. In this process, since the number of second grids is small, the amount of calculation can be reduced; but because the size of the second grid is large, the feature points in the second grid may be concentrated in the local area of ​​the second grid, while other areas of the second grid do not extract feature points, which will cause the final extracted feature points to be unevenly distributed. Therefore, when setting W, it is necessary to balance the amount of calculation and the need for homogenization in order to obtain enough and relatively evenly distributed feature points so that the final feature points can be screened out from these feature points based on the above-mentioned spatial division method.

[0080] In one embodiment, the number T of second grids may be greater than the expected number M of feature points, so that at least one feature point can be extracted from each second grid, which can make the number of initially extracted feature points sufficient to facilitate subsequent screening of feature points with higher distribution uniformity and higher quality from a large number of feature points.

[0081] This embodiment first divides the input image into a second grid and extracts feature points from each second grid to obtain multiple feature points, which can ensure that feature points can be extracted from each second grid as much as possible, preparing for finally obtaining evenly distributed feature points. In addition, this embodiment divides the input image into a first grid and screens the feature points of each first grid based on the statistical confidence score, which can further improve the uniformity of the distribution of the feature points finally obtained, and at the same time can improve the overall quality of the feature points finally obtained as much as possible.

[0082] According to an embodiment of the present application, the feature point extraction method also includes: constructing an image pyramid for the input image, wherein preliminary extraction of feature points is performed on each of the T second grids to obtain multiple feature points, including: extracting extreme points of the second grid based on the image pyramid and the initial threshold; when the number of extreme points is 0, obtaining a first update threshold based on the first rule and the initial threshold, wherein the first update threshold is less than the initial threshold; re-extracting extreme points of the second grid based on the image pyramid and the first update threshold, wherein the extreme points extracted from the second grid are used as feature points.

[0083] Specifically, the image pyramid can be a differential Gaussian pyramid. The initial threshold (which can be expressed as T0) is pre-set. The extreme points of each grid are extracted according to the initial threshold. The initial threshold can be used to measure the difference between a certain pixel point on the image and the surrounding pixels, such as the pixel value difference (grayscale or brightness difference). Specifically, if the difference between the pixel value of a certain pixel point and the surrounding pixels is greater than the initial threshold, the pixel point can be extracted as an extreme point. Of course, in other embodiments, the specific meaning of the initial threshold and the specific method of extracting the extreme points can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0084] For any pixel point in the second grid, in the current differential Gaussian pyramid image I i And the previous layer of difference Gaussian pyramid image I i-1 And the next layer of differential Gaussian pyramid image I i+1 The intensity values ​​(e.g., brightness values, or pixel values) are counted in the 26 neighborhoods in the image. If the intensity value of the current pixel point in the three image layers is the largest or the smallest, the pixel point is taken as a candidate extreme point.

[0085] Further, by performing sub-pixel calculation on the pixel point, for example, interpolating the neighboring pixels of the candidate extreme point, the sub-pixel point is found. It is determined whether the difference in intensity between the sub-pixel point and the surrounding pixels is greater than or equal to the initial threshold. If so, the candidate extreme point corresponding to the sub-pixel point is converted into a formal extreme point.

[0086] If the number of formal extreme points extracted from the second grid is 0, it indicates that the current second grid is in a scene with weak texture or severe repeated texture, such as wheat fields, corn fields and other farmland scenes. At this time, it is necessary to lower the threshold and relax the extraction conditions to extract extreme points (formal extreme points) in the current second grid to ensure the uniformity of the distribution of the feature points finally extracted. That is, the first update threshold T1 is obtained based on the first rule and the initial threshold T0. Specifically, the first rule can be to take half of the existing threshold as the new threshold, that is, T1=T0 / 2.

[0087] Based on the image pyramid and threshold T1, the extreme points of the second grid are extracted again. If the number of the extracted extreme points is still 0, a new threshold T2 is obtained based on the first rule and threshold T1. This is repeated until the number of the extreme points extracted for the current second grid is greater than 0. The extreme points extracted from the second grid can be used as feature points.

[0088] In this embodiment, the first rule may be other content as long as it can ensure that the first update threshold is less than the initial threshold.

[0089] By setting the first rule, this embodiment can appropriately lower the threshold for scenes with weak textures or severe repeated textures to extract feature points in the current grid, thereby ensuring uniform distribution of the feature points finally extracted. In addition, by extracting extreme points based on the image pyramid, the invariance of the scale space can be ensured, thereby obtaining feature points with more robust scale factors.

[0090] In one embodiment, when performing sub-pixel calculation on candidate extreme points based on the image pyramid, main directions may be further counted to facilitate subsequent calculation of feature descriptors.

[0091] According to an embodiment of the present application, preliminary extraction of feature points is performed on each of the T second grids to obtain multiple feature points, and the method further includes: when the number of extreme points is greater than a threshold value, obtaining a second update threshold based on a second rule and an initial threshold, wherein the second update threshold is greater than the initial threshold; and re-extracting extreme points of the second grid based on the image pyramid and the second update threshold.

[0092] In this embodiment, the threshold value may be preset according to actual needs, for example, it may be a value such as 5 or 10.

[0093] If the number of formal extreme points extracted from the second grid is greater than the threshold value, it indicates that the current second grid is in a scene with richer textures, such as a scene with more types of urban or farmland crops. At this time, the threshold needs to be increased to prevent the subsequent feature extraction from taking too much time. That is, the second updated threshold T1' is obtained based on the second rule and the initial threshold T0. Specifically, the second rule can be to take twice the existing threshold as the new threshold, that is, T1'=2×T0.

[0094] Based on the image pyramid and the threshold T1', the extreme points of the second grid are extracted again. If the number of the extracted extreme points is still greater than the threshold, a new threshold T2' is obtained based on the second rule and the threshold T1'. This process is repeated until the number of the extreme points extracted for the current second grid is less than or equal to the threshold. The extreme points extracted from the second grid can be used as feature points.

[0095] In this embodiment, the second rule may be other content as long as it can ensure that the second update threshold is greater than the initial threshold.

[0096] By setting the second rule and the threshold value, this embodiment can appropriately increase the threshold value for scenes with richer textures to avoid extracting a large number of feature points, thereby saving computing time and improving computing efficiency.

[0097] For the feature point extraction of repeated texture scenes, the embodiment of the present application adopts a gridding strategy to evenly extract feature points of the scene, which can prevent the feature points from clumping, and adopts an adaptive dynamic threshold adjustment for each grid to ensure that feature points can be extracted from each area with the highest probability, providing sufficient preparation for the final acquisition of evenly distributed feature points.

[0098] According to one embodiment of the present application, the feature point extraction method also includes: performing a scale pyramid pooling operation on the input image to obtain multiple images in different scale spaces; calculating the feature descriptor corresponding to the feature point with the highest statistical confidence score in the first grid in the multiple images in different scale spaces.

[0099] Specifically, in agricultural scenarios, drones are not always on the same horizontal line during flight, especially in mountainous areas, fruit forests and other scenes, where the height changes are very obvious. Relying only on the scale invariance of the extreme point (or feature point) detection stage is not robust enough for 3D reconstruction. Therefore, when calculating the feature descriptors corresponding to the feature points, multi-scale factors can be introduced. In order to consider multi-scale constraints as much as possible, a scale pyramid pooling operation can be performed on the input image.

[0100] For example, if the number of pooling layers is L, the convolution kernel sizes are 3×3, 5×5, …, (1+2L)×(1+2L). The input image is convolved L times according to the above convolution kernel to obtain images of different scale spaces. For any feature point, the feature descriptor corresponding to the feature point can be counted in L+1 images (including the original input image), so that a feature point with a more robust scale factor can be obtained.

[0101] In one embodiment, the size of the image (a plurality of images in different scale spaces) obtained by the scale pyramid pooling operation is consistent with that of the input image, but the degree of blur is different. That is, the size of the L+1 images is consistent, but the degree of blur is different. Finding different gradient angles from the L+1 images, and then counting the number of different gradient angles can obtain a vector (such as a 128-dimensional vector) that represents the feature descriptor. Compared to only counting different gradient angles in the input image, the embodiment of the present application can obtain feature points with more robust scale factors by counting different gradient angles in L+1 images and then calculating the feature descriptors corresponding to the feature points.

[0102] In this embodiment, different gradient angles may include 30 degrees, 60 degrees, 90 degrees, etc., and the specific angle value may be set according to actual needs. In one embodiment, statistics of the number of different gradient angles may be performed in the process of extracting extreme points, wherein the main direction may be the gradient angle with the largest value among the different gradient angles.

[0103] Figure 3 Shown is a flow chart of a feature point extraction method provided by another exemplary embodiment of the present application. Figure 3 The embodiment is Figure 2 In order to avoid repetition, the same parts of the examples in the embodiments will not be described again. Figure 3 As shown, the feature point extraction method includes the following contents.

[0104] 310: Divide the input image into second grids based on a preset grid size to obtain T second grids.

[0105] T is an integer greater than 1.

[0106] 320: Construct an image pyramid for the input image, and extract extreme points of the second grid based on the image pyramid and the initial threshold.

[0107] 330: When the number of extreme value points is 0, obtain an updated threshold based on the first rule and the initial threshold.

[0108] In this step, the updated threshold is smaller than the initial threshold. The acquisition process of the first rule and the updated threshold can refer to the description in the above embodiment.

[0109] 340: When the number of extreme value points is greater than the threshold value, an updated threshold value is obtained based on the second rule and the initial threshold value.

[0110] In this step, the updated threshold is greater than the initial threshold. The second rule and the acquisition process of the updated threshold can refer to the description in the above embodiment.

[0111] 350: Re-extract extreme points from the second grid based on the image pyramid and the updated threshold.

[0112] 360: taking the extreme points extracted from each second grid as feature points, and determining the number of feature points.

[0113] The specific process of obtaining the extreme value points can refer to the description in the above embodiment, and will not be repeated here to avoid repetition.

[0114] In this step, if the total number of feature points obtained from the second grid is less than or equal to the expected number of feature points, these feature points can be directly used as the final feature points.

[0115] 370: When the number of the plurality of feature points is greater than the expected number of feature points, perform first grid division on the input image based on a space division method to obtain K first grids.

[0116] K is greater than or equal to the expected number of feature points. The specific content of the space partitioning method (such as the quadtree partitioning method) and the first grid partitioning can be found in the description of the above embodiment, and will not be described here to avoid repetition.

[0117] 380: When the number of feature points in the first grid is greater than 1, determine a statistical confidence score of each feature point in the first grid, and screen the feature points of the first grid based on the statistical confidence scores of each feature point to obtain a feature point with a highest statistical confidence score in the first grid.

[0118] Specifically, the statistical confidence score of each feature point may be determined by performing weighted summation on the inverse of the distance of each feature point in the first grid from the center of the first grid and the response value of each feature point.

[0119] 390: Summarize the feature points with the highest statistical confidence scores in each first grid as the final feature points.

[0120] The number of feature points finally obtained may be M or slightly larger than M, which is mainly related to the number of nodes divided by the quadtree. Optionally, when the number of feature points obtained by summarization is larger than M, all feature points may be sorted based on the response value to extract the top M feature points as the final feature points.

[0121] For the final feature points extracted, the feature descriptor corresponding to each feature point can be calculated and the result can be output.

[0122] The feature point extraction method provided in the embodiment of the present application is a grid-based and adaptive dynamic threshold feature point extraction method for farmland repetitive texture scenes. In addition, in the feature point extraction process, the feature points closer to the center of the grid (first grid) are more reliable, and the feature points closer to the edge of the grid have lower reliability. By screening the feature points based on the statistical confidence score, the embodiment of the present application can comprehensively consider the distribution position and response value of the feature points in the grid, so as to obtain more robust feature points.

[0123] Figure 4 FIG. 1 is a flow chart of a feature matching method provided by an exemplary embodiment of the present application. Figure 4 As shown, the feature matching method includes the following contents.

[0124] 410: Extract feature points in the first image, and calculate first feature descriptors corresponding to the feature points in the first image.

[0125] Specifically, the feature point extraction method provided in the above embodiment may be used to extract the feature points in the first image.

[0126] 420: Extract feature points in the second image, and calculate second feature descriptors corresponding to the feature points in the second image.

[0127] Specifically, the feature point extraction method provided in the above embodiment may be used to extract the feature points in the second image.

[0128] 430: Perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result.

[0129] Specifically, the approximate nearest neighbor ratio method can be used to select the initial matching pairs, and the essential matrix F is calculated and RANSAC is performed to filter out the matching pairs that do not meet the geometric constraints to obtain the final matching results.

[0130] The embodiment of the present application provides a feature matching method, which divides the input image into a first grid based on a space division method to obtain multiple first grids, and screens the feature points in each first grid according to the statistical confidence score to retain the feature points with the highest statistical confidence score in each first grid. Since the feature points with higher statistical confidence scores are closer to the center of the first grid, the feature point extraction method provided by the embodiment of the present application can improve the uniformity of the distribution of all feature points finally obtained, avoid the situation where the feature points cannot be extracted from the repeated texture area in the input image, and effectively prevent the feature points from clumping.

[0131] Figure 5 FIG. 2 is a flow chart of an image reconstruction method provided by an exemplary embodiment of the present application. Figure 5 As shown, the image reconstruction method includes the following contents.

[0132] 510: Extract feature points in the first image, and calculate first feature descriptors corresponding to the feature points in the first image.

[0133] Specifically, the feature point extraction method provided in the above embodiment may be used to extract the feature points in the first image.

[0134] 520: Extract feature points in the second image, and calculate second feature descriptors corresponding to the feature points in the second image.

[0135] Specifically, the feature point extraction method provided in the above embodiment may be used to extract the feature points in the second image.

[0136] 530: Perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result.

[0137] Specifically, the approximate nearest neighbor ratio method can be used to select the initial matching pairs, and the essential matrix F is calculated and RANSAC is performed to filter out the matching pairs that do not meet the geometric constraints to obtain the final matching results.

[0138] 540: Reconstruct the object to be reconstructed according to the matching result.

[0139] The objects to be reconstructed can be farmland, roads, buildings, etc.

[0140] The embodiment of the present application provides an image reconstruction method, which divides the input image into a first grid based on a spatial division method to obtain multiple first grids, and screens the feature points in each first grid according to the statistical confidence score to retain the feature points with the highest statistical confidence score in each first grid. Since the feature points with higher statistical confidence scores are closer to the center of the first grid, the feature point extraction method provided by the embodiment of the present application can improve the uniformity of the distribution of all feature points finally obtained, avoid the situation where the feature points cannot be extracted from the repeated texture area in the input image, and effectively prevent the feature points from clumping.

[0141] The feature point extraction method provided in the embodiment of the present application can be used not only in the field of three-dimensional reconstruction, but also in fields that require feature point extraction, such as moving target tracking, object recognition, image registration, and panoramic image stitching.

[0142] Figure 6a The figure shows a schematic diagram of an extraction result obtained by using a traditional feature point extraction method. Figure 6b Shown is a schematic diagram of an extraction result obtained by using a feature point extraction method provided by an embodiment of the present application. Figure 7a Shown is a schematic diagram of another extraction result obtained by using a traditional feature point extraction method. Figure 7b FIG. 2 is a schematic diagram showing another extraction result obtained by using the feature point extraction method provided by an embodiment of the present application. Figure 6a and Figure 6b ,as well as Figure 7a and Figure 7b It can be seen that the feature point extraction method provided in the embodiment of the present application can extract feature points in almost all areas of the entire image, ensuring the uniform distribution of feature points and providing reliable data input for subsequent motion estimation and three-dimensional reconstruction.

[0143] Exemplary Devices

[0144] Figure 8 FIG. 8 is a schematic diagram of a structure of a feature point extraction device 800 provided by an exemplary embodiment of the present application. Figure 8As shown, the extraction device 800 includes: a first determination module 810 , a first division module 820 , a second determination module 830 and a screening module 840 .

[0145] The first determination module 810 is used to determine the number of multiple feature points initially extracted from the input image. The first division module 820 is used to perform a first grid division on the multiple feature points based on a spatial division method to obtain K first grids when the number of the multiple feature points is greater than the expected number of feature points, wherein K is greater than or equal to the expected number of feature points. The second determination module 830 is used to determine the number of feature points of each first grid in the K first grids, and when the number of feature points in the first grid is greater than 1, determine the statistical confidence score of each feature point in the first grid based on the distance of each feature point from the center of the first grid. The screening module 840 is used to screen the feature points of the first grid based on the statistical confidence scores of each feature point to obtain the feature point with the highest statistical confidence score in the first grid.

[0146] The embodiment of the present application provides a feature point extraction device, which divides the input image into a first grid based on a space division method to obtain multiple first grids, and screens the feature points in each first grid according to the statistical confidence score to retain the feature points with the highest statistical confidence score in each first grid. Since the feature points with higher statistical confidence scores are closer to the center of the first grid, the feature point extraction method provided by the embodiment of the present application can improve the uniformity of the distribution of all feature points finally obtained, avoid the situation where the feature points cannot be extracted from the repeated texture area in the input image, and effectively prevent the feature points from clumping.

[0147] According to an embodiment of the present application, the second determination module 830 is used to determine the statistical confidence score of each feature point based on the distance of each feature point in the first grid from the center of the first grid and the response value of each feature point.

[0148] According to an embodiment of the present application, the second determination module 830 is used to determine the statistical confidence score of each feature point by weighted summing the inverse of the distance of each feature point in the first grid from the center of the first grid and the response value of each feature point.

[0149] According to an embodiment of the present application, the first grid division is performed based on a binary tree, quadtree or octree partitioning method.

[0150] According to one embodiment of the present application, the extraction device 800 also includes a second division module 850, which is used to: divide the input image into a second grid based on a preset grid size to obtain T second grids, where T is an integer greater than 1; and perform preliminary extraction of feature points for each of the T second grids to obtain multiple feature points.

[0151] According to an embodiment of the present application, the second division module 850 is also used to construct an image pyramid for the input image. The second division module 850 is used to: extract extreme value points from the second grid based on the image pyramid and the initial threshold; when the number of extreme value points is 0, obtain a first update threshold based on the first rule and the initial threshold, wherein the first update threshold is less than the initial threshold; re-extract extreme value points from the second grid based on the image pyramid and the first update threshold, wherein the extreme value points extracted from the second grid are used as feature points.

[0152] According to one embodiment of the present application, the second division module 850 is used to: when the number of extreme points is greater than a threshold value, obtain a second update threshold based on a second rule and an initial threshold, wherein the second update threshold is greater than the initial threshold; and re-extract extreme points from the second grid based on the image pyramid and the second update threshold.

[0153] According to one embodiment of the present application, the extraction device 800 also includes a computing module 860, which is used to: perform a scale pyramid pooling operation on the input image to obtain multiple images in different scale spaces; and calculate the feature descriptor corresponding to the feature point with the highest statistical confidence score in the first grid in the images of the multiple different scale spaces.

[0154] It should be understood that the operations and functions of the first determination module 810, the first division module 820, the second determination module 830, the screening module 840, the second division module 850 and the calculation module 860 in the above embodiment can refer to the above Figure 2 or Figure 3 The description of the feature point extraction method provided in the embodiment will not be repeated here to avoid repetition.

[0155] Fig. 9 FIG. 9 is a schematic diagram of the structure of a feature matching device 900 provided by an exemplary embodiment of the present application. Fig. 9 As shown, the feature matching device 900 includes: a first extraction module 910 , a second extraction module 920 and a matching module 930 .

[0156] The first extraction module 910 is used to extract feature points in the first image using the feature point extraction method provided in the above embodiment, and calculate the first feature descriptor corresponding to the feature points in the first image. The second extraction module 920 is used to extract feature points in the second image using the feature point extraction method provided in the above embodiment, and calculate the second feature descriptor corresponding to the feature points in the second image. The matching module 930 is used to perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result.

[0157] The embodiment of the present application provides a feature matching device, which divides the input image into a first grid based on a space division method to obtain multiple first grids, and screens the feature points in each first grid according to the statistical confidence score to retain the feature points with the highest statistical confidence score in each first grid. Since the feature points with higher statistical confidence scores are closer to the center of the first grid, the feature point extraction method provided by the embodiment of the present application can improve the uniformity of the distribution of all feature points finally obtained, avoid the situation where the feature points cannot be extracted from the repeated texture area in the input image, and effectively prevent the feature points from clumping.

[0158] It should be understood that the operations and functions of the first extraction module 910, the second extraction module 920 and the matching module 930 in the above embodiment can refer to the above Figure 4 The description of the feature matching method provided in the embodiment will not be repeated here to avoid repetition.

[0159] Fig.10 FIG. 1 is a schematic diagram of the structure of an image reconstruction device 1000 provided by an exemplary embodiment of the present application. Fig.10 As shown, the image reconstruction device 1000 includes: a first extraction module 1010 , a second extraction module 1020 , a matching module 1030 and a reconstruction module 1040 .

[0160] The first extraction module 1010 is used to extract feature points in the first image using the feature point extraction method provided in the above embodiment, and calculate the first feature descriptor corresponding to the feature points in the first image. The second extraction module 1020 is used to extract feature points in the second image using the feature point extraction method provided in the above embodiment, and calculate the second feature descriptor corresponding to the feature points in the second image. The matching module 1030 is used to perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result. The reconstruction module 1040 is used to reconstruct the object to be reconstructed according to the matching result.

[0161] The embodiment of the present application provides an image reconstruction method, which divides the input image into a first grid based on a spatial division method to obtain multiple first grids, and screens the feature points in each first grid according to the statistical confidence score to retain the feature points with the highest statistical confidence score in each first grid. Since the feature points with higher statistical confidence scores are closer to the center of the first grid, the feature point extraction method provided by the embodiment of the present application can improve the uniformity of the distribution of all feature points finally obtained, avoid the situation where the feature points cannot be extracted from the repeated texture area in the input image, and effectively prevent the feature points from clumping.

[0162] It should be understood that the operations and functions of the first extraction module 1010, the second extraction module 1020, the matching module 1030, and the reconstruction module 1040 in the above embodiment can refer to the above Figure 5 In order to avoid repetition, the description of the image reconstruction method provided in the embodiment will not be repeated here.

[0163] Fig.11 FIG. 1 is a block diagram of an unmanned device 1100 for executing a feature point extraction method provided by an exemplary embodiment of the present application.

[0164] Reference Fig.11 , the unmanned device 1100 includes a processing component 1110, which further includes one or more processors, and a memory resource represented by a memory 1120 for storing instructions executable by the processing component 1110, such as an application. The application stored in the memory 1120 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1110 is configured to execute instructions to perform the above-mentioned feature point extraction method, feature matching method, or image reconstruction method. In one embodiment, the unmanned device may be a drone.

[0165] The unmanned device 1100 may also include a power supply component configured to perform power management of the unmanned device 1100, a wired or wireless network interface configured to connect the unmanned device 1100 to a network, and an input / output (I / O) interface. The unmanned device 1100 may be operated based on an operating system stored in the memory 1120, such as Windows Server 2008. TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0166] A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the unmanned device 1100, enables the unmanned device 1100 to perform a feature point extraction method, a feature matching method, or an image reconstruction method. The feature point extraction method includes: determining the number of multiple feature points initially extracted from an input image; when the number of multiple feature points is greater than the expected number of feature points, performing a first grid division on the input image based on a space division method to obtain K first grids, wherein K is greater than or equal to the expected number of feature points; determining the number of feature points of each first grid in the K first grids, and when the number of feature points in the first grid is greater than 1, determining the statistical confidence score of each feature point based on the distance of each feature point in the first grid from the center of the first grid; screening the feature points of the first grid based on the statistical confidence scores of each feature point to obtain the feature point with the highest statistical confidence score in the first grid. The feature matching method includes: extracting feature points in the first image using the feature point extraction method provided in the above embodiment, and calculating the first feature descriptor corresponding to the feature points in the first image; extracting feature points in the second image using the feature point extraction method provided in the above embodiment, and calculating the second feature descriptor corresponding to the feature points in the second image; performing feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result. The image reconstruction method includes: extracting feature points in the first image using the feature point extraction method provided in the above embodiment, and calculating the first feature descriptor corresponding to the feature points in the first image; extracting feature points in the second image using the feature point extraction method provided in the above embodiment, and calculating the second feature descriptor corresponding to the feature points in the second image; performing feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result; and reconstructing the object to be reconstructed according to the matching result.

[0167] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, and will not be described one by one here.

[0168] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0169] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0170] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0171] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0172] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0173] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program check codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0174] It should be noted that, in the description of this application, the terms "first", "second", "third", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.

[0175] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for extracting feature points, characterized in that: include: Determine the number of multiple feature points initially extracted from the input image; When the number of the plurality of feature points is greater than the expected number of feature points, performing a first grid division on the input image based on a space division method to obtain K first grids, wherein K is greater than or equal to the expected number of feature points; Determine the number of feature points of each of the K first grids, and when the number of feature points of the first grid is greater than 1, determine a statistical confidence score of each feature point based on a distance of each feature point in the first grid from a center of the first grid; The feature points of the first grid are screened based on the statistical confidence scores of the respective feature points to obtain the feature points with the highest statistical confidence scores in the first grid.

2. The method for extracting feature points according to claim 1, characterized in that: The determining the statistical confidence score of each feature point based on the distance between each feature point in the first grid and the center of the first grid comprises: Based on the distance of each feature point in the first grid from the center of the first grid and the response value of each feature point, a statistical confidence score of each feature point is determined.

3. The method for extracting feature points according to claim 2, characterized in that: Determining the statistical confidence score of each feature point based on the distance of each feature point in the first grid from the center of the first grid and the response value of each feature point includes: The statistical confidence score of each feature point is determined by performing weighted summation on the inverse of the distance of each feature point in the first grid from the center of the first grid and the response value of each feature point.

4. The method for extracting feature points according to claim 1, characterized in that: The first grid division is performed based on a binary tree, quadtree or octree partitioning method.

5. The method for extracting feature points according to any one of claims 1 to 4, characterized in that: Also includes: Dividing the input image into a second grid based on a preset grid size to obtain T second grids, where T is an integer greater than 1; Preliminary feature point extraction is performed on each of the T second grids to obtain the plurality of feature points.

6. The method for extracting feature points according to claim 5, characterized in that: Also includes: constructing an image pyramid for the input image, The preliminary extraction of feature points for each of the T second grids to obtain the plurality of feature points includes: Extracting extreme points from the second grid based on the image pyramid and an initial threshold; When the number of extreme value points is 0, obtaining a first update threshold based on a first rule and the initial threshold, wherein the first update threshold is less than the initial threshold; The extreme points of the second grid are re-extracted based on the image pyramid and the first update threshold, wherein the extreme points extracted from the second grid are used as feature points.

7. The method for extracting feature points according to claim 6, characterized in that: The performing preliminary extraction of feature points for each of the T second grids to obtain the plurality of feature points further includes: When the number of extreme value points is greater than a threshold value, obtaining a second update threshold value based on a second rule and the initial threshold value, wherein the second update threshold value is greater than the initial threshold value; The extreme points of the second grid are extracted again based on the image pyramid and the second update threshold.

8. The method for extracting feature points according to any one of claims 1 to 4, characterized in that: Also includes: Performing a scale pyramid pooling operation on the input image to obtain multiple images in different scale spaces; The feature descriptor corresponding to the feature point with the highest statistical confidence score in the first grid is calculated in the images of the multiple different scale spaces.

9. A method for reconstructing an image, characterized in that: include: Extracting feature points in the first image using the feature point extraction method described in any one of claims 1 to 8, and calculating first feature descriptors corresponding to the feature points in the first image; Extracting feature points in the second image using the feature point extraction method described in any one of claims 1 to 8, and calculating second feature descriptors corresponding to the feature points in the second image; Performing feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result; The object to be reconstructed is reconstructed according to the matching result.

10. A feature point extraction device, characterized in that: include: A first determination module is used to determine the number of multiple feature points initially extracted from the input image; A first division module is used for performing a first grid division on the plurality of feature points based on a space division method to obtain K first grids when the number of the plurality of feature points is greater than the expected number of feature points, wherein K is greater than or equal to the expected number of feature points; A second determination module is used to determine the number of feature points of each first grid in the K first grids, and when the number of feature points of the first grid is greater than 1, determine the statistical confidence score of each feature point in the first grid based on the distance of each feature point in the first grid from the center of the first grid; A screening module is used to screen the feature points of the first grid based on the statistical confidence scores of the respective feature points to obtain the feature points with the highest statistical confidence scores in the first grid.

11. An image reconstruction device, characterized in that: include: A first extraction module, configured to extract feature points in a first image using the feature point extraction method according to any one of claims 1 to 8, and calculate a first feature descriptor corresponding to the feature points in the first image; A second extraction module, configured to extract feature points in a second image by using the feature point extraction method according to any one of claims 1 to 8, and calculate a second feature descriptor corresponding to the feature points in the second image; a matching module, configured to perform feature matching on the first image and the second image based on the first feature descriptor and the second feature descriptor to obtain a matching result; A reconstruction module is used to reconstruct the object to be reconstructed according to the matching result.

12. An unmanned device, characterized in that: include: processor; a memory for storing instructions executable by the processor, The processor is used to execute the feature point extraction method described in any one of claims 1 to 8, or to execute the image reconstruction method described in claim 9.

13. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the feature point extraction method described in any one of claims 1 to 8, or the image reconstruction method described in claim 9.

Images