Feature point extraction method and device, computer device and readable storage medium

Abstract

Embodiments of the present application disclose a feature point extraction method and device, computer equipment and a readable storage medium. The method comprises: obtaining a first corner point in a current frame image collected by an image collection device by using an optical flow method; calculating a pose of the image collection device by using the first corner point; projecting a first map point corresponding to the first corner point to a pixel plane by using the pose, and calculating a distance between a theoretical projection position and an actual projection position of the first map point on the pixel plane, taking the first map point with a distance less than a preset distance threshold as an inlier; judging whether the number of the inliers is greater than a preset number threshold and a ratio of the number of the inliers to the number of the first map points is greater than a preset ratio threshold; if yes, taking the first corner point as an extraction result; if no, taking the current frame image as a key frame, and updating the first corner point by using feature point detection, and taking the updated first corner point as the extraction result. The present application does not need to extract and match feature points for each frame, thereby improving the calculation efficiency.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to a feature point extraction method, apparatus, computer device, and readable storage medium. Background Technology

[0002] Visual simultaneous localization and mapping (vSLAM) systems can acquire image information from the environment using an image acquisition device, infer the movement of the image acquisition device within the environment, and thus achieve localization of the image acquisition device. Simultaneously, they will model the entire environment, creating a globally consistent map.

[0003] However, existing visual simultaneous localization and mapping systems require a large amount of computational resources when extracting feature points, resulting in low computational efficiency. Summary of the Invention

[0004] In view of this, the purpose of the embodiments of this application is to provide a feature point extraction method, apparatus, computer device and readable storage medium, which can solve the problem of low computational efficiency in existing visual synchronous positioning and mapping systems when performing feature point extraction.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a feature point extraction method, including:

[0007] The first corner point in the current frame image acquired by the image acquisition device is obtained using the optical flow method;

[0008] The pose of the image acquisition device is calculated using the first corner point;

[0009] Using the pose, the first map point corresponding to the first corner point is projected onto the pixel plane, and the distance between the theoretical projection position and the actual projection position of the first map point on the pixel plane is calculated. The first map point whose distance is less than a preset distance threshold is taken as an interior point.

[0010] Determine whether the number of interior points is greater than a preset number threshold and whether the ratio of the number of interior points to the number of the first map points is greater than a preset ratio threshold;

[0011] If so, then the first corner point is taken as the extraction result;

[0012] If not, the current frame image is used as a keyframe, and the first corner point is updated using feature point detection. The updated first corner point is then used as the extraction result.

[0013] According to a specific embodiment disclosed in this application, the step of updating the first corner point using feature point detection and using the updated first corner point as the extraction result includes:

[0014] An image pyramid is created using the current frame image, and all second corner points on the image pyramid are extracted;

[0015] The first corner point is corrected using the second corner point to obtain the third corner point;

[0016] Extract the fourth corner point that meets the preset conditions using all map points in the current frame image;

[0017] Based on the number of the third and fourth corner points, the total number of the first corner points is supplemented to the preset number of corner points;

[0018] The first corner point, representing a preset number of corner points, is used as the extraction result.

[0019] According to a specific embodiment disclosed in this application, the step of correcting the first corner point using the second corner point to obtain the third corner point includes:

[0020] Based on the position information of the first corner point, determine the fifth corner point within a preset range among all the second corner points, and calculate the descriptor of the fifth corner point;

[0021] Based on the descriptor of the fifth corner point, the fifth corner point that is closest to the first corner point is determined and designated as the third corner point.

[0022] According to a specific embodiment disclosed in this application, the step of extracting the fourth corner point that meets preset conditions using all map points in the current frame image includes:

[0023] Based on the pose of the image acquisition device, the current frame image is spatially transformed to obtain a three-dimensional scene, and all map points in the three-dimensional scene are used as candidate map points.

[0024] The candidate map points are projected onto the camera plane. Based on the position information of the candidate map points, the sixth corner point within a preset range is determined from all candidate map points, and the descriptor of the sixth corner point is calculated.

[0025] Based on the descriptor of the sixth corner point, the seventh corner point, which is closest to the candidate map point among the sixth corner points, is determined as the fourth corner point.

[0026] According to a specific embodiment disclosed in this application, the step of supplementing the total number of the first corner points to a preset number of corner points based on the number of the third corner point and the fourth corner point includes:

[0027] Based on the preset number of corner points in the current frame image, the number of the third corner points, and the number of the fourth corner points, determine the number of corner points to be supplemented.

[0028] Remove the corner points from the second corner points whose distance to the third and fourth corner points is less than a preset distance threshold to obtain the remaining corner points;

[0029] Select the number of corner points to be supplemented from the remaining corner points to supplement the total number of the first corner points to the preset number of corner points.

[0030] According to one specific embodiment disclosed in this application, the image pyramid has 8 layers and a scale of 1.2.

[0031] According to a specific embodiment disclosed in this application, the method further includes:

[0032] A map is constructed using the extracted results.

[0033] Secondly, embodiments of this application provide a feature point extraction device, comprising:

[0034] The corner point acquisition module is used to acquire the first corner point in the current frame image acquired by the image acquisition device using the optical flow method;

[0035] The pose calculation module is used to calculate the pose of the image acquisition device using the first corner point;

[0036] The inlier determination module is used to project the first map point corresponding to the first corner point onto the pixel plane using the pose, and calculate the distance between the theoretical projection position and the actual projection position of the first map point on the pixel plane, and take the first map point whose distance is less than a preset distance threshold as an inlier.

[0037] The judgment module is used to determine whether the number of interior points is greater than a preset number threshold and whether the ratio of the number of interior points to the number of the first map points is greater than a preset ratio threshold.

[0038] The first extraction result determination module is used to take the first corner point as the extraction result if the condition is met.

[0039] The second extraction result determination module is used to, if not, take the current frame image as a key frame, update the first corner point using feature point detection, and take the updated first corner point as the extraction result.

[0040] Thirdly, embodiments of this application provide a computer device including a processor and a memory, wherein the memory stores a program or instructions, and when the program or instructions are executed by the processor, they implement the steps of the method described in the first aspect.

[0041] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0042] The feature point extraction method provided in this embodiment uses optical flow to obtain the first corner point in the current frame image acquired by the image acquisition device; calculates the pose of the image acquisition device using the first corner point; projects the first map point corresponding to the first corner point onto the pixel plane using the pose, and calculates the distance between the theoretical projection position and the actual projection position of the first map point on the pixel plane, and designates the first map points whose distance is less than a preset distance threshold as inliers; determines whether the number of inliers is greater than a preset number threshold and whether the ratio of the number of inliers to the number of first map points is greater than a preset ratio threshold; if yes, the first corner point is used as the extraction result; if no, the current frame image is used as a keyframe, and the first corner point is updated using feature point detection, and the updated first corner point is used as the extraction result. This eliminates the need for feature point extraction and matching for every frame, improving computational efficiency and thus increasing the operating speed of the vslam system. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be considered as a limitation on the scope of protection of this application. In the various drawings, similar components are numbered similarly.

[0044] Figure 1 A schematic flowchart of a feature point extraction method provided in an embodiment of this application is shown;

[0045] Figure 2 A schematic diagram of a feature point extraction device provided in an embodiment of this application is shown. Detailed Implementation

[0046] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0047] The components of the embodiments of this application described and illustrated in the accompanying drawings can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0048] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0049] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0050] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0051] To address the low computational efficiency of existing visual simultaneous localization and mapping (VSMR) systems during feature point extraction, this application provides a feature point extraction method. Please refer to... Figure 1 , Figure 1 This is a flowchart illustrating the feature point extraction method provided in an embodiment of this application. The method includes the following steps:

[0052] Step 110: Use optical flow to obtain the first corner point in the current frame image acquired by the image acquisition device.

[0053] Specifically, image acquisition devices are typically installed in electronic devices such as drones and robotic vacuum cleaners to collect image information of the environment surrounding the electronic device in real time and obtain video images. The image acquisition device can be any of a monocular camera, a binocular camera, or an RGB-D camera, and can be set according to actual needs; this application embodiment does not limit this.

[0054] Understandably, after an image acquisition device captures an image, it can be preprocessed. Preprocessing may include filtering and / or distortion correction. Filtering removes noise and reduces interference, while distortion correction reduces image distortion and restores the true scene. Preprocessing improves image quality, thereby enhancing the accuracy of subsequent processing.

[0055] In a specific implementation, the Lucas Kanade optical flow algorithm can be used to search and match the first corner point in the current frame image. A corner point is an extreme point, that is, a point with particularly prominent attributes in a certain aspect; it is an isolated point or the endpoint of a line segment where the intensity of certain attributes is the highest or lowest. For images, corner points generally refer to the connection points of object contour lines. Corner points are very important features of images and play a crucial role in the understanding and analysis of image graphics. While preserving important image graphics features, corner points can effectively reduce the amount of data, resulting in high information content, effectively improving computational speed, facilitating reliable image matching, and enabling real-time processing. For the same scene, even if the viewpoint changes, corner points usually possess stable properties. It is precisely because of this stable property that corner points are applied in computer vision fields such as motion estimation for 3D scene reconstruction, object tracking, object recognition, and image registration and matching. Meanwhile, since optical flow estimates the movement of a moving object by comparing the differences between two consecutive frames, the Lucas Kanade optical flow algorithm needs to know the coordinates of the corner point to be acquired in the previous frame beforehand. Therefore, this step actually starts from the second frame of the video captured by the image acquisition device, while the initial feature points need to be calculated for the first frame.

[0056] Step 120: Calculate the pose of the image acquisition device using the first corner point.

[0057] Specifically, after step 110, the first corner point in the current frame image is associated with the corner point in the previous frame image. Since there is a correspondence between the corner point in the previous frame image and the map point, the correspondence between the first corner point in the current frame image and the map point can be obtained. Then, using the coordinates of the map point and the coordinates of the first corner point, the pose of the image acquisition device is obtained using the PnP (Perspective-n-Point) method. The PnP method is an algorithm that uses multiple pairs of 3D (3-Dimensional) and 2D matching points to solve for the extrinsic parameters of the image acquisition device by minimizing the reprojection error, given that the intrinsic parameters of the image acquisition device are known or unknown.

[0058] Step 130: Using the pose, project the first map point corresponding to the first corner point onto the pixel plane, and calculate the distance between the theoretical projection position and the actual projection position of the first map point on the pixel plane. Take the first map point whose distance is less than a preset distance threshold as an interior point.

[0059] Specifically, using the pose of the image acquisition device obtained in step 120, the first map point corresponding to the first corner point in the current frame can be projected onto the pixel plane. Then, the theoretical projection position and actual projection position of each first map point on the pixel plane are calculated, and first map points whose distance between the theoretical and actual projection positions is less than a preset distance threshold are designated as interior points. Interior points indicate that the deviation between the theoretical and actual projection positions is small.

[0060] It is understood that the preset distance threshold can be 5.991 pixels, and can be set according to actual needs. This application embodiment does not limit this.

[0061] Step 140: Determine whether the number of interior points is greater than a preset number threshold and whether the ratio of the number of interior points to the number of the first map points is greater than a preset ratio threshold.

[0062] Specifically, the optical flow tracking effect can be judged based on the number of interior points. In a specific embodiment, the judgment is made by considering whether the number of interior points is greater than a preset threshold, and whether the ratio of the number of interior points to the number of the first map points is greater than a preset ratio threshold.

[0063] It is understood that the preset quantity threshold and the preset ratio threshold can be set according to actual needs, and this application embodiment does not limit them. For example, the preset quantity threshold can be 50, and the preset ratio threshold can be 50%.

[0064] Step 150: If yes, then the first corner point is taken as the extraction result.

[0065] Specifically, if the number of interior points is greater than a preset threshold, and the ratio of the number of interior points to the number of the first map points is greater than a preset ratio threshold, it indicates that the optical flow tracking is in good condition, and the first corner point can be directly used as the extraction result to perform optical flow tracking on the next frame of the current frame image.

[0066] Step 160: If not, then the current frame image is used as a keyframe, and the first corner point is updated using feature point detection, and the updated first corner point is used as the extraction result.

[0067] Specifically, if the number of inliers is less than or equal to a preset threshold, and / or the ratio of the number of inliers to the number of the first map points is less than or equal to a preset ratio threshold, it indicates that the optical flow tracking is in poor condition, and the current frame image is designated as a keyframe. Furthermore, since the current frame image is determined to be a keyframe, the first corner point needs to be updated, and the updated first corner point is used as the extraction result.

[0068] In one optional implementation, updating the first corner point and using the updated first corner point as the extraction result includes:

[0069] Create an image pyramid using the current frame image, and extract all second corner points on the image pyramid;

[0070] The first corner point is corrected using the second corner point to obtain the third corner point;

[0071] Extract the fourth corner point that meets the preset conditions using all map points in the current frame image;

[0072] Based on the number of the third and fourth corner points, the total number of the first corner points is supplemented to the preset number of corner points;

[0073] The first corner point, representing a preset number of corner points, is used as the extraction result.

[0074] Specifically, an image pyramid is a multi-scale representation of an image, primarily used for image segmentation. It's an efficient yet conceptually simple structure for interpreting images at multiple resolutions. Simply put, an image pyramid is a collection of sub-images of the same image at different resolutions. The base of the image pyramid is a high-resolution representation of the image to be processed, while the top is a low-resolution approximation. The higher the level, the smaller the image and the lower the resolution. In a specific embodiment, the image pyramid can have 8 levels, with a ratio of 1.2.

[0075] Then, a corner extraction algorithm is used to extract all second corner points on the image pyramid. In a specific embodiment, the corner extraction algorithm used can be FAST (Features from Accelerated Segment Test). It is understood that other corner extraction algorithms can also be used, such as SIFT (Scale-invariant feature transform) and SURF (Speeded Up Robust Features), which can be set according to actual needs. This application embodiment does not limit this.

[0076] Considering that the corner point positions may gradually become less accurate after multiple frames of optical flow tracking, it is necessary to recalibrate the corner point coordinates. In a specific embodiment, the first corner point is corrected using the second corner point to obtain the third corner point; it can be understood that the third corner point is a part of the first corner point.

[0077] Then, the fourth corner point that meets the preset conditions is extracted using all map points in the current frame image; based on the number of the third and fourth corner points, the total number of corner points is supplemented to the preset number of corner points; the corner points of the preset number of corner points are used as the extraction result. It can be understood that the fourth corner point is part of the second corner point.

[0078] In one optional implementation, the step of correcting the first corner point using the second corner point to obtain the third corner point includes:

[0079] Based on the position information of the first corner point, determine the fifth corner point within a preset range among all the second corner points, and calculate the descriptor of the fifth corner point;

[0080] Based on the descriptor of the fifth corner point, the fifth corner point that is closest to the first corner point is determined and designated as the third corner point.

[0081] Specifically, each first corner point includes four elements: [u, v, desc, layer]. Here, u and v are the coordinates of the first corner point on the current frame image, desc is the descriptor of the first corner point, and layer indicates the layer number of the pyramid containing the first corner point. Among all second corner points, a fifth corner point is determined within a preset range: [u±s, v±s, layer±1]. This preset range means a rectangle is constructed using the coordinates of the first corner point and is located within the layer above and below the first corner point. The value of s can be 5 pixels, and can be set according to actual needs; this embodiment does not limit this. It can be understood that the fifth corner point is a part of the second corner points, and the third corner point is obtained by filtering the fifth corner points.

[0082] Next, calculate the descriptor for the fifth corner point. A descriptor describes the characteristics of a corner point within a certain region.

[0083] Finally, the fifth corner point, which is closest to the first corner point, is selected as the third corner point. It is understood that if the calculated distance between the fifth corner point and the first corner point is greater than a preset distance threshold, the fifth corner point is discarded. It is also understood that the preset distance threshold can be 50, and can be set according to actual needs; this embodiment does not limit this setting.

[0084] In this specific embodiment, Hamming distance is used for distance calculation. It is understood that other distances can also be used for distance calculation, and this application does not limit this approach.

[0085] In one optional implementation, extracting the fourth corner point that meets preset conditions using all map points in the current frame image includes:

[0086] Based on the pose of the image acquisition device, the current frame image is spatially transformed to obtain a three-dimensional scene, and all map points in the three-dimensional scene are used as candidate map points.

[0087] The candidate map points are projected onto the camera plane. Based on the position information of the candidate map points, the sixth corner point within a preset range is determined from all candidate map points, and the descriptor of the sixth corner point is calculated.

[0088] Based on the descriptor of the sixth corner point, the seventh corner point, which is closest to the candidate map point among the sixth corner points, is determined as the fourth corner point.

[0089] Specifically, the current frame image is a two-dimensional scene, which needs to be converted into a three-dimensional scene. All map points in the three-dimensional scene are used as candidate map points. These candidate map points are then projected onto the camera plane. The subsequent logic is similar to the previous steps and will not be repeated here. It can be understood that the sixth corner point is part of the second corner point and is an intermediate result in calculating the fourth corner point; the sixth corner point is filtered to obtain the seventh corner point, which is then used as the fourth corner point.

[0090] Since map points do not have a layer attribute, they do have a size attribute, which represents the actual size of the map point in scale space. If the distance between the map point and the image acquisition device is d, then size / d is the projected size of the map point on the pixel plane. The number of layers of the map point in the image pyramid can be calculated from the projected size.

[0091] In one optional implementation, the step of supplementing the total number of the first corner points to a preset number of corner points based on the number of the third corner points and the fourth corner points includes:

[0092] Based on the preset number of corner points in the current frame image, the number of the third corner points, and the number of the fourth corner points, determine the number of corner points to be supplemented.

[0093] Remove the corner points from the second corner points whose distance to the third and fourth corner points is less than a preset distance threshold to obtain the remaining corner points;

[0094] Select the number of corner points to be supplemented from the remaining corner points to supplement the total number of the first corner points to the preset number of corner points.

[0095] Specifically, taking a preset number of corner points of 400 as an example, if the number of third corner points is 100 and the number of fourth corner points is 200, then the number of corner points to be supplemented is 100. First, corner points from the second set of corner points that are less than a preset distance threshold from the third and fourth corner points are removed, resulting in the remaining corner points. In other words, corner points that are too close to the third and fourth corner points are removed, thereby avoiding duplicate corner point acquisition and improving the accuracy of corner point acquisition.

[0096] Then, select the required number of corner points to be supplemented from the remaining corner points. In a specific embodiment, a quadtree is used for selecting and supplementing the corner points. It can be understood that using a quadtree to select corner points can make the selected corner points more evenly distributed.

[0097] By supplementing and improving the first corner point, the number of the first corner point is increased, which means that the number of detectable corner points in the current frame image is increased for the next frame image.

[0098] In one optional implementation, the method further includes:

[0099] A map is constructed using the extracted results.

[0100] Specifically, because the accuracy and computational efficiency of the extraction results are improved, the accuracy of the map is enhanced, and a globally consistent map can be maintained.

[0101] The feature point extraction method provided in this embodiment uses optical flow to obtain the first corner point in the current frame image acquired by the image acquisition device; calculates the pose of the image acquisition device using the first corner point; projects the first map point corresponding to the first corner point onto the pixel plane using the pose, and calculates the distance between the theoretical projection position and the actual projection position of the first map point on the pixel plane, and designates the first map points whose distance is less than a preset distance threshold as inliers; determines whether the number of inliers is greater than a preset number threshold and whether the ratio of the number of inliers to the number of first map points is greater than a preset ratio threshold; if yes, the first corner point is used as the extraction result; if no, the current frame image is used as a keyframe, and the first corner point is updated using feature point detection, and the updated first corner point is used as the extraction result. This eliminates the need for feature point extraction and matching for every frame, improving computational efficiency and thus increasing the operating speed of the vslam system.

[0102] For a method embodiment corresponding to the above, please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram of the feature point extraction device provided in an embodiment of this application. The feature point extraction device 1000 includes:

[0103] The corner point acquisition module 1010 is used to acquire the first corner point in the current frame image acquired by the image acquisition device using the optical flow method;

[0104] Pose calculation module 1020 is used to calculate the pose of the image acquisition device using the first corner point;

[0105] The interior point determination module 1030 is used to project the first map point corresponding to the first corner point onto the pixel plane using the pose, and calculate the distance between the theoretical projection position and the actual projection position of the first map point on the pixel plane, and take the first map point whose distance is less than a preset distance threshold as an interior point.

[0106] The judgment module 1040 is used to determine whether the number of interior points is greater than a preset number threshold and whether the ratio of the number of interior points to the number of the first map points is greater than a preset ratio threshold.

[0107] The first extraction result determination module 1050 is used to determine the first corner point as the extraction result if the condition is met.

[0108] The second extraction result determination module 1060 is used to, if not, take the current frame image as a key frame, update the first corner point using feature point detection, and take the updated first corner point as the extraction result.

[0109] The extraction method and apparatus provided in this application embodiment can implement each process of the feature point extraction method in the above method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0110] Optionally, this application embodiment also provides a computer device, including a processor and a memory, wherein the memory stores a program or instructions, which, when executed by the processor, implement the various processes of the above-described feature point extraction method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0111] Optionally, embodiments of this application also provide a computer-readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described feature point extraction method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0112] The processor is the processor in the computer device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0114] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0115] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A feature point extraction method, characterized in that, include: The first corner point in the current frame image acquired by the image acquisition device is obtained using the optical flow method; The pose of the image acquisition device is calculated using the first corner point; Using the pose, the first map point corresponding to the first corner point is projected onto the pixel plane, and the distance between the theoretical projection position and the actual projection position of the first map point on the pixel plane is calculated. The first map point whose distance is less than a preset distance threshold is taken as an interior point. Determine whether the number of interior points is greater than a preset number threshold and whether the ratio of the number of interior points to the number of the first map points is greater than a preset ratio threshold; If so, then the first corner point is taken as the extraction result; If not, the current frame image is used as a keyframe, and the first corner point is updated using feature point detection, and the updated first corner point is used as the extraction result. The step of updating the first corner point using feature point detection and using the updated first corner point as the extraction result includes: An image pyramid is created using the current frame image, and all second corner points on the image pyramid are extracted; The first corner point is corrected using the second corner point to obtain the third corner point; Extract the fourth corner point that meets the preset conditions using all map points in the current frame image; Based on the number of the third and fourth corner points, the total number of the first corner points is supplemented to the preset number of corner points; The first corner point, representing a preset number of corner points, is used as the extraction result.

2. The feature point extraction method according to claim 1, characterized in that, The step of correcting the first corner point using the second corner point to obtain the third corner point includes: Based on the position information of the first corner point, determine the fifth corner point within a preset range among all the second corner points, and calculate the descriptor of the fifth corner point; Based on the descriptor of the fifth corner point, the fifth corner point that is closest to the first corner point is determined and designated as the third corner point.

3. The feature point extraction method according to claim 1, characterized in that, The step of extracting the fourth corner point that meets the preset conditions using all map points in the current frame image includes: Based on the pose of the image acquisition device, the current frame image is spatially transformed to obtain a three-dimensional scene, and all map points in the three-dimensional scene are used as candidate map points. The candidate map points are projected onto the camera plane. Based on the position information of the candidate map points, the sixth corner point within a preset range is determined from all candidate map points, and the descriptor of the sixth corner point is calculated. Based on the descriptor of the sixth corner point, the seventh corner point, which is closest to the candidate map point among the sixth corner points, is determined as the fourth corner point.

4. The feature point extraction method according to claim 1, characterized in that, The step of supplementing the total number of the first corner points to the preset number of corner points based on the number of the third corner points and the fourth corner points includes: Based on the preset number of corner points in the current frame image, the number of the third corner points, and the number of the fourth corner points, determine the number of corner points to be supplemented. Remove the corner points from the second corner points whose distance to the third and fourth corner points is less than a preset distance threshold to obtain the remaining corner points; Select the number of corner points to be supplemented from the remaining corner points to supplement the total number of the first corner points to the preset number of corner points.

5. The feature point extraction method according to claim 1, characterized in that, The image pyramid has 8 layers.

6. The feature point extraction method according to claim 1, characterized in that, The method further includes: A map is constructed using the extracted results.

7. A feature point extraction device, characterized in that, include: The corner point acquisition module is used to acquire the first corner point in the current frame image acquired by the image acquisition device using the optical flow method; The pose calculation module is used to calculate the pose of the image acquisition device using the first corner point; The inlier determination module is used to project the first map point corresponding to the first corner point onto the pixel plane using the pose, and calculate the distance between the theoretical projection position and the actual projection position of the first map point on the pixel plane, and take the first map point whose distance is less than a preset distance threshold as an inlier. The judgment module is used to determine whether the number of interior points is greater than a preset number threshold and whether the ratio of the number of interior points to the number of the first map points is greater than a preset ratio threshold. The first extraction result determination module is used to take the first corner point as the extraction result if the condition is met. The second extraction result determination module is used to, if not, take the current frame image as a key frame, update the first corner point using feature point detection, and take the updated first corner point as the extraction result. The step of updating the first corner point using feature point detection and using the updated first corner point as the extraction result includes: An image pyramid is created using the current frame image, and all second corner points on the image pyramid are extracted; The first corner point is corrected using the second corner point to obtain the third corner point; Extract the fourth corner point that meets the preset conditions using all map points in the current frame image; Based on the number of the third and fourth corner points, the total number of the first corner points is supplemented to the preset number of corner points; The first corner point, representing a preset number of corner points, is used as the extraction result.

8. A computer device, characterized in that, It includes a processor and a memory, wherein the memory stores a program or instructions that, when executed by the processor, implement the steps of the feature point extraction method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the feature point extraction method as described in any one of claims 1-6.

Images