Image Feature Point Selection Method and Related Devices, Equipment, and Storage Media

Abstract

The present application discloses an image feature point selection method and related devices, equipment, and storage media. Among them, the method includes: obtaining a plurality of feature points in an image frame; based on the plurality of feature points, pre-configuring a node storage space with a fixed size; using a tree-based partitioning method to partition the plurality of feature points into at least one node, and storing the first information of the node into the node storage space, where the first information of the node includes feature-related information of the feature points corresponding to the node; selecting feature points from each of the finally partitioned nodes to obtain a selection result of the plurality of feature points. Through this method, the execution efficiency of feature point selection can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and particularly to an image feature point selection method and related devices, equipment, and storage media. Background Art

[0002] Feature extraction of images is an important underlying technology in fields such as computer vision, robotics, autonomous vehicles, 3D reconstruction, and augmented reality. Considering the efficiency of subsequent feature matching and other processing of feature points, feature points obtained after image feature extraction are often selected to reduce the number of feature points.

[0003] However, the process of selecting feature points often involves dividing the feature points and then making selections. The existing feature point division process often requires dynamically and continuously applying for and releasing space, thereby affecting the execution efficiency of feature point selection and severely restricting the development of technologies such as computer vision and robotics.

[0004] Therefore, how to improve the execution efficiency of feature point selection is of great significance. Summary of the Invention

[0005] This application provides an image feature point selection method and related devices, equipment, and storage media.

[0006] A first aspect of this application provides an image feature point selection method, including: obtaining a plurality of feature points in an image frame; based on the plurality of feature points, pre-configuring a node storage space with a fixed size; dividing the plurality of feature points into at least one node by using a tree-based division method, and storing first information of the node into the node storage space, where the first information of the node includes feature-related information of the feature points corresponding to the node; selecting feature points from each of the finally divided nodes to obtain a selection result of the plurality of feature points.

[0007] Therefore, by applying for a node storage space with a fixed size and storing the first information of the node into the node storage space, during the process of dividing the plurality of feature points by using the tree-based division method, there is no need to repeatedly apply for storage space to store the first information of the node, reducing operations on the storage space, thereby improving the execution efficiency of feature point selection.

[0008] Among them, the above-mentioned pre-configuring a node storage space with a fixed size based on the plurality of feature points includes: determining a target node division number based on a target selected feature point number, where the target selected feature point number is the number of feature points to be selected from the plurality of feature points, and the target node division number is the number of nodes finally divided; applying for a node storage space with a size matching the target node division number for the plurality of feature points.

[0009] Therefore, by applying for a node storage space whose size matches the number of target node partitions, it is possible to ensure that the node storage space can store the first information of all the finally partitioned nodes, eliminating the need to apply for a new storage space subsequently, thereby reducing operations related to the storage space and improving the execution efficiency of feature point selection.

[0010] Among them, the above method of partitioning a number of feature points into at least one node using a tree-like partitioning method and storing the first information of the nodes in the node storage space includes: partitioning some or all of the feature points of the number of feature points into a node and storing the first information of the node in the node storage space; sequentially performing a partitioning step on each node in the node storage space until the node storage space no longer meets the first requirement; where the partitioning step includes: determining the node as a parent node, partitioning the feature points corresponding to the parent node into at least one new node, and storing the first information of the new node in the node storage space.

[0011] Therefore, it is possible to achieve the partitioning of feature points during the feature point selection process.

[0012] Among them, the execution order of other nodes belonging to the same parent node as the currently executed node precedes the execution order of the nodes obtained by partitioning the currently executed node.

[0013] Therefore, by determining that nodes belonging to the same parent node are preferentially partitioned, the uniformity of feature point partitioning is improved, and thus the rationality of subsequent feature selection is enhanced.

[0014] Among them, the above sequential execution of the partitioning step on each node in the node storage space includes: for the child nodes belonging to the same parent node, determining the execution order of the child nodes of the child nodes according to the number of feature points corresponding to the child nodes, and performing the partitioning step on the child nodes according to the child node execution order, where the child nodes are the nodes obtained by partitioning the parent node.

[0015] Therefore, by preferentially partitioning the nodes with a larger number of corresponding feature points, it is possible to ensure that the nodes with a larger number of corresponding feature points are preferentially partitioned, which can improve the uniformity of feature point partitioning.

[0016] Among them, after partitioning the feature points corresponding to the parent node into at least one new node, the method further includes: performing a link-like pointing between the nodes belonging to the same parent node, and making the node at the end of the link point to the node pointed to by the parent node, and making the node pointing to the parent node update its pointing to the node at the head of the link; where the pointing of the node is used to determine the execution order of the node.

[0017] Therefore, the order of subsequent node partitioning can be achieved through the link pointing method.

[0018] Among them, storing the first information of the new node in the node storage space as described above includes: storing the first information of one of the new nodes by overwriting the first information of the parent node, and storing the first information of the remaining new nodes in the remaining storage space in the node storage space.

[0019] Therefore, by storing the first information of a new node by overwriting the first information of the parent node, the reuse of the storage space is achieved, and the storage space can be saved.

[0020] Among them, determining the node as the parent node as described above includes: when the number of feature points corresponding to the node meets the second requirement, taking the node as the parent node; and / or, dividing the feature points corresponding to the parent node into at least one new node, including: dividing the parent region corresponding to the parent node into a first number of sub-regions, where the parent region is the region of the feature points corresponding to the parent node in the image frame; when the number of feature points in the sub-region meets the third requirement, generating a corresponding child node for the sub-region, where the feature points corresponding to the child node are the feature points in the sub-region; when the number of feature points in the sub-region does not meet the third requirement, not generating a corresponding child node for the sub-region.

[0021] Therefore, by judging whether the number of feature points corresponding to the node meets the second requirement, a node that does not meet the second requirement can be used as the parent node, improving the uniformity of feature point division, and also reducing the number of nodes that need to be divided, improving the execution efficiency of image feature point selection. In addition, by dividing the parent region corresponding to the parent node into a first number of sub-regions and judging whether the number of feature points in the sub-region meets the third requirement, sub-regions that do not meet the requirements can be excluded so that corresponding child nodes can be not generated, and whether to form nodes can be flexibly determined according to the feature quantity of the sub-region.

[0022] Among them, the first requirement above is that the current number of stored nodes in the node storage space is less than the target node division quantity; the second requirement is that the number of feature points corresponding to the node is greater than the first threshold; the third requirement is that the number of feature points in the sub-region is greater than the second threshold; the first number is 4 or 8.

[0023] Therefore, by determining the first requirement as that the current number of stored nodes in the node storage space is less than the target node division quantity, the step of node division can be stopped after obtaining the required number of nodes. In addition, by setting the first threshold, the number of nodes that need to be divided can be reduced. Further, by judging whether the number of feature points in the sub-region is greater than the second threshold, regions with the number of feature points less than the second threshold can be excluded. Moreover, limiting the number of sub-regions divided each time to 4 or 8 is equivalent to implementing tree-shaped division in the form of a quadtree or an octree.

[0024] Among them, the step of dividing some or all of several feature points into a node as described above includes: using the area in the image frame except for the preset boundary area as the root area; dividing the feature points in the root area into a node; and / or, sequentially performing the division step on each node in the node storage space until the node storage space no longer meets the first requirement, which is performed when it is detected that the number of features corresponding to the currently divided node is greater than the target number of selected feature points.

[0025] Therefore, by removing the edge area of the image frame, the feature points on the edge area can be removed, the number of feature points can be reduced, and the execution speed of the image feature point selection method can be accelerated. In addition, by determining whether the number of features corresponding to the currently divided node is greater than the target number of selected feature points, it can be determined whether it is necessary to continue to perform the subsequent step of sequentially performing the division step on each node in the node storage space.

[0026] Among them, after obtaining several feature points in the image frame, the method further includes: applying for a feature storage space for the several feature points, and storing the second information of the several feature points in the feature storage space; wherein, the feature information corresponding to the node includes the storage position of the feature points corresponding to the node in the feature storage space.

[0027] Therefore, by applying for a feature storage space, the storage position of the feature points can be determined, which is convenient for subsequent division of the feature points.

[0028] Among them, the feature-related information of the feature points corresponding to the above node includes the start storage position and the end storage position of the feature points corresponding to the node in the feature storage space, and the method further includes: each time a feature point is divided into a node, adjusting the storage position of the second information in the feature storage space so that the second information of the feature points corresponding to the same node is stored in adjacent positions.

[0029] Therefore, by setting the feature-related information of the feature points corresponding to the node to include the start storage position and the end storage position of the feature points corresponding to the node in the feature storage space, the feature-related information of all the feature points corresponding to the node can be stored, without having to record the feature information of each feature point, thereby saving storage space.

[0030] Among them, the above feature storage space is a first array, and each element in the first array is used to store the second information of a feature point; and / or, the second information of the feature point includes at least one of the following: the position information of the feature point in the image frame, the response degree of the feature point, the identifier of the feature map to which the feature point belongs, where the feature map to which the feature point belongs is one of the multiple feature maps with different resolutions corresponding to the image frame.

[0031] Therefore, by determining the specific form of the feature storage space, it will facilitate the subsequent division of feature points. Additionally, by determining the second information of the feature points, these information can be utilized for the subsequent division of feature points.

[0032] Among them, the above node storage space is a second array, and each element in the second array is used to store the first information of a node; or, the node storage space is a singly linked list, and each node in the singly linked list is used to store the first information of a node; and / or, the first information of the node further includes at least one of the following: the position information of the area corresponding to the node in the image frame, whether the node is a preset node, the storage position information of the next node pointed to by the node in the node storage space, and the layer identifier of the node in the tree structure obtained by the tree division method.

[0033] Therefore, by determining the specific storage form of the node storage space, it can be used for the subsequent division of feature points. Additionally, by determining the specific information included in the first information of the node, these information can be used for the subsequent division of feature points.

[0034] Among them, the above tree division method includes at least one of the quadtree division method and the octree division method; and / or, selecting feature points from each of the finally divided nodes to obtain a selection result of a plurality of feature points, including: selecting a second quantity of feature points from each of the finally divided nodes to obtain a selection result of a plurality of feature points.

[0035] Therefore, by determining the specific method of the tree division method, the feature points can be divided by a quadtree or an octree.

[0036] The second aspect of the present application provides an image feature point selection method, including: obtaining a plurality of feature maps of an image frame, where the resolution of each feature map is different; performing the following feature point selection on each feature map in parallel: obtaining a plurality of feature points from the feature map, and obtaining a selection result of the plurality of feature points, where the selection result of the plurality of feature points is selected from the plurality of feature points by using the method described in the first aspect above.

[0037] Thus, by performing the following feature point selection on feature maps with different resolutions in parallel, the device executing the image feature point selection method of the present application can synchronously perform feature point selection on multiple feature maps with different resolutions and obtain the selection result of the feature points, improving the overall execution efficiency of feature point selection for multiple feature maps with different resolutions.

[0038] Among them, in the above solution, the more the total number of feature points included in the feature map, the more processing resources are used for performing feature point selection on the feature map.

[0039] Because the greater the total number of feature points contained in the feature map, the greater the number of feature points to be partitioned and the more computing power required. Therefore, by determining that the greater the total number of feature points contained in the feature map, the more processing resources are used for feature point selection on the feature map, the overall execution efficiency of feature point selection can be improved.

[0040] The third aspect of this application provides an image feature point selection device, including: an acquisition module, a memory application module, a feature point partitioning module, and a feature point selection module; the acquisition module is used to acquire a number of feature points in an image frame; the memory application module is used to pre-configure a node storage space of a fixed size based on the number of feature points; the feature point partitioning module is used to partition the number of feature points into at least one node in a tree partitioning manner and store the first information of the node in the node storage space, where the first information of the node includes the feature-related information of the feature points corresponding to the node; the feature point selection module is used to select feature points from each finally partitioned node to obtain a selection result of a number of feature points.

[0041] The fourth aspect of this application provides an image feature point selection device, including: an acquisition module, a feature point selection module; the acquisition module is used to acquire multiple feature maps of an image frame, where the resolution of each feature map is different; the feature point selection module is used to perform the following feature point selection in parallel on each feature map: obtain a number of feature points from the feature map and use the method described in the first aspect above to obtain a selection result of the number of feature points.

[0042] The fifth aspect of this application provides an electronic device, including a memory and a processor coupled to each other, and the processor is used to execute program instructions stored in the memory to implement the image feature point selection methods described in the first aspect and the second aspect above.

[0043] The sixth aspect of this application provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the image feature point selection methods described in the first aspect and the second aspect above are implemented.

[0044] In the above solution, by applying a node storage space of a fixed size and storing the first information of the node in the node storage space, during the process of partitioning the number of feature points in a tree partitioning manner, there is no need to repeatedly apply for a storage space to store the first information of the node, reducing the operations on the storage space, thereby improving the execution efficiency of feature point selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is the first flowchart of the first embodiment of the image feature point selection method of this application;

[0046] Figure 2This is an embodiment of the method for selecting feature points in the present application, which divides feature points by using a tree-like partitioning method;

[0047] Figure 3 This is the second process schematic diagram of the first embodiment of the method for selecting image feature points in the present application;

[0048] Figure 4 This is the third process schematic diagram of the first embodiment of the method for selecting image feature points in the present application;

[0049] Figure 5 This is a schematic diagram of adjusting the storage position of the second information in the feature storage space in the method for selecting image feature points in the present application;

[0050] Figure 6 This is another schematic diagram of adjusting the storage position of the second information in the feature storage space in the method for selecting image feature points in the present application;

[0051] Figure 7 This is the fourth process schematic diagram of the first embodiment of the method for selecting image feature points in the present application;

[0052] Figure 8 This is the process schematic diagram of the second embodiment of the method for selecting image feature points in the present application;

[0053] Figure 9 This is the framework schematic diagram of an embodiment of the device for selecting image feature points in the present application;

[0054] Figure 10 This is the framework schematic diagram of another embodiment of the device for selecting image feature points in the present application;

[0055] Figure 11 This is the framework schematic diagram of an embodiment of the electronic device in the present application;

[0056] Figure 12 This is the framework schematic diagram of an embodiment of the computer-readable storage medium of the present application. Detailed implementation manners

[0057] The following will describe the solutions of the embodiments of the present application in detail with reference to the accompanying drawings of the specification.

[0058] In the following description, specific details such as specific system structures, interfaces, and technologies are presented for the purpose of illustration rather than limitation, so as to understand the present application thoroughly.

[0059] The terms "system" and "network" are often used interchangeably herein. The term "and / or" herein merely describes an association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects. Furthermore, "plurality" herein means two or more than two.

[0060] The method for selecting image feature points of the present application can be applied to technical fields such as computer vision, robotics, autonomous vehicles, 3D reconstruction, and augmented reality. The device for executing the method for selecting image feature points of the present application can be an electronic device such as a computer, a mobile phone, a tablet computer, and smart glasses.

[0061] Please refer to Figure 1 , Figure 1 which is the first process schematic diagram of the first embodiment of the method for selecting image feature points of the present application. Specifically, it can include the following steps:

[0062] Step S11: Obtain a plurality of feature points in the image frame.

[0063] The image frame can be an image captured by an electronic device such as a mobile phone, a tablet computer, or smart glasses, or an image captured by a surveillance camera, which is not limited herein. Other scenarios can be deduced by analogy and will not be exemplified one by one herein.

[0064] In one embodiment, the image frame can be an image obtained by removing the edge region of the original image. For example, after removing a certain number of pixel points at the edge of the original image, the obtained image is the image frame.

[0065] After obtaining the image frame, feature extraction can be performed on the image frame to obtain a plurality of feature points. In one implementation, the obtained plurality of feature points are stored in the memory and stored in the form of an array. The feature extraction algorithm is, for example, the FAST (features from accelerated segment test) algorithm, the SIFT (Scale-invariant feature transform) algorithm, the ORB (Oriented FAST and Rotated BRIEF) algorithm, etc. In a specific implementation scenario, the feature extraction algorithm is the ORB (Oriented FAST and Rotated BRIEF) algorithm. In addition, after obtaining the feature points, a feature representation corresponding to each feature point can also be obtained, and the feature representation is, for example, a feature vector.

[0066] In a specific embodiment, the device that executes the image feature point selection method of the present application may perform image acquisition and feature extraction to obtain a number of feature points. In another specific implementation, the device that executes the image feature point selection method of the present application may also directly obtain the feature points obtained by other devices through feature extraction of the image frame.

[0067] In an embodiment, after obtaining a number of feature points in the image frame, a feature storage space may be applied for the number of feature points, and the second information of the number of feature points may be stored in the feature storage space. The feature storage space may be the memory space of the memory. Therefore, by applying for the feature storage space, the storage location of the feature points can be determined, which is convenient for subsequent division of the feature points.

[0068] In an embodiment, the feature storage space in the memory may be an array, defined as the first array. The first array contains a certain number of elements. In a specific implementation, each element in the first array is used to store the second information of a feature point. Therefore, by determining the specific form of the feature storage space, it will be convenient for subsequent division of the feature points.

[0069] In an embodiment, the second information of the feature point includes at least one of the following: the position information of the feature point in the image frame, the response degree of the feature point, and the identifier of the feature map to which the feature point belongs. The position information of the feature point in the image frame may be the pixel coordinates of the feature point. The response degree of the feature point may be a specific value determined according to the feature extraction algorithm, such as the response value of the feature point determined when the FAST feature extraction algorithm extracts the feature point. The feature map to which the feature point belongs is one of the multiple feature maps with different resolutions corresponding to the image frame, and the identifier of the feature map to which the feature point belongs may be the number of the feature map to which the feature point belongs among the multiple feature maps with different resolutions. The multiple feature maps with different resolutions may be obtained by performing feature extraction based on the image pyramid of the image frame, or may be obtained by performing different feature extractions on the image frame to obtain multiple feature maps with different resolutions. Therefore, by determining the second information of the feature point, this information can be used for subsequent division of the feature points.

[0070] Step S12: Based on a number of feature points, pre-configure a node storage space with a fixed size.

[0071] After obtaining a number of feature points, these feature points can be partitioned to obtain the selection result of the feature points. Partitioning the feature points can be to partition a number of feature points into at least one node in a tree partitioning manner. Therefore, before performing the step of partitioning the feature points, a node storage space with a fixed size can be preconfigured based on a number of feature points, and the node storage space with the fixed size can be used to store the relevant information of the partitioned feature points. In one embodiment, a fixed-size memory space is allocated in the memory as the node storage space. The tree partitioning manner can be selected as needed. In one embodiment, the node storage space is the second array in the memory.

[0072] Step S13: Partition a number of feature points into at least one node in a tree partitioning manner, and store the first information of the node into the node storage space.

[0073] After determining the node storage space, a number of feature points can be partitioned into at least one node in a tree partitioning manner. The number of nodes can be set as needed and is not limited here. Partitioning a number of feature points into at least one node in a tree partitioning manner means, for an image frame, dividing the image frame into several regions, each region corresponding to a node, and determining the feature points included in each region. The feature points included in each region are the feature points corresponding to the node corresponding to that region. Thus, it is possible to partition a number of feature points into at least one node. In some embodiments, the tree partitioning manner includes at least one of a quadtree partitioning manner and an octree partitioning manner. By determining the specific manner of the tree partitioning manner, the feature points can be partitioned by a quadtree or an octree.

[0074] In one embodiment, if the positions of some feature points are the boundaries of several image regions, the feature points on the boundary can be partitioned into several image regions as needed.

[0075] After obtaining the nodes by partitioning, the first information of the nodes can be stored into the node storage space. The first information of the nodes can include the feature-related information of the feature points corresponding to the nodes. The feature-related information of the feature points corresponding to the nodes can be understood as the information related to the feature points corresponding to the nodes, such as the position information of the feature points in the image, the storage position information of the feature points in the memory, the feature representation information of the feature points (such as feature vectors), etc. In one embodiment, the feature information corresponding to the node includes the storage position of the feature points corresponding to the node in the feature storage space. For example, when the feature storage space is an array, the storage position of the feature points corresponding to the node in the feature storage space is the position subscript of the element where the feature points are stored in the array (feature storage space).

[0076] In one embodiment, the node storage space is the second array in the memory, and each element in the second array is used to store the first information of a node. In another embodiment, the node storage space is a singly linked list, and each node in the singly linked list is used to store the first information of a node. Therefore, by determining the specific storage form of the node storage space, it can be used for subsequent division of feature points.

[0077] In one embodiment, the first information of a node further includes at least one of the following: the position information of the region corresponding to the node in the image frame, whether the node is a preset node, the storage position information of the next node pointed to by the node in the node storage space, and the layer identifier of the node in the tree structure obtained by the tree division method. The position information of the region corresponding to the node in the image frame may be the pixel coordinates of the four vertices of the region corresponding to the node in the image frame. The preset node may be a leaf node. The storage position information of the next node pointed to by the node in the node storage space, and the storage position information of the next node in the node storage space may be the position subscript of the next node in the second array. Therefore, by determining the specific information included in the first information of the node, these information can be used for subsequent division of feature points.

[0078] Refer to Figure 2 , Figure 2 , which is an embodiment of using the tree division method to divide feature points in the method for selecting image feature points of the present application. In this embodiment, the tree division method is a quadtree division method. When dividing the image frame 10, four regions are obtained in the first division, namely region 11, region 12, region 13, and region 14, and the layer identifier of these four regions is 1. At this time, region 101 can be divided to obtain four regions, namely region 111, region 112, region 113, and region 114.

[0079] Step S14: Select feature points from each node obtained by the final division to obtain the selection results of a number of feature points.

[0080] After division, a certain number of nodes can be obtained, and a certain number of feature points will be included in the regions of the image frames corresponding to the nodes. At this time, feature points can be selected from the nodes obtained by the final division to obtain the selection results of a number of feature points, that is, from each node obtained by the final division, a certain number of feature points are selected, and these are used as the selection results of the feature points.

[0081] In one embodiment, it may be to select a second number of feature points from each of the final divided nodes to obtain a selection result of a number of feature points. The second number can be set as required and is not limited here. For example, in each node obtained by the final division, 1 feature point can be selected from each node, and the feature points selected from each node are the selection result of a number of feature points.

[0082] Therefore, by applying for a node storage space of a fixed size and storing the first information of the node in the node storage space, when using the tree division method to divide a number of feature points, there is no need to repeatedly apply for memory space from the memory to store the first information of the node, reducing the operations on the memory, thereby improving the execution efficiency of the feature point division method.

[0083] Please refer to Figure 3 , Figure 3 which is the second process schematic diagram of the first embodiment of the image feature point selection method of the present application. This embodiment further expands on the "pre-configuring a node storage space of a fixed size based on a number of feature points" mentioned in the above steps, and specifically includes the following steps S121 and S122.

[0084] Step S121: Determine the target node division number based on the target number of selected feature points.

[0085] The target number of selected feature points is the number of feature points to be selected from a number of feature points, and the specific number of the target number of selected feature points can be set as needed. The target node division number is the number of nodes finally obtained by the division. In one embodiment, the target node division number may be the same as the target number of selected feature points. For example, if 10 feature points are selected as the target selected feature points from a number of feature points, the target node division number may be 10 nodes.

[0086] Step S122: Apply for a node storage space with a size matching the target node division number for a number of feature points.

[0087] After determining the number of nodes finally obtained by the division, the regions of the corresponding image frames of these nodes will contain a number of feature points extracted from the image frames. Therefore, a node storage space with a size matching the target node division number can be applied for a number of feature points to store the storage of the first information of the node.

[0088] In one embodiment, the size of the first information of the nodes to be stored by a node can be determined first, and then a storage space for nodes with a size matching the number of target node partitions can be allocated in the memory. In another embodiment, a memory space equal to the space of the second information of several feature points in the memory can be allocated in the memory, and then according to the size of the first information of the nodes stored by each node in the memory.

[0089] Therefore, by allocating a storage space for nodes with a size matching the number of target node partitions, the storage space for nodes can store the first information of all the finally partitioned nodes, so that there is no need to allocate a new storage space subsequently, thereby reducing the operations on the storage space and improving the execution efficiency of feature point selection.

[0090] Please refer to Figure 4 , Figure 4 which is the third process schematic diagram of the first embodiment of the image feature point selection method of the present application. This embodiment is a further expansion of the step of "partitioning several feature points into at least one node by using a tree partitioning method and storing the first information of the nodes in the node storage space" mentioned in the above steps, and specifically includes:

[0091] Step S131: Partition some or all of the several feature points into a node, and store the first information of the node in the node storage space.

[0092] When performing the partitioning of specific feature points, first, all the feature points to be partitioned can be included in one node. Specifically, some or all of the several feature points can be partitioned into a node. In one embodiment, when using a tree partitioning method to partition feature points, a node will be determined first, and this node can be regarded as the root node. At this time, it can be considered that all the feature points are included in one node, or some of the feature points are partitioned into the root node.

[0093] In a specific embodiment, the area in the image frame except for the preset boundary area is used as the root area. The preset boundary area can be considered as the edge area of the image frame, such as the area within a certain number of pixel points from the image edge. Then, the feature points in the root area are partitioned into a node. By removing the edge area of the image frame, the feature points on the edge area can be removed, reducing the number of feature points and accelerating the execution speed of the image feature point selection method.

[0094] In one embodiment, when performing the subsequent step of sequentially performing a partitioning step on each node in the node storage space until the node storage space no longer meets the first requirement, it is performed when it is detected that the number of features corresponding to the currently partitioned node is greater than the target number of selected feature points. That is, after partitioning some or all of the feature points of a number of feature points into a node, it is determined whether to perform the subsequent step of sequentially performing a partitioning step on each node in the node storage space by determining whether the number of features corresponding to the currently partitioned node is greater than the target number of selected feature points. It can be understood that if the number of features corresponding to the currently partitioned node is not greater than the target number of selected feature points, it means that all the feature points corresponding to this node can be directly determined as the selected feature points, and thus there is no need to perform the subsequent sequential partitioning steps. Therefore, by determining whether the number of features corresponding to the currently partitioned node is greater than the target number of selected feature points, it can be determined whether to continue performing the subsequent step of sequentially performing a partitioning step on each node in the node storage space.

[0095] Step S132: Sequentially perform a partitioning step on each node in the node storage space until the node storage space no longer meets the first requirement; wherein, the partitioning step includes: determining the node as the parent node, and partitioning the feature points corresponding to the parent node into at least one new node, and storing the first information of the new node in the node storage space.

[0096] After partitioning some or all of the feature points of the feature points into a node, the node can be sequentially partitioned. After partitioning, the first information corresponding to the partitioned node is stored in the node storage space. In this way, the partitioning of the feature points in the feature point selection process can be realized.

[0097] For example, when using the quadtree partitioning method, the root node is first partitioned to obtain four nodes, and then these 4 nodes are respectively used as the parent nodes, and these 4 parent nodes are sequentially partitioned. After partitioning, the newly partitioned nodes are used as the parent nodes, and these parent nodes are sequentially partitioned again.

[0098] Sequential partitioning means partitioning in a certain order.

[0099] In one embodiment, the nodes obtained by dividing a parent node are the child nodes of the parent node. For the child nodes belonging to the same parent node, the execution order of the child nodes of the child nodes can be determined according to the number of feature points corresponding to the child nodes. Subsequently, the division steps can be performed on the child nodes according to the execution order of the child nodes. That is, for the child nodes belonging to the same parent node, the order of performing the division steps depends on the number of feature points corresponding to the child nodes. The node with more feature points will have an earlier execution order of the division steps, that is, the order of execution is determined according to the number of feature points corresponding to the nodes. For example, a node is divided into 4 child nodes, namely child node A, child node B, child node C, and child node D. The magnitude relationship of the number of feature points corresponding to the nodes is: A > B > C > D. Then, the division steps will be performed on child node A first, and then on child node B, and so on. Therefore, by preferentially dividing the child nodes with more corresponding feature points, the nodes with more corresponding feature points can be preferentially divided, which can improve the uniformity of the divided feature points.

[0100] In one embodiment, for the nodes belonging to the same parent node, the order of performing the division steps can also depend on the maximum response degree of the feature points corresponding to the nodes. The node with a larger maximum response degree will be executed first. Taking the above 4 nodes as an example again, the magnitude relationship of the maximum response degrees of the feature points corresponding to the nodes is: A > B > C > D. Then, the division steps will be performed on node A first, and then on node B, and so on. In another embodiment, for the nodes belonging to the same parent node, the order of performing the division steps can also depend on the sum of the response degrees of all the feature points corresponding to the nodes. The node with a larger sum of the response degrees will be executed first. Still taking the above 4 nodes as an example, the magnitude relationship of the sum of the response degrees of the feature points corresponding to the nodes is: A > B > C > D. Then, the division steps will be performed on node A first, and then on node B, and so on.

[0101] In one embodiment, it can be determined that the execution order of the other nodes belonging to the same parent node as the currently executed node is prior to the execution order of the nodes obtained by dividing the currently executed node. For example, 4 nodes have been divided, namely node A, node B, node C, and node D. At this time, if node A is determined as the parent node and the feature points corresponding to the parent node are divided into at least one new node. Then, node B will be divided, and then node C will be divided, and so on. After nodes A, B, C, and D have all been divided, the newly divided nodes (the nodes obtained by dividing nodes A, B, C, and D) will be divided. In this way, by determining that the nodes belonging to the same parent node are preferentially divided, the uniformity of the divided feature points is improved, and thus the rationality of subsequent feature selection is improved.

[0102] It can be understood that when a node is divided, it means that the node no longer stores the storage space for nodes. The current number of stored nodes in the node storage space is the undivided nodes included in the node storage space, because the divided nodes no longer store in the node storage space.

[0103] In one embodiment, the first requirement is that the current number of stored nodes in the node storage space is less than the target node division quantity. At this time, it means that the current number of stored nodes is equal to or greater than the target node quantity, so the division can no longer continue. Therefore, by determining that the first requirement is that the current number of stored nodes in the node storage space is less than the target node division quantity, the step of node division can be stopped after obtaining the required number of nodes.

[0104] In one embodiment, the "determining a node as the parent node" mentioned in this step will only take a node as the parent node when the number of feature points corresponding to the node meets the second requirement. That is, when the number of feature points corresponding to a node meets the second requirement, the node will be taken as the parent node, and the feature points corresponding to the node will be divided into at least one new node. The second requirement is, for example, that when the number of feature points corresponding to a node is greater than the first threshold, the node will be taken as the parent node. For example, the first threshold can be 3, 4, or 5, etc., which can be specifically set according to needs and is not limited here. By setting the first threshold, the number of nodes that need to be divided can be reduced. Therefore, by judging whether the number of feature points corresponding to a node meets the second requirement, the nodes that do not meet the second requirement can be taken as the parent nodes, improving the uniformity of feature point division and also reducing the number of nodes that need to be divided, thereby improving the execution efficiency of image feature point selection.

[0105] In one embodiment, after each division of feature points into nodes, the storage position of the second information in the feature storage space can be adjusted so that the second information of the feature points corresponding to the same node is stored in adjacent positions. In a specific embodiment, after dividing a node into four new nodes, namely node A, node B, node C, and node D, the storage position of the second information in the feature storage space can be adjusted so that the second information of the feature points corresponding to node A is stored in adjacent positions.

[0106] Refer to Figure Figure 5 , Figure 5 is a schematic diagram of adjusting the storage position of the second information in the feature storage space in the method for selecting image feature points of the present application. Please see Figure 5 part a of. Region 10 is a certain region of the image frame corresponding to a node Y. In region 10, there are 9 feature points, namely feature points 1 - 9, that is, node Y corresponds to 9 feature points. The storage positions of the second information of these 9 feature points in the feature storage space 20 (an array in memory) are asFigure 5 As shown in part a). At this time, the parent node of node Y will be determined, and node Y will be divided into 4 new nodes, namely node A, node B, node C, and node D. Correspondingly, region 10 will also be divided into 4 new regions, namely region 11, region 12, region 13, and region 14. At this time, it can be determined that the characteristic points corresponding to node A are characteristic point 1 and characteristic point 5, the characteristic points corresponding to node B are characteristic point 2 and characteristic point 6, the characteristic points corresponding to node C are characteristic point 7 and characteristic point 9, and the characteristic points corresponding to node D are characteristic point 3, characteristic point 4, and characteristic point 8. Based on this, the storage position of the second information in the characteristic storage space can be adjusted to obtain the adjusted characteristic storage space 21. The storage positions of the second information of these 9 characteristic points in the characteristic storage space 21 are as Figure 5 shown in part b). It can be seen that at this time, the second information of the characteristic points corresponding to the same node is stored in adjacent positions.

[0107] In a specific embodiment, two pointers can be set to adjust the storage position of the second information in the characteristic storage space. The two pointers are pointer A and pointer B respectively, and the sorting step of the second information of the characteristic points is executed.

[0108] First, the two pointers respectively point to the starting position of the sorting. The starting position is the second information of the characteristic points corresponding to the divided parent node in the first element of the characteristic storage space.

[0109] Second, after the parent node is divided into different new nodes, determine the image frame region corresponding to each node, and determine whether the characteristic point corresponding to the second information pointed to by pointer A (hereinafter simply referred to as the characteristic point pointed to by the pointer) is within the image frame region corresponding to the new node.

[0110] Third, if it is within the image frame region corresponding to the new node, then exchange the characteristic points pointed to by pointer A and pointer B (if pointer A and pointer B point to the same characteristic point, there is no need to exchange).

[0111] Fourth, if the characteristic point pointed to by pointer A is not within the image frame region corresponding to the new node, then pointer A points to the next characteristic point.

[0112] Fifth, repeat the above steps until pointer A exceeds the end position. The end position is the second information of the characteristic points corresponding to the divided parent node in the last element of the characteristic storage space.

[0113] In this way, the second information of the characteristic points corresponding to the same node can be stored in adjacent positions.

[0114] At this time, if it is necessary to continue to adjust the second information of the feature points corresponding to another node and store it in adjacent positions, the starting position will be adjusted, specifically, it is the first element after excluding the element where the second information of the feature points that have been adjusted to be stored in adjacent positions is located.

[0115] Refer to Figure Figure 6 , Figure 6 which is another schematic diagram of adjusting the storage position of the second information in the feature storage space in the method for selecting image feature points of the present application. In combination with referring to Figure 5 and Figure 6 , in Figure 5 , node Y is determined as the parent node for partitioning, obtaining node A, node B, node C, and node D. When adjusting the storage position of the second information of the feature points, the storage position of the second information of the feature points corresponding to node A can be adjusted first. As shown in part a of Figure 6 , the starting position at this time is the position of the second information of the feature point corresponding to feature point 1 in the feature storage space, the ending position is the position of the second information of the feature point corresponding to feature point 9 in the feature storage space, and pointer A and pointer B point to feature point 1. Execute the above sorting steps, and when the feature point pointed to by pointer A exceeds feature point 9, stop sorting. At this time, the positions of the elements in the feature storage space are as shown in part b of Figure 6 . When continuing to adjust the storage position of the second information of the feature points corresponding to node B, the starting position at this time is the position of the second information of the feature point corresponding to feature point 3 in part b of Figure 6 in the feature storage space, and the ending position is still the position of the second information of the feature point corresponding to feature point 9 in the feature storage space.

[0116] By adjusting the storage position of the second information in the feature storage space, the second information of the feature points corresponding to the same node can be stored in adjacent positions. Because the first information of the nodes stored in the node storage space will include the feature-related information of the feature points corresponding to the nodes. At this time, the feature-related information of the feature points corresponding to the nodes can include the starting storage position and the ending storage position of the feature points corresponding to the nodes in the feature storage space. For example, in Figure 5 , the feature-related information of the feature points corresponding to node D can include the starting storage position of feature point 3 in the feature storage space, and the ending storage position of feature point 8 in the feature storage space. In this way, by setting the feature-related information of the feature points corresponding to the nodes to include the starting storage position and the ending storage position of the feature points corresponding to the nodes in the feature storage space, the feature-related information of all the feature points corresponding to the node can be stored, without having to record the feature information of each feature point, thereby saving storage space.

[0117] Please refer to Figure 7 , Figure 7It is the fourth process schematic diagram of the first embodiment of the method for selecting image feature points in this application. This embodiment is a further expansion of "dividing the feature points corresponding to the parent node into at least one new node" mentioned in the above steps, specifically including:

[0118] Step S1321: Divide the parent region corresponding to the parent node into a first number of sub-regions.

[0119] After determining the parent node, the parent region corresponding to the parent node can be divided into a first number of sub-regions. The first number is, for example, 4 or 8. When using the tree-shaped division method to divide feature points, the image frame will also be divided into several regions, so as to determine the node corresponding to a certain region and the feature points corresponding to the node. Therefore, the parent region can be considered as the region of the feature points corresponding to the parent node in the image frame. For example, in Figure 5 part b of, if node D is determined as the parent node, then region 14 in the image frame where feature points 3, 4, and 8 corresponding to node D are located is the parent region. By limiting the number of sub-regions divided each time to 4 or 8, it is equivalent to implementing the tree-shaped division in the form of a quadtree or octree.

[0120] Step S1322: Determine whether the number of feature points in the sub-region meets the third requirement.

[0121] When dividing the parent region into a first number of sub-regions, it can be determined whether the number of feature points contained in each sub-region meets the third requirement, so as to determine whether to generate a sub-node for this region. In one embodiment, the third requirement is that the number of feature points in the sub-region is greater than the second threshold. In a specific embodiment, the second threshold can be 0. Therefore, by determining whether the number of feature points in the sub-region is greater than the second threshold, regions with fewer feature points than the second threshold can be excluded.

[0122] Step S1323: When the number of feature points in the sub-region meets the third requirement, generate a corresponding sub-node for the sub-region.

[0123] If the number of feature points in the sub-region meets the third requirement, it means that a corresponding sub-node can be generated for this sub-region. At this time, the feature points corresponding to the sub-node are the feature points in this sub-region.

[0124] Step S1324: When the number of feature points in the sub-region does not meet the third requirement, do not generate a corresponding sub-node for the sub-region.

[0125] Therefore, by dividing the parent region corresponding to the parent node into a first number of sub-regions and determining whether the number of feature points in the sub-region meets the third requirement, sub-regions that do not meet the requirements can be excluded so that corresponding sub-nodes do not need to be generated, and it is possible to flexibly determine whether to form a node according to the feature quantity of the sub-region.

[0126] In an exemplary embodiment, "storing the first information of the new nodes in the node storage space" mentioned in step S132 above may specifically be: storing the first information of one of the new nodes by overwriting the first information of the parent node, and storing the first information of the remaining new nodes in the remaining storage space in the node storage space.

[0127] The remaining storage space can be understood as the storage space in the node storage space that has not been used to store the first information of the nodes. It can be understood that since the parent node has been divided, it can be considered that the node no longer exists. Therefore, the first information of one of the new nodes obtained by division can be used to overwrite the first information of the parent node for storage, and then the first information of the remaining new nodes can be stored in the remaining storage space in the node storage space. Therefore, by using the first information of one new node to overwrite the first information of the parent node for storage, the reuse of the storage space can be achieved, and the storage space can be saved.

[0128] In an exemplary embodiment, after "dividing the feature points corresponding to the parent node into at least one new node" mentioned in step S132 above, the method for selecting image feature points of the present application may further include the steps of: performing a link-type pointing between the nodes belonging to the same parent node, and making the node at the end of the link point to the node pointed to by the parent node, and making the node pointing to the parent node update to point to the node at the head of the link; wherein, the pointing of the nodes is used to determine the execution order of the nodes.

[0129] In an embodiment, a link-type pointing between nodes can be achieved by storing, in the first information of each node, the storage location information of the next node to be pointed to in the node storage space.

[0130] In an example, node A points to node B, and node C points to node A. After node A is divided into nodes A1, A2, A3, and A4. Performing a link-type pointing between the nodes belonging to the same parent node, and making the node at the end of the link point to the node pointed to by the parent node, and making the node pointing to the parent node update to point to the node at the head of the link, may specifically be that node C points to node A1, node A1 points to node A2, node A2 points to node A3, node A3 points to node A4, and node A4 points to node B.

[0131] Therefore, the order of subsequent node division can be achieved through the link pointing method.

[0132] In the above solution, by applying for a node storage space of a fixed size and storing the first information of the node in the node storage space, when dividing a number of feature points using a tree-based division method, it is not necessary to repeatedly apply for memory space in the memory to store the first information of the node, reducing the operations on the memory, thereby improving the execution efficiency of the feature point division method.

[0133] Please refer to Figure 8 , Figure 8 which is a schematic flowchart of the second embodiment of the method for selecting image feature points of the present application. This embodiment specifically includes the following steps:

[0134] Step S21: Obtain multiple feature maps of an image frame, where each feature map has a different resolution.

[0135] The multiple feature maps with different resolutions can be obtained by performing feature extraction based on the image pyramid of the image frame, or by performing different feature extractions on one image frame to obtain multiple feature maps with different resolutions.

[0136] Step S22: Parallelly perform the following feature point selection on each feature map: Obtain several feature points from the feature map and obtain the selection results of the several feature points.

[0137] In this embodiment, obtaining the selection results of several feature points for each feature map is obtained by performing feature point selection according to the technical solution described in the first embodiment of the above method for selecting image feature points.

[0138] Thus, by parallelly performing the following feature point selection on feature maps with different resolutions, the device for executing the method for selecting image feature points of the present application can synchronously perform feature point selection on multiple feature maps with different resolutions and obtain the selection results of the feature points, improving the overall execution efficiency of performing feature point selection on multiple feature maps with different resolutions.

[0139] In one embodiment, when parallelly executing the method for selecting image feature points on multiple feature maps with different resolutions, the larger the total number of feature points included in the feature map, the more processing resources are used for performing feature point selection on the feature map. It can be understood that the larger the total number of feature points included in the feature map, the larger the number of feature points that need to be divided, and the more computing power is required. Therefore, by determining that the larger the total number of feature points included in the feature map, the more processing resources are used for performing feature point selection on the feature map, the overall execution efficiency of feature point selection is improved.

[0140] In the above solution, by executing the method for selecting image feature points on multiple feature maps with different resolutions, the overall execution efficiency is improved.

[0141] Please refer to Figure 9 , Figure 9It is a schematic framework diagram of an embodiment of the image feature point selection device of the present application. The image feature point selection device 90 includes an acquisition module 91, a memory application module 92, a feature point division module 93, and a feature point selection module 94. The acquisition module 91 is used to acquire a number of feature points in the image frame; the memory application module 92 is used to pre-configure a node storage space with a fixed size based on the number of feature points; the feature point division module 93 is used to divide the number of feature points into at least one node in a tree division manner, and store the first information of the node in the node storage space, where the first information of the node includes the feature-related information of the feature points corresponding to the node; the feature point selection module 94 is used to select feature points from each of the finally divided nodes to obtain the selection results of the number of feature points.

[0142] Among them, the above-mentioned memory application module 92 is used to pre-configure a node storage space with a fixed size based on the number of feature points, including: determining the target node division number based on the target selected number of feature points, where the target selected number of feature points is the number of feature points to be selected from the number of feature points, and the target node division number is the number of nodes finally divided; applying a node storage space with a size matching the target node division number for the number of feature points.

[0143] Among them, the above-mentioned feature point division module 93 is used to divide the number of feature points into at least one node in a tree division manner, and store the first information of the node in the node storage space, specifically including: dividing some or all of the number of feature points into a node, and storing the first information of the node in the node storage space; sequentially performing a division step on each node in the node storage space until the node storage space no longer meets the first requirement; where the division step includes: determining the node as the parent node, and dividing the feature points corresponding to the parent node into at least one new node, and storing the first information of the new node in the node storage space.

[0144] Among them, in the above solution, the execution order of other nodes belonging to the same parent node as the currently executed node is prior to the execution order of the nodes obtained by dividing the currently executed node; and / or, the above-mentioned sequentially performing a division step on each node in the node storage space includes: for the child nodes belonging to the same parent node, determining the child node execution order of the child nodes according to the number of feature points corresponding to the child nodes, and performing a division step on the child nodes according to the child node execution order, where the child node is the node obtained by dividing the parent node; and / or, after the feature point division module 93 divides the feature points corresponding to the parent node into at least one new node, the feature point division module 93 can also be used to perform a link type pointing between the nodes belonging to the same parent node, and make the node located at the end of the link point to the node pointed to by the parent node, and make the node pointing to the parent node update to point to the node located at the head of the link; where the pointing of the node is used to determine the execution order of the node.

[0145] Among them, the above-mentioned feature point division module 93 is used to store the first information of the new node in the node storage space, specifically including: storing the first information of one of the new nodes by overwriting the first information of the parent node, and storing the first information of the remaining new nodes in the remaining storage space in the node storage space.

[0146] Among them, the above-mentioned feature point division module 93 is used to determine a node as a parent node, specifically including: when the number of feature points corresponding to the node meets the second requirement, taking the node as the parent node; and / or, the feature point division module 93 is used to divide the feature points corresponding to the parent node into at least one new node, including: dividing the parent region corresponding to the parent node into a first number of sub-regions, where the parent region is the region of the feature points corresponding to the parent node in the image frame; when the number of feature points in the sub-region meets the third requirement, generating a corresponding sub-node for the sub-region, where the feature points corresponding to the sub-node are the feature points in the sub-region; when the number of feature points in the sub-region does not meet the third requirement, not generating a corresponding sub-node for the sub-region.

[0147] Among them, the first requirement in the above solution is that the current number of stored nodes in the node storage space is less than the target node division quantity; the second requirement is that the number of feature points corresponding to the node is greater than the first threshold; the third requirement is that the number of feature points in the sub-region is greater than the second threshold; the first number is 4 or 8.

[0148] Among them, the above-mentioned feature point division module 93 is used to divide some or all of the feature points of several feature points into a node, including: taking the region in the image frame except for the preset boundary region as the root region; dividing the feature points in the root region into a node; and / or, sequentially performing a division step on each node in the node storage space until the node storage space no longer meets the first requirement, which is executed when it is detected that the number of features corresponding to the currently divided node is greater than the target selection number of features.

[0149] Among them, the device 90 further includes a feature point space application module. After the acquisition module 91 is used to acquire several feature points in the image frame, the feature point space application module is used to apply for a feature storage space for the several feature points and store the second information of the several feature points in the feature storage space; among them, the feature information corresponding to the node includes the storage position of the feature points corresponding to the node in the feature storage space.

[0150] Among them, in the above solution, the feature-related information of the feature points corresponding to the nodes includes the starting storage position and the ending storage position of the feature points corresponding to the nodes in the feature storage space. After the feature point division module 93 divides the feature points into nodes each time, the feature point division module 93 is further configured to adjust the storage position of the second information in the feature storage space so that the second information of the feature points corresponding to the same node is stored in adjacent positions.

[0151] Among them, in the above solution, the feature storage space is a first array, and each element in the first array is used to store the second information of a feature point; and / or, the second information of the feature point includes at least one of the following: the position information of the feature point in the image frame, the response degree of the feature point, and the identifier of the feature map to which the feature point belongs, where the feature map to which the feature point belongs is one of the multiple feature maps with different resolutions corresponding to the image frame.

[0152] Among them, in the above solution, the node storage space is a second array, and each element in the second array is used to store the first information of a node; or, the node storage space is a singly linked list, and each node in the singly linked list is used to store the first information of a node; and / or, the first information of the node further includes at least one of the following: the position information of the area corresponding to the node in the image frame, whether the node is a preset node, the storage position information of the next node pointed to by the node in the node storage space, and the layer identifier of the node in the tree structure obtained by the tree division method.

[0153] Among them, in the above solution, the tree division method includes at least one of a quadtree division method and an octree division method; and / or, the feature point selection module 94 is configured to select feature points from each of the finally divided nodes to obtain a selection result of several feature points, specifically including: selecting a second number of feature points from each of the finally divided nodes to obtain a selection result of several feature points.

[0154] Please refer to Figure 10 , Figure 10 is a schematic framework diagram of another embodiment of the image feature point selection device of the present application. The image feature point selection device 10 includes an acquisition module 100 and a feature point selection module 110. The acquisition module 100 is configured to acquire multiple feature maps of an image frame, where the resolution of each feature map is different; the feature point selection module 110 is configured to perform the following feature point selection on each feature map in parallel: acquire several feature points from the feature map and the selection result of the several feature points, where the selection result of the several feature points is obtained by using the method embodiment of the above image feature point selection method.

[0155] Among them, the more the total number of feature points included in the feature map in the above solution, the more processing resources are used to perform feature point selection on the feature map.

[0156] Please refer to Figure 11 , Figure 11 which is a schematic diagram of the framework of an embodiment of the electronic device of the present application. The electronic device 11 includes a memory 111 and a processor 112 that are coupled to each other. The processor 112 is configured to execute program instructions stored in the memory 111 to implement the steps of any of the above-described embodiments of the method for selecting image feature points. In a specific implementation scenario, the electronic device 11 may include, but is not limited to: a microcomputer, a server. In addition, the electronic device 11 may also include mobile devices such as a laptop computer, a tablet computer, etc., which are not limited herein.

[0157] Specifically, the processor 112 is configured to control itself and the memory 111 to implement the steps of any of the above-described embodiments of the method for selecting image feature points. The processor 112 may also be referred to as a CPU (Central Processing Unit). The processor 112 may be an integrated circuit chip having the ability to process signals. The processor 112 may also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Additionally, the processor 112 may be implemented jointly by integrated circuit chips.

[0158] Please refer to Figure 12 , Figure 12 which is a schematic diagram of the framework of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 120 stores program instructions 121 that can be run by a processor. The program instructions 121 are used to implement the steps of any of the above-described embodiments of the method for selecting image feature points.

[0159] The above solution can improve the execution efficiency of the feature point division method.

[0160] In some embodiments, the functions or modules included in the apparatus provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0161] The descriptions of the above embodiments tend to emphasize the differences between the embodiments. The same or similar parts can be referred to each other. For the sake of brevity, they will not be repeated herein.

[0162] In several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical or other forms.

[0163] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0164] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0165] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs and other various media that can store program codes.

Claims

1. An image feature point selection method, characterized in that, Including: Obtaining a plurality of feature points in an image frame; Applying node storage space with a size matching the number of target node partitions for the plurality of feature points, where the number of target node partitions is the number of nodes finally partitioned; At any moment, the number of nodes stored in the node storage space is less than or equal to the number of target node partitions; Using a tree partitioning method to partition the plurality of feature points into at least one node, and storing first information of the node in the node storage space, where the first information of the node includes feature-related information of the feature points corresponding to the node; Selecting the feature points from each of the finally partitioned nodes to obtain a selection result of the plurality of feature points.

2. The method according to claim 1, wherein Before applying the node storage space with a size matching the number of target node partitions for the plurality of feature points, the method further includes: Determining the number of target node partitions based on the number of target selected feature points, where the number of target selected feature points is the number of feature points to be selected from the plurality of feature points.

3. The method according to claim 1 or 2, characterized in that, The using a tree partitioning method to partition the plurality of feature points into at least one node and storing the first information of the node in the node storage space includes: Partitioning some or all of the plurality of feature points into a node, and storing the first information of the node in the node storage space; Sequentially performing a partitioning step on each node in the node storage space until the node storage space no longer meets the first requirement; where the partitioning step includes: determining the node as a parent node, and partitioning the feature points corresponding to the parent node into at least one new node, and storing the first information of the new node in the node storage space.

4. The method according to claim 3, characterized in that The execution order of other nodes belonging to the same parent node as the currently executed node is prior to the execution order of the nodes partitioned from the currently executed node; And / or, the sequentially performing the partitioning step on each node in the node storage space includes: for the child nodes belonging to the same parent node, determining the child node execution order according to the number of feature points corresponding to the child nodes, and performing the partitioning step on the child nodes according to the child node execution order, where the child nodes are the nodes partitioned from the parent node; And / or, after partitioning the feature points corresponding to the parent node into at least one new node, the method further includes: Performing a link-type pointing between the nodes belonging to the same parent node, and making the node at the end of the link point to the node pointed to by the parent node, and making the node pointing to the parent node update to point to the node at the head of the link; where the pointing of the node is used to determine the execution order of the node.

5. The method according to claim 3 or 4, characterized in that, The storing the first information of the new node in the node storage space includes: Storing the first information of one of the new nodes by overwriting the first information of the parent node, and storing the first information of the remaining new nodes in the remaining storage space in the node storage space.

6. The method according to any one of claims 3 to 5, characterized in that, The determining the node as a parent node includes: When the number of feature points corresponding to the node meets the second requirement, the node is used as the parent node; and / or, the dividing the feature points corresponding to the parent node into at least one new node includes: dividing the parent region corresponding to the parent node into a first number of sub-regions, where the parent region is the region of the feature points corresponding to the parent node in the image frame; when the number of feature points in the sub-region meets the third requirement, generating a corresponding sub-node for the sub-region, where the feature points corresponding to the sub-node are the feature points in the sub-region; when the number of feature points in the sub-region does not meet the third requirement, not generating a corresponding sub-node for the sub-region.

7. The method according to claim 6, characterized in that The first requirement is that the current number of stored nodes in the node storage space is less than the target node division number; The second requirement is that the number of feature points corresponding to the node is greater than the first threshold; The third requirement is that the number of feature points in the sub-region is greater than the second threshold.

8. The method according to any one of claims 3 to 7, characterized in that The dividing some or all of the several feature points into a node includes: taking the region in the image frame except the preset boundary region as the root region; dividing the feature points in the root region into a node; and / or, the step of sequentially performing the division step on each node in the node storage space until the node storage space no longer meets the first requirement is executed when it is detected that the number of features corresponding to the currently divided node is greater than the target selected number of features.

9. The method according to any one of claims 1 to 8, characterized in that After obtaining the several feature points in the image frame, the method further includes: applying for a feature storage space for the several feature points and storing the second information of the several feature points in the feature storage space; where the feature information corresponding to the node includes the storage position of the feature points corresponding to the node in the feature storage space.

10. The method according to claim 9, wherein The feature-related information of the feature points corresponding to the node includes the starting storage position and the ending storage position of the feature points corresponding to the node in the feature storage space, and the method further includes: each time the feature points are divided into the node, adjusting the storage position of the second information in the feature storage space so that the second information of the feature points corresponding to the same node is stored in adjacent positions.

11. The method according to claim 9 or 10, characterized in that The feature storage space is a first array, and each element in the first array is used to store the second information of a feature point; and / or, the second information of the feature point includes at least one of the following: the position information of the feature point in the image frame, the response degree of the feature point, the identifier of the feature map to which the feature point belongs, where the feature map to which the feature point belongs is one of the multiple feature maps with different resolutions corresponding to the image frame.

12. The method according to any one of claims 1 to 11, characterized in that, The node storage space is a second array, and each element in the second array is used to store the first information of a node; or, the node storage space is a singly linked list, and each node in the singly linked list is used to store the first information of a node; And / or, the first information of the node further includes at least one of the following: the position information of the area corresponding to the node in the image frame, whether the node is a preset node, the storage position information of the next node pointed to by the node in the node storage space, and the layer identifier of the node in the tree structure obtained by the tree division method.

13. The method according to any one of claims 1 to 12, characterized in that, The tree division method includes at least one of a quadtree division method and an octree division method; And / or, the selecting the feature points from each node obtained by the final division to obtain the selection result of the several feature points includes: Selecting a second number of the feature points from each of the nodes obtained by the final division respectively to obtain the selection result of the several feature points.

14. A method for selecting image feature points, characterized in that, Includes: Obtaining a plurality of feature maps of an image frame, wherein the resolution of each feature map is different; Performing the following feature point selection on each feature map in parallel: obtaining several feature points from the feature map, and obtaining the selection result of the several feature points by the method according to any one of claims 1-13.

15. The method according to claim 14, characterized in that, The more the total number of feature points included in the feature map, the more processing resources are used to perform the feature point selection on the feature map.

16. An image feature point selection device, characterized in that, Includes: An obtaining module, configured to obtain several feature points in an image frame; A memory application module, configured to apply for a node storage space with a size matching the number of target node divisions for the several feature points, where the number of target node divisions is the number of nodes obtained by the final division; The number of nodes stored in the node storage space at any time is less than or equal to the number of target node divisions; A feature point division module, configured to divide the several feature points into at least one node by using a tree division method, and store the first information of the node into the node storage space, where the first information of the node includes the feature related information of the feature points corresponding to the node; A feature point selection module, configured to select the feature points from each node obtained by the final division to obtain the selection result of the several feature points.

17. An image feature point selection device, characterized in that, Includes: An obtaining module, configured to obtain a plurality of feature maps of an image frame, wherein the resolution of each feature map is different; A feature point selection module, configured to perform the following feature point selection on each feature map in parallel: obtaining several feature points from the feature map, and obtaining the selection result of the several feature points by the method according to any one of claims 1-13.

18. An image feature point selection device, characterized in that, Includes a processor and a memory coupled to each other, wherein, The processor is configured to execute the computer program stored in the memory to execute the method according to any one of claims 1 to 13, or the method according to any one of claims 14-15.

19. A computer-readable storage medium, characterized in that, Stores a computer program that can be run by a processor, and the computer program is used to implement the method according to any one of claims 1 to 13, or the method according to any one of claims 14-15.

Images