Feature extraction method and device and computer equipment

By dividing the feature value image into multiple image blocks and extracting the target feature value, the existing feature extraction algorithm has solved the problem of large amount and slow calculation speed, and an efficient feature extraction process is achieved.

CN119006850APending Publication Date: 2024-11-22ZHUHAI MOJIE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310556324.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-16
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing feature extraction algorithm has a large amount of calculation in image expansion processing and feature value sorting processing, resulting in slow feature extraction speed and poor real-time performance.

Method used

By dividing the feature value image of the image to be extracted into multiple image blocks, and generating a new feature value image based on the feature value that meets the preset conditions in each image block, and further dividing it into multiple image blocks, the extraction satisfies The characteristic value of the preset condition is used as the target characteristic value, thereby extracting the corresponding characteristic points from the image to be extracted, avoiding image expansion processing.

Benefits of technology

The calculation amount of feature extraction is significantly reduced, the speed and real-time nature of feature extraction are improved, and the calculation amount during feature extraction is reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006850A_ABST
    Figure CN119006850A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a feature extraction method and device and computer equipment, and belongs to the field of computer vision. The method comprises the steps of generating a first feature value image corresponding to a to-be-extracted image; dividing the first feature value image into a plurality of first image blocks, and generating a second feature value image according to a feature value meeting a preset condition in each first image block; dividing the second feature value image into a plurality of second image blocks, and extracting a feature value meeting a preset condition in each second image block as a target feature value; and obtaining a first pixel coordinate of each target feature value in the to-be-extracted image, and extracting a corresponding feature point from the to-be-extracted image according to the first pixel coordinate. According to the method, the calculation amount of feature extraction is reduced, and the speed and the real-time performance of feature extraction are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a feature extraction method, device and computer equipment. Background Art

[0002] Feature point extraction is a basic part in the field of computer vision, which can provide data support for specific applications in the field of computer vision such as image matching, object recognition, target tracking and 3D reconstruction. The current feature extraction algorithms mainly include SIFT (Scale-invariant feature transform) feature extraction algorithm, SURF (Speeded Up Robust Features) algorithm and Harris algorithm. However, the above feature extraction algorithms involve image expansion processing and eigenvalue sorting processing. Image expansion processing increases the amount of calculation, and eigenvalue sorting processing is very time-consuming, which makes the feature extraction calculation amount high, the calculation speed is slow, and the real-time performance is not good. Summary of the invention

[0003] The embodiments of the present invention provide a feature extraction method, an apparatus and a computer device, aiming to reduce the computational complexity of feature extraction so as to improve the speed and real-time performance of feature extraction.

[0004] In a first aspect, an embodiment of the present invention provides a feature extraction method, comprising:

[0005] Generate a first eigenvalue image corresponding to the image to be extracted;

[0006] Dividing the first eigenvalue image into a plurality of first image blocks, and generating a second eigenvalue image according to an eigenvalue satisfying a preset condition in each of the first image blocks;

[0007] Dividing the second eigenvalue image into a plurality of second image blocks, and extracting an eigenvalue satisfying the preset condition in each of the second image blocks as a target eigenvalue;

[0008] The first pixel coordinates of each target feature value in the image to be extracted are obtained, and the corresponding feature points are extracted from the image to be extracted according to the first pixel coordinates.

[0009] In a second aspect, an embodiment of the present invention further provides a feature extraction device, the feature extraction device comprising:

[0010] An image processing module, configured to generate a first eigenvalue image corresponding to the image to be extracted;

[0011] The image processing module is further configured to divide the first eigenvalue image into a plurality of first image blocks, and generate a second eigenvalue image according to an eigenvalue satisfying a preset condition in each of the first image blocks;

[0012] The image processing module is further configured to divide the second eigenvalue image into a plurality of second image blocks, and extract the eigenvalue satisfying the preset condition in each of the second image blocks as the target eigenvalue;

[0013] The feature extraction module is configured to obtain the first pixel coordinates of each target feature value in the image to be extracted, and extract corresponding feature points from the image to be extracted according to the first pixel coordinates.

[0014] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the feature extraction method described in the first aspect is implemented.

[0015] The embodiment of the present invention provides a feature extraction method, device and computer equipment. The embodiment of the present invention divides the feature value image corresponding to the image to be extracted into multiple image blocks, and generates a new feature value image according to the feature value that meets the preset conditions in each image block, and then divides the new feature value image into multiple new image blocks, and extracts the feature values ​​that meet the preset conditions in the new image blocks as target feature values, so that multiple target feature values ​​can be quickly selected, and then feature points corresponding to the multiple target feature values ​​can be extracted from the image to be extracted. The entire feature extraction process does not involve image expansion processing, which greatly reduces the calculation amount of feature extraction and improves the speed and real-time performance of feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 is a schematic block diagram of the structure of a computer device provided by an embodiment of the present invention;

[0018] Figure 2 It is a flowchart of a feature extraction method provided by an embodiment of the present invention;

[0019] Figure 3is a schematic diagram of generating a first eigenvalue image in an embodiment of the present invention;

[0020] Figure 4 for Figure 2 A schematic diagram of the sub-steps of the feature extraction method in FIG.

[0021] Figure 5 is a schematic diagram of a second eigenvalue image and a second image block in an embodiment of the present invention;

[0022] Figure 6 It is a schematic block diagram of the structure of a feature extraction device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0025] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0026] Feature point extraction is a basic part in the field of computer vision, which can provide data support for specific applications in the field of computer vision such as image matching, object recognition, target tracking and 3D reconstruction. The current feature extraction algorithms mainly include SIFT (Scale-invariant feature transform) feature extraction algorithm, SURF (Speeded Up Robust Features) algorithm and Harris algorithm. However, the above feature extraction algorithms involve image expansion processing and eigenvalue sorting processing. Image expansion processing increases the amount of calculation, and eigenvalue sorting processing is very time-consuming, which makes the feature extraction calculation amount high, the calculation speed is slow, and the real-time performance is not good.

[0027] To solve the above problems, an embodiment of the present invention provides a feature extraction method, device and computer equipment. The feature extraction method divides the feature value image corresponding to the image to be extracted into multiple image blocks, and generates a new feature value image according to the feature value that meets the preset conditions in each image block, and then divides the new feature value image into multiple new image blocks, and extracts the feature value that meets the preset conditions in the new image blocks as the target feature value, so that multiple target feature values ​​can be quickly selected, and then the feature points corresponding to the multiple target feature values ​​can be extracted from the image to be extracted. The entire feature extraction process does not involve image expansion processing, which greatly reduces the computational complexity of feature extraction and improves the speed and real-time performance of feature extraction.

[0028] In an embodiment of the present invention, the computer device may be a server or an electronic device. Among them, the server may be an independent server, or it may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The electronic device includes but is not limited to cameras, smart phones, tablet computers, laptops, desktop computers, personal digital assistants, wearable devices, and self-mobile devices. Wearable devices may be smart watches, smart bracelets, augmented reality (AR) devices, or virtual reality (VR) devices. AR devices may be AR glasses, and VR devices may be VR glasses. Self-mobile devices may be sweepers, food delivery machines, drones, or logistics vehicles.

[0029] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0030] Figure 1 It is a schematic block diagram of the structure of a computer device provided by an embodiment of the present invention.

[0031] like Figure 1 As shown, the computer device 100 includes a processor 101 and a memory 102 , and the processor 101 and the memory 102 are connected via a bus 103 , such as an I2C (Inter-integrated Circuit) bus.

[0032] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire computer device 100. The processor 101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0033] Specifically, the memory 102 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0034] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the embodiment of the present invention, and does not constitute a limitation on the computer device to which the embodiment of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0035] The processor 101 is used to run the computer program stored in the memory 102, and implement any one of the optical flow calculation methods provided by the embodiments of the present invention when executing the computer program.

[0036] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and implement the following steps when executing the computer program:

[0037] Generate a first eigenvalue image corresponding to the image to be extracted;

[0038] Dividing the first eigenvalue image into a plurality of first image blocks, and generating a second eigenvalue image according to an eigenvalue satisfying a preset condition in each of the first image blocks;

[0039] Dividing the second eigenvalue image into a plurality of second image blocks, and extracting an eigenvalue satisfying the preset condition in each of the second image blocks as a target eigenvalue;

[0040] The first pixel coordinates of each target feature value in the image to be extracted are obtained, and the corresponding feature points are extracted from the image to be extracted according to the first pixel coordinates.

[0041] In some embodiments, when the processor 101 generates the first eigenvalue image corresponding to the image to be extracted, it is used to implement:

[0042] According to 1 / 2 N Scaling the image to be extracted to obtain an N-th layer image corresponding to the image to be extracted, where N is 1 or 2;

[0043] Generate an eigenvalue image corresponding to the N-th layer image, and determine the eigenvalue image as the first eigenvalue image.

[0044] In some embodiments, when obtaining the first pixel coordinate of each target feature value in the image to be extracted, the processor 101 is used to implement:

[0045] When N is 1, obtaining the second pixel coordinate of each target feature value in the first layer image;

[0046] Each of the second pixel coordinates is multiplied by 2 to be enlarged, so as to obtain the first pixel coordinate of each of the target feature values ​​in the image to be extracted.

[0047] In some embodiments, when extracting corresponding feature points from the image to be extracted according to the first pixel coordinates, the processor 101 is used to implement:

[0048] Determine a first neighborhood of a pixel point corresponding to the first pixel coordinate in the image to be extracted;

[0049] Determine a characteristic value of each pixel in the first neighborhood;

[0050] Pixel points corresponding to feature values ​​that meet the preset condition are extracted from the first neighborhood as feature points.

[0051] In some embodiments, when obtaining the first pixel coordinate of each target feature value in the image to be extracted, the processor 101 is used to implement:

[0052] When N is 2, obtaining the third pixel coordinate of each target feature value in the second layer image corresponding to the image to be extracted;

[0053] Multiply each of the third pixel coordinates by 2 to enlarge the coordinates, and obtain a second pixel coordinate of each of the target feature values ​​in the first layer image;

[0054] Determine in the first layer image a second neighborhood of a pixel point corresponding to each of the second pixel coordinates, and determine a feature value of each pixel point in each of the second neighborhoods;

[0055] The pixel coordinates of the pixel points corresponding to each feature value satisfying the preset condition in the second neighborhood are multiplied by 2 to obtain the first pixel coordinates of each target feature value in the image to be extracted.

[0056] In some embodiments, when implementing dividing the second eigenvalue image into a plurality of second image blocks, the processor 101 is configured to implement:

[0057] Perform non-maximum suppression processing on the second eigenvalue image to obtain a third eigenvalue image, and divide the third eigenvalue image into a plurality of second image blocks.

[0058] In some embodiments, before implementing dividing the first eigenvalue image into a plurality of first image blocks and generating a second eigenvalue image according to the eigenvalue satisfying the preset condition in each of the first image blocks, the processor 101 is further configured to implement:

[0059] Get the number of feature points to be extracted;

[0060] Determine a first size of the first image block and a second size of the second image block according to the number to be extracted, wherein the first size is larger than the second size;

[0061] The step of dividing the first eigenvalue image into a plurality of first image blocks comprises:

[0062] Dividing the first eigenvalue image into a plurality of first image blocks having a size of the first size;

[0063] The step of dividing the second eigenvalue image into a plurality of second image blocks comprises:

[0064] The second eigenvalue image is divided into a plurality of second image blocks having a size of the second size.

[0065] In some embodiments, after extracting corresponding feature points from the image to be extracted according to the first pixel coordinates, the processor 101 is further configured to implement:

[0066] Using a preset optical flow tracking algorithm, multiple tracking points are determined in the image to be tracked based on the extracted multiple feature points;

[0067] When the number of the tracking points is less than the number of the feature points, determining a target image block in the image to be tracked where the tracking points do not exist;

[0068] generating an eigenvalue image corresponding to the target image block, and dividing the eigenvalue image corresponding to the target image block into a plurality of third image blocks;

[0069] generating a fourth eigenvalue image according to the eigenvalues ​​satisfying the preset condition in each of the third image blocks and the eigenvalues ​​of each of the tracking points;

[0070] Dividing the fourth eigenvalue image into a plurality of fourth image blocks, and determining an eigenvalue satisfying the preset condition in each of the fourth image blocks;

[0071] The fourth pixel coordinates of the feature values ​​satisfying the preset condition in each of the fourth image blocks in the image to be tracked are obtained, and the corresponding tracking points are extracted from the image to be tracked according to the fourth pixel coordinates.

[0072] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the computer device described above can refer to the corresponding process in the following feature extraction method embodiment, and will not be repeated here.

[0073] The following will be combined Figure 1 The scenario in the embodiment of the present invention is described in detail. Figure 1 The scenario is only used to explain the feature extraction method provided in the embodiment of the present invention, but does not constitute a limitation on the application scenario of the feature extraction method provided in the embodiment of the present invention.

[0074] See also Figure 2 , Figure 2 The present invention provides a flow chart of a feature extraction method provided by an embodiment of the present invention. The feature extraction method can be applied to a computer device to reduce the computational complexity of feature extraction and to improve the speed and real-time performance of feature extraction.

[0075] like Figure 2 As shown, the feature extraction method includes steps S101 to S104.

[0076] Step S101: Generate a first eigenvalue image corresponding to the image to be extracted.

[0077] In this embodiment, the image to be extracted is an image from which feature points need to be extracted. For example, in the scenario of optical flow tracking, optical flow tracking involves two frames of images, and the previous frame of image can be used as the image to be extracted, and the next frame of image can be used as the image to be tracked. In this way, after the feature points are extracted from the image to be extracted, the feature points can be tracked in the image to be tracked. For another example, in the scenario of feature point matching, feature point matching involves at least two images of feature points to be matched. In this case, the at least two images can be used as two images to be extracted. In this way, after the feature points are extracted from the two images to be extracted, matching can be performed based on the extracted feature points.

[0078] In some embodiments, the method of generating the first eigenvalue image corresponding to the image to be extracted may be: the computer device determines the eigenvalue of each pixel in the image to be extracted; and generates the first eigenvalue image corresponding to the image to be extracted according to the eigenvalue of each pixel in the image to be extracted. Figure 3 As shown, the image to be extracted 11 includes 3*3=9 pixels. The characteristic value of each of the 9 pixels can be obtained by eigenvalue calculation: 1 The eigenvalue of 1 , pixel A 2 The eigenvalue of 2 , pixel A 3 The eigenvalue of 3 , pixel A 4 The eigenvalue of 4 , pixel A 5 The eigenvalue of 5 , pixel A 6 The eigenvalue of 6 , pixel A 7 The eigenvalue of 7 , pixel A 8 The eigenvalue of 8 , pixel A 9 The eigenvalue of 9 According to the characteristic value of each pixel in these 9 pixels, Figure 3 The first eigenvalue image 12 in .

[0079] In some embodiments, the method for determining the eigenvalues ​​of the pixels in the image to be extracted may be: the computer device determines the neighborhood of the pixels in the image to be extracted, and generates a grayscale value matrix according to the grayscale value of each pixel in the neighborhood; determines the eigenvalues ​​of the grayscale value matrix, and uses the eigenvalues ​​of the grayscale value matrix as the eigenvalues ​​of the corresponding pixels. The size of the neighborhood can be set based on actual conditions, and the embodiments of the present invention do not specifically limit this. For example, the size of the neighborhood is 2*2, such as Figure 3 As shown, pixel A1 The neighborhood includes pixel A 1 , pixel A 2 , pixel A 4 and pixel A 5 , let pixel A 1 , pixel A 2 , pixel A 4 and pixel A 5 The gray values ​​are g 1 , g 2 , g 3 and g 4 , then the gray value matrix can be generated So the gray value matrix can be The characteristic value of pixel A is determined as 1 The characteristic value of .

[0080] In some embodiments, Figure 4 As shown, step S101 includes: sub-step S1011 to sub-step S1012.

[0081] Sub-step S1011, according to 1 / 2 N The image to be extracted is scaled to obtain the Nth layer image corresponding to the image to be extracted.

[0082] In this embodiment, N is 1 or 2. When N is 1, the image to be extracted can be scaled by 1 / 2 to obtain the first layer image corresponding to the image to be extracted, that is, the first layer image is 1 / 2 of the image to be extracted, so that the first layer image and the image to be extracted can form a two-layer image pyramid. When N is 2, the image to be extracted can be scaled by 1 / 2 to obtain the first layer image corresponding to the image to be extracted, and the first layer image can be scaled by 1 / 2. 2 The image to be extracted is scaled to obtain the second layer image corresponding to the image to be extracted, that is, the second layer image is 1 / 4 of the image to be extracted, so that the first layer image, the second layer image and the image to be extracted can form a three-layer image pyramid. In this embodiment, the image to be extracted is scaled to obtain a scaled image, and subsequent calculations are all calculated based on the scaled image, which can greatly reduce the amount of calculation for feature extraction and improve the speed of feature extraction.

[0083] Sub-step S1012: Generate an eigenvalue image corresponding to the Nth layer image, and determine the eigenvalue image as the first eigenvalue image.

[0084] In this embodiment, the computer device can determine the feature value of each pixel in the Nth layer image; generate a feature value image corresponding to the Nth layer image according to the feature value of each pixel in the Nth layer image, and use the feature value image corresponding to the Nth layer image as the first feature value image. Since the first feature value image is generated according to the feature value of each pixel in the Nth layer image, when extracting feature points from the image to be extracted later, it is necessary to convert the pixel coordinates of the target feature value in the Nth layer image to the pixel coordinates of the target feature value in the image to be extracted, and then extract feature points based on the pixel coordinates of the target feature value in the image to be extracted, so as to ensure the accuracy of feature point extraction.

[0085] Step S102: Divide the first eigenvalue image into a plurality of first image blocks, and generate a second eigenvalue image according to the eigenvalue in each first image block that meets a preset condition.

[0086] In this embodiment, the sizes of the multiple first image blocks are the same, for example, the sizes of the multiple first image blocks are all the first size, and the first size is a power of 2. For example, the first size is 16*16 or 8*8. Since the sizes of the multiple first image blocks are all powers of 2, they can be obtained by displacement, which simplifies the calculation process and reduces the amount of calculation for transplantation to a Field-Programmable Gate Array (FPGA) platform.

[0087] In some embodiments, the preset condition includes a characteristic value that is the largest characteristic value, a characteristic value that is greater than any characteristic value in the preset characteristic values, or a characteristic value that is the smallest characteristic value. The preset characteristic value can be set based on actual conditions, and the embodiments of the present invention do not specifically limit this.

[0088] Exemplarily, a second eigenvalue image is generated according to the largest eigenvalue in each first image block. By selecting the largest eigenvalue, there is no need to sort the eigenvalues, which can further reduce the computational complexity of feature extraction. For example, the preset condition is that the eigenvalue is the largest eigenvalue, and the size of the first eigenvalue image is 576*576, and the first size is 16*16. Then, the first eigenvalue image with a size of 576*576 can be divided into 36*36 first image blocks with a size of 16*16, so that the largest eigenvalue in each of the 36*36 first image blocks can form a second eigenvalue image with a size of 36*36.

[0089] Step S103: Divide the second eigenvalue image into a plurality of second image blocks, and extract the eigenvalue that meets the preset conditions in each second image block as the target eigenvalue.

[0090] In this embodiment, the sizes of the multiple second image blocks are the same. For example, the sizes of the multiple second image blocks are all the second size, and the second size is smaller than the first size. Among them, the second size can be set based on the actual situation, and the embodiment of the present invention does not specifically limit this. For example, the second size is 2*2, 3*3 or 4*4, etc. In this embodiment, by further dividing the second eigenvalue image into multiple second image blocks, and extracting the eigenvalues ​​that meet the preset conditions in each second image block as the target eigenvalues, the eigenvalues ​​can be further screened. Since the screened target eigenvalues ​​correspond to the feature points one by one, this can ensure that the feature points subsequently extracted according to the target eigenvalues ​​are evenly distributed.

[0091] For example, if the preset condition is that the eigenvalue is the largest eigenvalue and the second size is 3*3, then the second eigenvalue image with a size of 36*36 can be divided into 12*12=144 3*3 second image blocks, and the largest eigenvalue in each second image block is extracted as the target eigenvalue, and 144 target eigenvalues ​​can be obtained. For another example, Figure 5 As shown, the size of the second eigenvalue image 14 is 12*12, so the second eigenvalue image 14 can be divided into 4*4=16 3*3 second image blocks 15, and the largest eigenvalue in each second image block 15 is extracted as the target eigenvalue, and 16 target eigenvalues ​​can be obtained.

[0092] In some embodiments, the second eigenvalue image is divided into a plurality of second image blocks in a manner as follows: a computer device performs non-maximum suppression (NMS) on the second eigenvalue image to obtain a third eigenvalue image, and divides the third eigenvalue image into a plurality of second image blocks. Among them, the window size used for non-maximum suppression can be set based on actual conditions, and the embodiment of the present invention does not specifically limit this. For example, the window size used for non-maximum suppression is 2*2, 3*3 or 4*4. This embodiment can ensure the sparsity of the eigenvalues ​​after the suppression process by performing non-maximum suppression on the eigenvalue image, thereby ensuring that the number of target eigenvalues ​​extracted subsequently is appropriate and will not aggregate.

[0093] Step S104: obtaining the first pixel coordinates of each target feature value in the image to be extracted, and extracting corresponding feature points from the image to be extracted according to the first pixel coordinates.

[0094] In this embodiment, when the first eigenvalue image is generated according to the eigenvalue of each pixel in the image to be extracted, the pixel corresponding to each first pixel coordinate in the image to be extracted can be extracted as the feature point. For example, a total of 10 target eigenvalues ​​are extracted, and the first pixel coordinates of these 10 target eigenvalues ​​in the image to be extracted are pixel coordinates P1 , pixel coordinate P 2 , pixel coordinate P 3 , pixel coordinate P 4 , pixel coordinate P 5 , pixel coordinate P 6 , pixel coordinate P 7 , pixel coordinate P 8 , pixel coordinate P 9 and pixel coordinates P 10 , then the pixel coordinates P can be extracted from the image to be extracted 1 , pixel coordinate P 2 , pixel coordinate P 3 , pixel coordinate P 4 , pixel coordinate P 5 , pixel coordinate P 6 , pixel coordinate P 7 , pixel coordinate P 8 , pixel coordinate P 9 and pixel coordinates P 10 The corresponding pixel points are used as feature points, so that 10 feature points can be obtained.

[0095] In some embodiments, when the first eigenvalue image is generated based on the eigenvalue of each pixel in the Nth layer image, it is necessary to convert the pixel coordinates of the target eigenvalue in the Nth layer image to the pixel coordinates of the target eigenvalue in the image to be extracted. Therefore, the computer device can obtain the first pixel coordinates of each target eigenvalue in the image to be extracted by: when N is 1, obtaining the second pixel coordinates of each target eigenvalue in the first layer image; multiplying each second pixel coordinate by 2 to enlarge it, and obtaining the first pixel coordinates of each target eigenvalue in the image to be extracted. For example, a total of 5 target eigenvalues ​​are extracted, and the second pixel coordinates of these 5 target eigenvalues ​​in the first layer image are (x 1 ,y 1 )、(x 2 ,y 2 )、(x 3 ,y 3 )、(x 4 ,y 4 ) and (x 5 ,y 5 ), then the first pixel coordinates of these five target feature values ​​in the image to be extracted are (x 1 *2,y 1 *2)、(x 2 *2,y 2 *2)、(x 3 *2,y 3 *2)、(x 4 *2,y 4*2) and (x 5 *2,y 5 *2).

[0096] In some embodiments, the first pixel coordinate is multiplied by the second pixel coordinate by 2 N In the case of zooming in, according to the first pixel coordinate, the method of extracting the corresponding feature point from the image to be extracted can be: determining the first neighborhood of the pixel corresponding to the first pixel coordinate in the image to be extracted; determining the feature value of each pixel in the first neighborhood; extracting the pixel corresponding to the feature value that meets the preset condition from the first neighborhood as the feature point. The size of the first neighborhood can be set based on the actual situation, and the embodiment of the present invention does not specifically limit this. For example, the size of the first neighborhood is 2*2, 3*3 or 4*4, etc.

[0097] For example, the preset condition is that the eigenvalue is the largest eigenvalue, and the first pixel coordinates of the five target eigenvalues ​​in the image to be extracted are (x 1 *2,y 1 *2)、(x 2 *2,y 2 *2)、(x 3 *2,y 3 *2)、(x 4 *2,y 4 *2) and (x 5 *2,y 5 *2), and (x 1 *2,y 1 *2)、(x 2 *2,y 2 *2)、(x 3 *2,y 3 *2)、(x 4 *2,y 4 *2) and (x 5 *2,y 5 *2) The pixel points corresponding to the five first pixel coordinates in the image to be extracted are pixel points B 11 , pixel B 21 , pixel B 31 , pixel B 41 and pixel B 51 , assume that the size of the first neighborhood is 2*2, and pixel B 11 The first neighborhood of includes pixel B 11 , pixel B 12 , pixel B 13 and pixel B 14 , since pixel B 12 The eigenvalue of is the largest, then pixel point B is extracted from the image to be extracted. 12 as a feature point.

[0098] Similarly, pixel B 21 The first neighborhood of includes pixel B 21 , pixel B 22 , pixel B 23 and pixel B 24 , since pixel B 21 The eigenvalue of is the largest, then pixel point B is extracted from the image to be extracted. 21 As a feature point. Similarly, pixel B 31 The first neighborhood of includes pixel B 31 , pixel B 32 , pixel B 33 and pixel B 34 , since pixel B 34 The eigenvalue of is the largest, then pixel point B is extracted from the image to be extracted. 34 as a feature point.

[0099] Similarly, pixel B 41 The first neighborhood of includes pixel B 41 , pixel B 42 , pixel B 43 and pixel B 44 , since pixel B 44 The eigenvalue of is the largest, then pixel point B is extracted from the image to be extracted. 44 As a feature point. Similarly, pixel B 51 The first neighborhood of includes pixel B 51 , pixel B 52 , pixel B 53 and pixel B 54 , since pixel B 51 The eigenvalue of is the largest, then pixel point B is extracted from the image to be extracted. 51 as a feature point.

[0100] In some embodiments, the computer device may obtain the first pixel coordinates of each target feature value in the image to be extracted in the following manner: when N is 2, the computer device obtains the third pixel coordinates of each target feature value in the second layer image corresponding to the image to be extracted; each third pixel coordinate is multiplied by 2 to enlarge, and the second pixel coordinates of each target feature value in the first layer image are obtained; the second neighborhood of the pixel point corresponding to each second pixel coordinate is determined in the first layer image, and the feature value of each pixel point in each second neighborhood is determined; the pixel coordinates of the pixel point corresponding to the feature value that meets the preset conditions in each second neighborhood are multiplied by 2 to obtain the first pixel coordinates of each target feature value in the image to be extracted. Among them, the size of the second neighborhood can be set based on the actual situation, and the embodiment of the present invention does not specifically limit this. For example, the size of the second neighborhood is 2*2, 3*3 or 4*4, etc.

[0101] For example, the preset condition is that the eigenvalue is the largest eigenvalue, and the third pixel coordinate of a target eigenvalue in the second layer image is (x 6 ,y 6 ), then the second pixel coordinate of this target feature value in the first layer image is (x 6 *2,y 6 *2), pixel coordinates (x 6 *2,y 6 *2) The pixel in the first layer image is pixel C 11 , pixel C 11 The second neighborhood includes pixel C 11 , pixel C 12 , pixel C 13 and pixel C 14 , since pixel C 12 The eigenvalue of is the largest, then for pixel C 12 The pixel coordinates (x 6 *2+1,y 6 *2+1) multiplied by 2, to obtain the first pixel coordinate (x 6 *4+1*2,y 6 *4+1*2).

[0102] In some embodiments, a computer device generates a first eigenvalue image corresponding to an image to be extracted; obtains the number of feature points to be extracted; determines the first size of the first image block and the second size of the second image block according to the number to be extracted, wherein the first size is larger than the second size; divides the first eigenvalue image into a plurality of first image blocks of the first size, and generates a second eigenvalue image according to the eigenvalues ​​that meet the preset conditions in each first image block; divides the second eigenvalue image into a plurality of second image blocks of the second size, and extracts the eigenvalues ​​that meet the preset conditions in each second image block as target eigenvalues; obtains the first pixel coordinates of each target eigenvalue in the image to be extracted, and extracts the corresponding feature points from the image to be extracted according to the first pixel coordinates. Determining the size of the first image block and the size of the second image block by the number of feature points to be extracted can ensure that the number of feature points extracted meets the set number to be extracted, and meet the user's various requirements for the number of feature points to be extracted.

[0103] In some embodiments, according to the number to be extracted, the first size of the first image block and the second size of the second image block may be determined in a manner that: the computer device obtains the first size and the second size corresponding to the number to be extracted from a preset relationship table. The preset relationship table is used to store the correspondence between the number to be extracted and the first size and the second size, and the correspondence between the number to be extracted and the first size and the second size can be set based on actual conditions, and the embodiments of the present invention do not specifically limit this.

[0104] In some embodiments, after step S104, it also includes: using a preset optical flow tracking algorithm to determine multiple tracking points in the image to be tracked based on the extracted multiple feature points; when the number of tracking points is less than the number of feature points, determining the target image block in the image to be tracked where there is no tracking point; generating a feature value image corresponding to the target image block, and dividing the feature value image corresponding to the target image block into multiple third image blocks; generating a fourth feature value image according to the feature value that meets the preset condition in each third image block and the feature value of each tracking point; dividing the fourth feature value image into multiple fourth image blocks, and determining the feature value that meets the preset condition in each fourth image block; obtaining the fourth pixel coordinates of the feature value that meets the preset condition in each fourth image block in the image to be tracked, and extracting the corresponding tracking point from the image to be tracked according to the fourth pixel coordinates. In some embodiments, when the number of tracking points is greater than or equal to the number of feature points, the computer device does not perform processing. Among them, the preset optical flow tracking algorithm can be an LKT sparse optical flow tracking algorithm or an LK pyramid optical flow tracking algorithm. In this embodiment, when the number of tracking points is less than the number of feature points, only the target image block without tracking points is used to detect new tracking points, which greatly reduces the amount of calculation and improves the tracking speed.

[0105] For example, the preset condition is that the eigenvalue is the largest eigenvalue, the image to be tracked with a size of 192*192 can be divided into 4*4=16 image blocks of 48*48, 16 feature points are extracted from the image to be extracted, and 15 tracking points are matched in the image to be tracked, and one tracking point corresponds to a 48*48 image block, and the 48*48 image block without a tracking point is the target image block D. The size of the eigenvalue image E corresponding to the target image block D is 48*48, then the eigenvalue image E with a size of 48*48 can be divided into 3*3=9 third image blocks of 16*16, that is, one tracking point corresponds to 9 16*16 The third image block can set each eigenvalue in each third image block corresponding to the tracking point as the eigenvalue of the tracking point, thereby obtaining 144 third image blocks, and then according to the largest eigenvalue in each third image block, a fourth eigenvalue image with a size of 12*12 can be generated, and then the 12*12 fourth eigenvalue image is divided into 4*4 3*3 fourth image blocks, and the largest eigenvalue in each fourth image block is extracted as the target eigenvalue, thereby obtaining 16 target eigenvalues, and finally according to the 16 target eigenvalues, the corresponding 16 tracking points can be extracted from the image to be tracked, so that the same number of tracking points can be tracked.

[0106] For example, in the scenario of gesture recognition, four hand feature points are extracted from the hand image of the previous frame, recorded as hand feature point A1, hand feature point B1, hand feature point C1 and hand feature point D1; using the preset optical flow tracking algorithm, based on these four hand feature points, the hand feature points B2, hand feature point C2 and hand feature point D2 corresponding to the hand feature point B1, hand feature point C1 and hand feature point D1 are tracked in the hand image of the current frame, but the hand feature point corresponding to the hand feature point A1 is not tracked. At this time, four hand feature points can be extracted from the hand image of the current frame according to the feature point extraction method provided in the embodiment of the present invention. The four hand feature points include hand feature point B2, hand feature point C2 and hand feature point D2, and also include hand feature point A2 corresponding to hand feature point A1. In this way, the gesture can be determined according to hand feature point A1, hand feature point B1, hand feature point C1 and hand feature point D1 as well as hand feature point A2, hand feature point B2, hand feature point C2 and hand feature point D2.

[0107] It should be noted that the specific implementation process of extracting tracking points in this embodiment is the same as the principle of the specific implementation process of extracting feature points in the above-mentioned embodiment, and will not be described in detail here.

[0108] See also Figure 6 , Figure 6 It is a schematic block diagram of the structure of a feature extraction device provided by an embodiment of the present invention.

[0109] like Figure 6 As shown, the feature extraction device 200 includes:

[0110] The image processing module 210 is configured to generate a first eigenvalue image corresponding to the image to be extracted;

[0111] The image processing module 210 is further configured to divide the first eigenvalue image into a plurality of first image blocks, and generate a second eigenvalue image according to the eigenvalue satisfying a preset condition in each of the first image blocks;

[0112] The image processing module 210 is further configured to divide the second eigenvalue image into a plurality of second image blocks, and extract the eigenvalue satisfying the preset condition in each of the second image blocks as the target eigenvalue;

[0113] The feature extraction module 220 is configured to obtain the first pixel coordinates of each target feature value in the image to be extracted, and extract corresponding feature points from the image to be extracted according to the first pixel coordinates.

[0114] In some embodiments, the image processing module 210 is further configured to:

[0115] According to 1 / 2 N Scaling the image to be extracted to obtain an N-th layer image corresponding to the image to be extracted, where N is 1 or 2;

[0116] Generate an eigenvalue image corresponding to the N-th layer image, and determine the eigenvalue image as the first eigenvalue image.

[0117] In some embodiments, the image processing module 210 is further configured to:

[0118] When N is 1, obtaining the second pixel coordinate of each target feature value in the first layer image;

[0119] Each of the second pixel coordinates is multiplied by 2 to be enlarged, so as to obtain the first pixel coordinate of each of the target feature values ​​in the image to be extracted.

[0120] In some embodiments, the image processing module 210 is further configured to:

[0121] Determine a first neighborhood of a pixel point corresponding to the first pixel coordinate in the image to be extracted;

[0122] Determine a characteristic value of each pixel in the first neighborhood;

[0123] Pixel points corresponding to feature values ​​that meet the preset condition are extracted from the first neighborhood as feature points.

[0124] In some embodiments, the image processing module 210 is further configured to:

[0125] When N is 2, obtaining the third pixel coordinate of each target feature value in the second layer image corresponding to the image to be extracted;

[0126] Multiply each of the third pixel coordinates by 2 to enlarge the coordinates, and obtain a second pixel coordinate of each of the target feature values ​​in the first layer image;

[0127] Determine in the first layer image a second neighborhood of a pixel point corresponding to each of the second pixel coordinates, and determine a feature value of each pixel point in each of the second neighborhoods;

[0128] The pixel coordinates of the pixel points corresponding to each feature value satisfying the preset condition in the second neighborhood are multiplied by 2 to obtain the first pixel coordinates of each target feature value in the image to be extracted.

[0129] In some embodiments, the image processing module 210 is further configured to:

[0130] Perform non-maximum suppression processing on the second eigenvalue image to obtain a third eigenvalue image, and divide the third eigenvalue image into a plurality of second image blocks.

[0131] In some embodiments, the feature extraction device 200 further includes:

[0132] a size determination module, configured to obtain the number of feature points to be extracted, and determine a first size of the first image block and a second size of the second image block according to the number of feature points to be extracted, wherein the first size is larger than the second size;

[0133] The image processing module 210 is further configured to divide the first eigenvalue image into a plurality of first image blocks having a size of the first size;

[0134] The image processing module 210 is further configured to divide the second eigenvalue image into a plurality of second image blocks having a size of the second size.

[0135] In some embodiments, the feature extraction device 200 further includes:

[0136] A tracking module is configured to use a preset optical flow tracking algorithm to determine a plurality of tracking points in the image to be tracked based on the extracted plurality of feature points;

[0137] The image processing module 210 is further configured to:

[0138] When the number of the tracking points is less than the number of the feature points, determining a target image block in the image to be tracked where the tracking points do not exist;

[0139] generating an eigenvalue image corresponding to the target image block, and dividing the eigenvalue image corresponding to the target image block into a plurality of third image blocks;

[0140] generating a fourth eigenvalue image according to the eigenvalues ​​satisfying the preset conditions in each of the third image blocks and the eigenvalues ​​of each of the tracking points;

[0141] Dividing the fourth eigenvalue image into a plurality of fourth image blocks, and determining an eigenvalue satisfying the preset condition in each of the fourth image blocks;

[0142] The feature extraction module 220 is further configured to obtain fourth pixel coordinates of the feature values ​​satisfying the preset condition in each of the fourth image blocks in the image to be tracked, and extract corresponding tracking points from the image to be tracked according to the fourth pixel coordinates.

[0143] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the feature extraction device described above can refer to the corresponding process in the aforementioned feature extraction method embodiment, and will not be repeated here.

[0144] An embodiment of the present invention further provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any feature extraction method provided in the description of the embodiment of the present invention.

[0145] The storage medium may be an internal storage unit of the computer device described in the above embodiment, such as a hard disk or memory of the computer device. The storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device.

[0146] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0147] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0148] The serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A feature extraction method, It is characterized in that include: Generate a first eigenvalue image corresponding to the image to be extracted; Dividing the first eigenvalue image into a plurality of first image blocks, and generating a second eigenvalue image according to an eigenvalue satisfying a preset condition in each of the first image blocks; Dividing the second eigenvalue image into a plurality of second image blocks, and extracting an eigenvalue satisfying the preset condition in each of the second image blocks as a target eigenvalue; The first pixel coordinates of each target feature value in the image to be extracted are obtained, and the corresponding feature points are extracted from the image to be extracted according to the first pixel coordinates.

2. The feature extraction method according to claim 1, It is characterized in that The step of generating a first eigenvalue image corresponding to the image to be extracted includes: According to 1 / 2 N Scaling the image to be extracted to obtain an N-th layer image corresponding to the image to be extracted, where N is 1 or 2; Generate an eigenvalue image corresponding to the N-th layer image, and determine the eigenvalue image as the first eigenvalue image.

3. The feature extraction method according to claim 2, It is characterized in that The obtaining of the first pixel coordinate of each target feature value in the image to be extracted includes: When N is 1, obtaining the second pixel coordinate of each target feature value in the first layer image; Each of the second pixel coordinates is multiplied by 2 to be enlarged, so as to obtain the first pixel coordinate of each of the target feature values ​​in the image to be extracted.

4. The feature extraction method according to claim 1, It is characterized in that The step of extracting corresponding feature points from the image to be extracted according to the first pixel coordinates includes: Determine a first neighborhood of a pixel point corresponding to the first pixel coordinate in the image to be extracted; Determine a characteristic value of each pixel in the first neighborhood; Pixel points corresponding to feature values ​​that meet the preset condition are extracted from the first neighborhood as feature points.

5. The feature extraction method according to claim 2, It is characterized in that The obtaining of the first pixel coordinate of each target feature value in the image to be extracted includes: When N is 2, obtaining the third pixel coordinate of each target feature value in the second layer image corresponding to the image to be extracted; Multiply each of the third pixel coordinates by 2 to enlarge the coordinates, and obtain a second pixel coordinate of each of the target feature values ​​in the first layer image; Determine in the first layer image a second neighborhood of a pixel point corresponding to each of the second pixel coordinates, and determine a feature value of each pixel point in each of the second neighborhoods; The pixel coordinates of the pixel points corresponding to each feature value satisfying the preset condition in the second neighborhood are multiplied by 2 to obtain the first pixel coordinates of each target feature value in the image to be extracted.

6. The feature extraction method according to claim 1, It is characterized in that The step of dividing the second eigenvalue image into a plurality of second image blocks comprises: Perform non-maximum suppression processing on the second eigenvalue image to obtain a third eigenvalue image, and divide the third eigenvalue image into a plurality of second image blocks.

7. The feature extraction method according to any one of claims 1 to 6, It is characterized in that Before dividing the first eigenvalue image into a plurality of first image blocks and generating a second eigenvalue image according to an eigenvalue satisfying a preset condition in each of the first image blocks, the method further includes: Get the number of feature points to be extracted; Determine a first size of the first image block and a second size of the second image block according to the number to be extracted, wherein the first size is larger than the second size; The step of dividing the first eigenvalue image into a plurality of first image blocks comprises: Dividing the first eigenvalue image into a plurality of first image blocks having a size of the first size; The step of dividing the second eigenvalue image into a plurality of second image blocks comprises: The second eigenvalue image is divided into a plurality of second image blocks having a size of the second size.

8. The feature extraction method according to any one of claims 1 to 6, It is characterized in that After extracting the corresponding feature points from the image to be extracted according to the first pixel coordinates, the method further includes: Using a preset optical flow tracking algorithm, multiple tracking points are determined in the image to be tracked based on the extracted multiple feature points; When the number of the tracking points is less than the number of the feature points, determining a target image block in the image to be tracked where the tracking points do not exist; generating an eigenvalue image corresponding to the target image block, and dividing the eigenvalue image corresponding to the target image block into a plurality of third image blocks; generating a fourth eigenvalue image according to the eigenvalues ​​satisfying the preset condition in each of the third image blocks and the eigenvalues ​​of each of the tracking points; Dividing the fourth eigenvalue image into a plurality of fourth image blocks, and determining an eigenvalue satisfying the preset condition in each of the fourth image blocks; The fourth pixel coordinates of the feature values ​​satisfying the preset condition in each of the fourth image blocks in the image to be tracked are obtained, and the corresponding tracking points are extracted from the image to be tracked according to the fourth pixel coordinates.

9. A feature extraction device, It is characterized in that The feature extraction device comprises: An image processing module, configured to generate a first eigenvalue image corresponding to the image to be extracted; The image processing module is further configured to divide the first eigenvalue image into a plurality of first image blocks, and generate a second eigenvalue image according to an eigenvalue satisfying a preset condition in each of the first image blocks; The image processing module is further configured to divide the second eigenvalue image into a plurality of second image blocks, and extract the eigenvalue satisfying the preset condition in each of the second image blocks as the target eigenvalue; The feature extraction module is configured to obtain the first pixel coordinates of each target feature value in the image to be extracted, and extract corresponding feature points from the image to be extracted according to the first pixel coordinates.

10. A computer device, It is characterized in that The computer device includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the feature extraction method according to any one of claims 1 to 8 is realized.