Image feature extraction methods

Abstract

The present application relates to a method for extracting image features, comprising: preprocessing an image to be processed to obtain a first preprocessed image and a second preprocessed image; wherein the first preprocessed image is a ROI color image, and the second preprocessed image is a ROI grayscale image obtained based on the first preprocessed image; extracting a set of feature points from the second preprocessed image; sampling the first preprocessed image based on the set of feature points to generate a sampled image; and extracting image features from the sampled image based on a DL2‑net network structure to generate a set of feature vectors corresponding to the set of feature points. Because the DL2‑net network structure in the present application includes multiple convolutional layers and multiple depth-separation convolutional layers, and employs a combination of convolutional layers and depth-separation convolutional layers, compared to the traditional L2‑net network structure that only uses convolutional layers, the number of model parameters is reduced and computational efficiency is improved without significantly differing in accuracy.

Description

Technical Field

[0001] The present application relates to the technical field of image feature extraction, and in particular to an image feature extraction method. Background Art

[0002] The image feature extraction methods in the existing technology mostly use manually designed description operators. In the main direction calculation stage, they rely too much on the gradient direction of local area pixels, which may cause inaccurate main direction positioning, resulting in too many parameters and high calculation amount during feature extraction, and serious errors in feature extraction and matching. Summary of the Invention

[0003] In order to at least to some extent overcome the problems of excessive parameters, high computational complexity, and serious errors in feature extraction and matching in related technologies, the present application provides an image feature extraction method.

[0004] The scheme of this application is as follows:

[0005] A method for extracting image features, comprising:

[0006] Preprocessing the image to be processed to obtain a first preprocessed image and a second preprocessed image; wherein the first preprocessed image is a color image of a region of interest (ROI), and the second preprocessed image is a grayscale image of the ROI obtained based on the first preprocessed image;

[0007] extracting a set of feature points of the second preprocessed image;

[0008] Based on the feature point set, sampling the first preprocessed image to generate a sampled image;

[0009] Image features are extracted from the sampled image based on a Depthwise separable convolution L2-net (DL2-net) network structure configured with a depthwise separable convolution layer to generate a feature vector set corresponding to the feature point set; wherein the DL2-net network structure includes multiple convolution layers and multiple depthwise separable convolution layers, and the image input channel of the DL2-net network structure is at least 3.

[0010] Preferably, in an implementable manner of the present application, the DL2-net network structure includes: a first convolutional layer, a second convolutional layer, a first depth-separation convolutional layer, a second depth-separation convolutional layer, a third depth-separation convolutional layer, a fourth depth-separation convolutional layer and a fifth depth-separation convolutional layer;

[0011] The first convolutional layer is the first layer of the DL2-net network structure;

[0012] The second convolutional layer is the tail layer of the DL2-net network structure;

[0013] The first depth separation convolution layer, the second depth separation convolution layer, the third depth separation convolution layer, the fourth depth separation convolution layer and the fifth depth separation convolution layer are intermediate layers of the DL2-net network structure.

[0014] Preferably, in an implementable method of the present application, the image input channel of the first convolutional layer is 32*32*3; the image input channel of the first depth separation convolutional layer is 16*16*32; the feature input channel of the second depth separation convolutional layer is 16*16*32; the feature input channel of the third depth separation convolutional layer is 8*8*64; the feature input channel of the fourth depth separation convolutional layer is 8*8*64; the feature input channel of the fifth depth separation convolutional layer is 4*4*128; and the feature input channel of the second convolutional layer is 2*2*128.

[0015] Preferably, in an implementable method of the present application, the convolution kernel size of the first convolution layer is 3x3, and the number of output channels is 32; the convolution kernel size of the first depth separation convolution layer is 3x3, and the number of output channels is 32; the convolution kernel size of the second depth separation convolution layer is 3x3, and the number of output channels is 64; the convolution kernel size of the third depth separation convolution layer is 3x3, and the number of output channels is 64; the convolution kernel size of the fourth depth separation convolution layer is 3x3, and the number of output channels is 128; the convolution kernel size of the fifth depth separation convolution layer is 3x3, and the number of output channels is 128; the convolution kernel size of the second convolution layer is 3x3, and the number of output channels is 128.

[0016] Preferably, in an implementable manner of the present application, the sliding step size of the first convolutional layer is 2; the sliding step size of the first depth separation convolutional layer is 1; the sliding step size of the second depth separation convolutional layer is 2; the sliding step size of the third depth separation convolutional layer is 1; the sliding step size of the fourth depth separation convolutional layer is 2; the sliding step size of the fifth depth separation convolutional layer is 2; and the sliding step size of the second convolutional layer is 1.

[0017] Preferably, in an implementable manner of the present application, the convolutional layer and the depth separation convolutional layer of the DL2-net network structure both use a batch normalization method and a Relu activation function.

[0018] Preferably, in an achievable manner of the present application, preprocessing the image to be processed to obtain a first preprocessed image and a second preprocessed image includes:

[0019] Performing noise reduction filtering on the image to be processed to obtain a noise-reduced image;

[0020] If a cropping instruction is received, cropping the denoised image based on the cropping region coordinates in the cropping instruction to obtain the first pre-processed image; if no cropping instruction is received, using the denoised image as the first pre-processed image;

[0021] Performing color space transformation on the first preprocessed image to obtain the second preprocessed image.

[0022] The technical solution provided by this application may include the following beneficial effects: The image feature extraction method in this application includes: preprocessing the image to be processed to obtain a first preprocessed image and a second preprocessed image; wherein the first preprocessed image is an ROI color image, and the second preprocessed image is an ROI grayscale image obtained based on the first preprocessed image; extracting a set of feature points from the second preprocessed image; sampling the first preprocessed image based on the feature point set to generate a sampled image; extracting image features from the sampled image based on the DL2-net network structure to generate a set of feature vectors corresponding to the feature point set. Since the DL2-net network structure in this application includes multiple convolutional layers and multiple depth-separated convolutional layers, and adopts a combination of convolutional layers and depth-separated convolutional layers, compared with the traditional L2-net network structure that only uses convolutional layers, the number of model parameters is reduced and the computational efficiency is improved without significantly differing in accuracy. In addition, the image input channels of the DL2-net network structure are at least 3, which better preserves the image color information than the single-channel input of the existing L2-net network structure.

[0023] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0025] Figure 1 This is a flow chart of an image feature extraction method provided by an embodiment of the present application;

[0026] Figure 2 This is a schematic diagram of the DL2-net network structure in an image feature extraction method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0028] A method for extracting image features, referring to Figure 1 ,include:

[0029] S11: Preprocessing the image to be processed to obtain a first preprocessed image and a second preprocessed image; wherein the first preprocessed image is a ROI color image, and the second preprocessed image is a ROI grayscale image obtained based on the first preprocessed image;

[0030] Preprocessing the image to be processed to obtain a first preprocessed image and a second preprocessed image includes:

[0031] The image to be processed is subjected to denoising filtering to obtain a denoised image; the denoising filtering may be, but is not limited to, Gaussian filtering, median filtering, box filtering, etc. Preferably, the present embodiment adopts a Gaussian filtering method with a kernel size of 3x3.

[0032] If a cropping instruction is received, the denoised image is cropped based on the cropping region coordinates in the cropping instruction to obtain a first pre-processed image; if no cropping instruction is received, the denoised image is used as the first pre-processed image;

[0033] Performing color space transformation on the first preprocessed image to obtain a second preprocessed image.

[0034] In this embodiment, the first pre-processed image is a ROI color image, and the second pre-processed image is a ROI grayscale image obtained based on the first pre-processed image. The ROI color image is subjected to color space transformation to obtain a corresponding ROI grayscale image.

[0035] S12: extracting a feature point set of the second pre-processed image;

[0036] A Difference of Gaussian (DoG) operator is used according to the second pre-processed image obtained in S11 to obtain a feature point set of the second pre-processed image.

[0037] Specifically, extract the feature point set [p1, p2…p n ], where p = (x, y).

[0038] The corresponding Hessian matrix is ​​calculated for the acquired feature points to obtain the response value of each point, and the feature point data [p1, p2…p m ], where (m≤n).

[0039] S13: Sampling the first preprocessed image based on the feature point set to generate a sampled image;

[0040] According to the feature point set, the first pre-processed image is sampled with the coordinates of each feature point as the center coordinate of the sampled image to generate a 32×32×m sampled image [img1,img2…img m ].

[0041] S14: Perform image feature extraction on the sampled image based on a DL2-net network structure to generate a feature vector set corresponding to the feature point set; wherein the DL2-net network structure includes multiple convolutional layers and multiple depth separation convolutional layers, and the image input channel of the DL2-net network structure is at least 3.

[0042] Based on the DL2-net network structure, the image features of the sampled image are extracted to generate a feature vector set, such as [f1,f2…f m ].

[0043] The image feature extraction method in the present application includes: preprocessing the image to be processed to obtain a first preprocessed image and a second preprocessed image; wherein the first preprocessed image is an ROI color image, and the second preprocessed image is an ROI grayscale image obtained based on the first preprocessed image; extracting a feature point set of the second preprocessed image; sampling the first preprocessed image based on the feature point set to generate a sampled image; extracting image features from the sampled image based on the DL2-net network structure to generate a feature vector set corresponding to the feature point set. Since the DL2-net network structure in the present application includes multiple layers of convolutional layers and multiple layers of depth separation convolutional layers, the convolutional layers and the depth separation convolutional layers are combined with each other. Compared with the traditional L2-net network structure that only uses convolutional layers, the model parameter amount is reduced and the computational efficiency is improved under the premise that the accuracy gap is not obvious. In addition, the image input channel of the DL2-net network structure is at least 3, which better retains the image color information than the single-channel input of the existing L2-net network structure.

[0044] The image feature extraction method in some embodiments refers to Figure 2 , the DL2-net network structure includes: the first convolutional layer, the second convolutional layer, the first depth separation convolutional layer, the second depth separation convolutional layer, the third depth separation convolutional layer, the fourth depth separation convolutional layer and the fifth depth separation convolutional layer;

[0045] The first convolutional layer is the first layer of the DL2-net network structure;

[0046] The second convolutional layer is the tail layer of the DL2-net network structure;

[0047] The first depth separation convolution layer, the second depth separation convolution layer, the third depth separation convolution layer, the fourth depth separation convolution layer and the fifth depth separation convolution layer are the intermediate layers of the DL2-net network structure.

[0048] Furthermore, the image input channel of the first convolutional layer is 32*32*3; the feature input channel of the first depth-wise separation convolutional layer is 16*16*32; the feature input channel of the second depth-wise separation convolutional layer is 16*16*32; the feature input channel of the third depth-wise separation convolutional layer is 8*8*64; the feature input channel of the fourth depth-wise separation convolutional layer is 8*8*64; the feature input channel of the fifth depth-wise separation convolutional layer is 4*4*128; and the feature input channel of the second convolutional layer is 2*2*128.

[0049] Furthermore, the convolution kernel size of the first convolution layer is 3x3, and the number of output channels is 32; the convolution kernel size of the first depth-wise separation convolution layer is 3x3, and the number of output channels is 32; the convolution kernel size of the second depth-wise separation convolution layer is 3x3, and the number of output channels is 64; the convolution kernel size of the third depth-wise separation convolution layer is 3x3, and the number of output channels is 64; the convolution kernel size of the fourth depth-wise separation convolution layer is 3x3, and the number of output channels is 128; the convolution kernel size of the fifth depth-wise separation convolution layer is 3x3, and the number of output channels is 128; the convolution kernel size of the second convolution layer is 3x3, and the number of output channels is 128.

[0050] Furthermore, the sliding step size of the first convolutional layer is 2; the sliding step size of the first depth-wise separation convolutional layer is 1; the sliding step size of the second depth-wise separation convolutional layer is 2; the sliding step size of the third depth-wise separation convolutional layer is 1; the sliding step size of the fourth depth-wise separation convolutional layer is 2; the sliding step size of the fifth depth-wise separation convolutional layer is 2; and the sliding step size of the second convolutional layer is 1.

[0051] Furthermore, the convolutional layers and depth separation convolutional layers of the DL2-net network structure both use batch normalization methods and ReLU activation functions.

[0052] In this example, experiments show that the existing L2-net architecture uses 878,112 parameters, while the DL2-net architecture in this example uses only 39,940 parameters, a reduction of nearly 95%. Feature extraction time for 1,000 sampled images was reduced from 4.36 seconds in the original L2-net architecture to 1.663 seconds, a 46% improvement in efficiency.

[0053] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0054] It should be noted that, in the description of this application, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0055] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0056] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0057] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0058] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0059] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0060] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0061] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for extracting image features, characterized in that: include: Preprocessing the image to be processed to obtain a first preprocessed image and a second preprocessed image; wherein the first preprocessed image is a ROI color image, and the second preprocessed image is a ROI grayscale image obtained based on the first preprocessed image; extracting a set of feature points of the second preprocessed image; Based on the feature point set, sampling the first preprocessed image to generate a sampled image; Performing image feature extraction on the sampled image based on a DL2-net network structure to generate a feature vector set corresponding to the feature point set; wherein the DL2-net network structure includes multiple convolutional layers and multiple depth separation convolutional layers, and the image input channel of the DL2-net network structure is at least 3; The DL2-net network structure includes: a first convolutional layer, a second convolutional layer, a first depth separation convolutional layer, a second depth separation convolutional layer, a third depth separation convolutional layer, a fourth depth separation convolutional layer and a fifth depth separation convolutional layer; The first convolutional layer is the first layer of the DL2-net network structure; The second convolutional layer is the tail layer of the DL2-net network structure; The first depth separation convolution layer, the second depth separation convolution layer, the third depth separation convolution layer, the fourth depth separation convolution layer and the fifth depth separation convolution layer are intermediate layers of the DL2-net network structure; The image input channel of the first convolutional layer is 32*32*3; the feature input channel of the first depth separation convolutional layer is 16*16*32; the feature input channel of the second depth separation convolutional layer is 16*16*32; the feature input channel of the third depth separation convolutional layer is 8*8*64; the feature input channel of the fourth depth separation convolutional layer is 8*8*64; the feature input channel of the fifth depth separation convolutional layer is 4*4*128; the feature input channel of the second convolutional layer is 2*2*128; The convolution kernel size of the first convolution layer is 3x3, and the number of output channels is 32; the convolution kernel size of the first depth separation convolution layer is 3x3, and the number of output channels is 32; the convolution kernel size of the second depth separation convolution layer is 3x3, and the number of output channels is 64; the convolution kernel size of the third depth separation convolution layer is 3x3, and the number of output channels is 64; the convolution kernel size of the fourth depth separation convolution layer is 3x3, and the number of output channels is 128; the convolution kernel size of the fifth depth separation convolution layer is 3x3, and the number of output channels is 128; the convolution kernel size of the second convolution layer is 3x3, and the number of output channels is 128.

2. The method according to claim 1, characterized in that The sliding step size of the first convolution layer is 2; the sliding step size of the first depth separation convolution layer is 1; the sliding step size of the second depth separation convolution layer is 2; the sliding step size of the third depth separation convolution layer is 1; the sliding step size of the fourth depth separation convolution layer is 2; the sliding step size of the fifth depth separation convolution layer is 2; and the sliding step size of the second convolution layer is 1.

3. The method according to claim 1, characterized in that The convolutional layer and depth separation convolutional layer of the DL2-net network structure both use batch normalization method and Relu activation function.

4. The method according to claim 1, wherein The preprocessing of the image to be processed to obtain a first preprocessed image and a second preprocessed image includes: Performing noise reduction filtering on the image to be processed to obtain a noise-reduced image; If a cropping instruction is received, cropping the denoised image based on the cropping region coordinates in the cropping instruction to obtain the first pre-processed image; if no cropping instruction is received, using the denoised image as the first pre-processed image; Performing color space transformation on the first preprocessed image to obtain the second preprocessed image.

Images