Feature extraction method, device and electronic equipment

By using mask images to mark the region of interest in image processing, feature extraction is directly performed in the feature extraction network, which solves the problem of time-consuming cutting process and improves feature extraction efficiency and accuracy.

CN112418243BActive Publication Date: 2024-12-03MEGVII (BEIJING) TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011176076.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-10-28
Publication Date
2024-12-03
Estimated Expiration
2040-10-28

AI Technical Summary

Technical Problem

In the prior art, feature extraction efficiency depends on the cutout efficiency, and the cutout process is time-consuming and low efficiency, affecting the feature extraction efficiency.

Method used

By acquiring the target image and the corresponding mask image, the feature extraction network is used to extract the target image based on the mask image, eliminating the image cutting process, and directly focusing on the area of ​​interest for feature extraction.

Benefits of technology

It improves feature extraction efficiency, reduces the time spent on cutting images, avoids the constraints of cutting images efficiency, and enhances the accuracy of feature extraction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112418243B_ABST
    Figure CN112418243B_ABST
Patent Text Reader

Abstract

The present invention provides a feature extraction method, device and electronic device, which relate to the technical field of image processing, including: obtaining a target image of features to be extracted; obtaining a mask image corresponding to the target image; wherein the mask image corresponds to a region of interest in the target image; inputting the target image and the mask image into a feature extraction network, and extracting features of the target image based on the mask image through the feature extraction network. The present invention can effectively improve the efficiency of feature extraction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a feature extraction method, device and electronic equipment. Background Art

[0002] Most existing image recognition tasks involve feature extraction operations to extract feature information of images. At present, before performing feature extraction operations on images, most of them need to extract the area where the object is located in the image by detection, and then perform feature extraction on the extracted area where the object is located. Based on this, the efficiency of feature extraction often depends on the efficiency of image extraction, which has become a bottleneck restricting the efficiency of feature extraction. Summary of the invention

[0003] In view of this, an object of the present invention is to provide a feature extraction method, device and electronic device, which can perform feature extraction operations without clipping, thereby effectively improving feature extraction efficiency.

[0004] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0005] In a first aspect, an embodiment of the present invention provides a feature extraction method, comprising: obtaining a target image of features to be extracted; obtaining a mask image corresponding to the target image; wherein the mask image corresponds to a region of interest in the target image; inputting the target image and the mask image into a feature extraction network, and performing feature extraction on the target image based on the mask image through the feature extraction network.

[0006] Furthermore, the feature extraction network includes a plurality of sequentially connected convolutional layers; the step of performing feature extraction on the target image based on the mask image through the feature extraction network includes: for each of the convolutional layers in the feature extraction network, determining an input feature map corresponding to the convolutional layer, and performing feature extraction on the input feature map through the convolutional layer to obtain an output feature map corresponding to the convolutional layer; wherein the input feature map corresponding to the first convolutional layer in the feature extraction network is related to the target image and the mask image, and the input feature map corresponding to the second convolutional layer in the feature extraction network is related to the output feature map of the previous convolutional layer and the mask image; and determining the features of the region of interest in the target image according to the output feature map of the feature extraction network.

[0007] Furthermore, the step of determining the input feature map corresponding to the convolution layer includes: when the convolution layer is the first convolution layer, bitwise multiplying the target image with the mask image to obtain the input feature map corresponding to the first convolution layer; when the convolution layer is the second convolution layer, bitwise multiplying the output feature map of the previous convolution layer of the second convolution layer with the mask image to obtain the input feature map corresponding to the second convolution layer.

[0008] Furthermore, the region of interest includes at least one target object; the step of determining the features of the region of interest in the target image based on the output feature map of the feature extraction network includes: determining the target feature points of each of the target objects in the target image; obtaining the features corresponding to the target feature points of each of the target objects in the output feature map of the feature extraction network; and determining the features corresponding to the target feature points of each of the target objects as the features of the region of interest in the target image.

[0009] Furthermore, the step of obtaining features corresponding to target feature points of each target object in the output feature map of the feature extraction network includes: for each target feature point of the target object, bitwise multiplying the output feature map of the feature extraction network with the target feature point of the target object to obtain features corresponding to the target feature point of the target object.

[0010] Furthermore, the step of determining target feature points of each target object in the target image includes: performing object detection on the target image to obtain each target object in the target image; and for each target object, determining the target feature points of the target object.

[0011] Further, the step of obtaining the mask image corresponding to the target image includes: establishing an initial image according to the size of the target image; wherein the pixels of the initial image are a first value; determining a region of interest in the target image; and adjusting the pixels in the region of the initial image corresponding to the region of interest of the target image from the first value to a second value, thereby obtaining the mask image corresponding to the target image.

[0012] In a second aspect, an embodiment of the present invention further provides a feature extraction device, comprising: a target image acquisition module, used to acquire a target image of a feature to be extracted; a mask image acquisition module, used to acquire a mask image corresponding to the target image; wherein the mask image corresponds to a region of interest in the target image; and a feature extraction module, used to input the target image and the mask image into a feature extraction network, and perform feature extraction on the target image based on the mask image through the feature extraction network.

[0013] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor and a memory; a computer program is stored in the memory, and when the computer program is executed by the processor, it executes any one of the methods provided in the first aspect.

[0014] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods provided in the first aspect are executed.

[0015] The embodiment of the present invention provides a feature extraction method, device and electronic device, firstly obtaining a target image of features to be extracted and a mask image corresponding to the target image (corresponding to the region of interest in the target image), then inputting the target image and the mask image into a feature extraction network, and extracting features of the target image based on the mask image through the feature extraction network. The above method uses the mask image to mark the region of interest in the target image, so that the feature extraction network can focus on the region of interest in the target image and extract features from the region of interest. Compared with the method of first cutting out the image and then extracting features in the prior art, the embodiment of the present invention can omit the cutting out process, thereby saving the time spent on cutting out the image before feature extraction, and is no longer restricted by the cutting out efficiency, thereby effectively improving the feature extraction efficiency.

[0016] Other features and advantages of the embodiments of the present invention will be described in the following description, or some features and advantages can be inferred or determined without doubt from the description, or can be learned by implementing the above-mentioned techniques of the embodiments of the present invention.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown;

[0020] Figure 2 A schematic flow chart of a feature extraction method provided by an embodiment of the present invention is shown;

[0021] Figure 3A schematic diagram of the structure of a feature extraction network provided by an embodiment of the present invention is shown;

[0022] Figure 4 A schematic flow chart of another feature extraction method provided by an embodiment of the present invention is shown;

[0023] Figure 5 A structural schematic diagram of a feature extraction device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be described below in conjunction with the drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments.

[0025] At present, before extracting image features, it is usually necessary to perform a cutout process on the area where the features are to be extracted, which results in the efficiency of feature extraction being dependent on the cutout efficiency. If the image contains a large number of areas to be extracted, the image will need to be cut out multiple times, and each cutout process is accompanied by the transfer of computer memory (including moving and copying), that is, the time required for cutout is directly related to the time required for computer DMA (Direct Memory Access). Therefore, the time required for DMA during the cutout process will increase with the increase in the number of cutouts, resulting in low cutout efficiency, which in turn affects the efficiency of feature extraction. In order to improve this problem, an embodiment of the present invention provides a feature extraction method, device, and electronic device. The technology can be applied to scenarios where feature extraction is required. The following is a detailed introduction to the embodiment of the present invention.

[0026] Embodiment 1:

[0027] First, refer to Figure 1 An exemplary electronic device 100 for implementing a feature extraction method and apparatus according to an embodiment of the present invention is described.

[0028] like Figure 1 The electronic device 100 includes one or more processors 102, one or more storage devices 104, an input device 106, an output device 108, and an image acquisition device 110. These components are interconnected via a bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that Figure 1 The components and structures of the electronic device 100 shown are only exemplary and not restrictive. The electronic device may have Figure 1 Some of the components shown may also have Figure 1 Other components and structures are not shown.

[0029] The processor 102 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 102 can be a central processing unit (CPU), a graphics processing unit (GPU), or one or a combination of other forms of processing units with data processing capabilities and / or instruction execution capabilities, and can control other components in the electronic device 100 to perform desired functions.

[0030] The storage device 104 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 102 may run the program instructions to implement the client functions (implemented by the processor) and / or other desired functions in the embodiments of the present invention described below. Various applications and various data, such as various data used and / or generated by the application, may also be stored in the computer-readable storage medium.

[0031] The input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.

[0032] The output device 108 may output various information (eg, images or sounds) to the outside (eg, a user), and may include one or more of a display, a speaker, and the like.

[0033] The image acquisition device 110 can capture images desired by the user (eg, photos, videos, etc.), and store the captured images in the storage device 104 for use by other components.

[0034] Exemplarily, the electronic device used to implement the feature extraction method and apparatus according to the embodiments of the present invention may be implemented as an intelligent terminal such as a smart phone, a tablet computer, or a computer.

[0035] Embodiment 2:

[0036] See also Figure 2 The flowchart of a feature extraction method shown in FIG. 1 mainly includes the following steps S202 to S206:

[0037] Step S202, obtaining a target image from which features are to be extracted.

[0038] The target image may include at least one target object whose features are to be extracted, and may also include at least one region whose features are to be extracted. In one embodiment, the electronic device that executes the feature extraction method may obtain the target image by photographing a device with a photographing function (such as a smart phone or a camera), or may provide a manual upload channel for the user, and the user may select an image whose features are to be extracted from a local storage area of ​​the electronic device or download an image whose features are to be extracted from a network, and upload the image selected or downloaded by the user through the manual upload channel, thereby obtaining the target image.

[0039] Step S204: acquiring a mask image corresponding to the target image.

[0040] Among them, the mask image corresponds to the region of interest in the target image, the size of the mask image is the same as the size of the target image, and the region of interest can be understood as the region of interest in the feature extraction process, which can be used to characterize the region where the target object (such as a person, animal or object, etc.) is located in the target image, and can also be used to characterize a specified area in the target image, which can be determined by the user according to actual needs, such as a sky area, etc. In one embodiment, the region of interest in the target image can be determined in advance, and an initial image with the same size as the target image is established, and then in the initial image, the mask value of the point in the region of interest is set to 1, and the mask value of the point outside the region of interest is set to 0, and the mask image corresponding to the target image can be obtained. When determining the region of interest in the target image, it can be determined by using a target detection algorithm, a target recognition algorithm, a target segmentation algorithm, etc., or it can be directly circled manually.

[0041] Step S206: input the target image and the mask image into a feature extraction network, and extract features of the target image based on the mask image through the feature extraction network.

[0042] The feature extraction network can be implemented by using CNN (Convolutional Neural Network), RNN (Recurrent Neural Network) or Transform (spatial transformation network), etc. The feature extraction network is used to extract features of the target image based on the mask image. The input of the feature extraction network is the target image and the mask image, and the output is the feature map of the target image. Taking the use of CNN network as the feature extraction network as an example, the feature extraction network contains multiple convolutional layers. Since the pixels in the region of interest in the mask image are 1 and the pixels outside the region of interest are 0, the various convolutional layers in the feature extraction network can focus on the region of interest based on the mask image, and then the features of the region of interest can be extracted to obtain the features of the target image.

[0043] The above-mentioned feature extraction method provided by the embodiment of the present invention utilizes a mask image to mark the region of interest in the target image, so that the feature extraction network can focus on the region of interest in the target image and perform feature extraction on the region of interest. Compared with the method of first cutting out the image and then extracting features in the prior art, the embodiment of the present invention can omit the cutting out process, thereby saving the time spent on cutting out before feature extraction, and is no longer restricted by the cutting out efficiency, thereby effectively improving the feature extraction efficiency.

[0044] For the above step S204, in a specific implementation, the steps of acquiring the mask image corresponding to the target image may be performed with reference to the following steps a to c:

[0045] Step a, establishing an initial image according to the size of the target image. The pixels in the initial image are a first value, which may be 0, and the size of the initial image is the same as the size of the target image. In one embodiment, the pixels in the initial image may be represented in the form of a two-dimensional matrix, that is, each element in the two-dimensional matrix corresponds to a pixel in the initial image. If the pixels in the initial image are all 0, the value of each element in the two-dimensional matrix will also be 0.

[0046] Step b, determining the region of interest in the target image. In some embodiments, object detection may be performed on the target image to detect the target object contained in the target image, and the region where the target object is located in the target image is determined as the region of interest; of course, a user-specified region in the target image may also be determined as the region of interest.

[0047] Step c, adjusting the pixels in the area corresponding to the area of ​​interest of the target image in the initial image from the first value to the second value, and obtaining a mask image corresponding to the target image. The second value may be 1. In practical applications, the points in the initial image correspond to the points in the target image one by one, and the area corresponding to the area of ​​interest in the initial image is determined, and the pixels in the area of ​​the initial image are adjusted from 0 to 1, so that a mask image for marking the area of ​​interest can be obtained, and the area where the pixels with a value of 1 in the mask image are located corresponds to the above-mentioned area of ​​interest.

[0048] In one embodiment, the feature extraction network includes a plurality of sequentially connected convolutional layers. The input of each convolutional layer is the input feature map corresponding to the convolutional layer, and the output of each convolutional layer is the output feature map corresponding to the convolutional layer. For the first convolutional layer in the feature extraction network, the input feature map corresponding to the first convolutional layer can be obtained based on the target image and the mask image; for the second convolutional layer in the feature extraction network, the input feature map corresponding to the second convolutional layer can be obtained based on the output feature map corresponding to the previous convolutional layer of the convolutional layer and the mask image; the output feature map corresponding to each convolutional layer can be obtained by extracting features from the input feature map corresponding to the convolutional layer through the convolutional layer.

[0049] Based on the above feature extraction network, an embodiment of the present invention provides an implementation method of extracting features of a target image based on a mask image through a feature extraction network, and specifically, see the following steps 1 to 2:

[0050] Step 1, for each convolution layer in the feature extraction network, determine the input feature map corresponding to the convolution layer, and perform feature extraction on the input feature map through the convolution layer to obtain the output feature map corresponding to the convolution layer. The input feature map corresponding to the first convolution layer in the feature extraction network is related to the target image and the mask image, and the input feature map corresponding to the second convolution layer in the feature extraction network is related to the output feature map of the previous convolution layer and the mask image. The first convolution layer can be the convolution layer located in the first section of the feature extraction network, the input feature map corresponding to the first convolution layer is related to the target image, and the input feature map corresponding to the first convolution layer is also related to the mask image; the second convolution layer can be the convolution layer other than the first convolution layer in the feature extraction network, the input feature map corresponding to the second convolution layer is related to the output feature map of the previous convolution layer, and the input feature map corresponding to the second convolution layer is also related to the mask image. The embodiment of the present invention is illustrated by taking an example in which a feature extraction network includes three convolutional layers, wherein an input feature map A1 corresponding to the first convolutional layer is commonly related to the target image and the mask image, and the first convolutional layer performs feature extraction on the input feature map A1 corresponding to the convolutional layer to obtain an output feature map B1 corresponding to the first convolutional layer; an input feature map A2 corresponding to the second convolutional layer is commonly related to the input feature map A1 corresponding to the first convolutional layer and the mask image, and the second convolutional layer performs feature extraction on the input feature map A2 corresponding to the convolutional layer to obtain an output feature map B2 corresponding to the second convolutional layer; an input feature map A3 corresponding to the third convolutional layer is commonly related to the output feature map B2 corresponding to the second convolutional layer and the mask image, and the third convolutional layer performs feature extraction on the input feature map A3 corresponding to the convolutional layer to obtain an output feature map B3 corresponding to the third convolutional layer.

[0051] In practical applications, the embodiment of the present invention needs to determine the input feature map corresponding to each convolutional layer. Specifically, the input feature map corresponding to each convolutional layer can be determined by the following methods (1) and (2):

[0052] (1) When the convolution layer is the first convolution layer, the target image is bitwise multiplied with the mask image to obtain the input feature map corresponding to the first convolution layer. The mask image has a shielding effect. When the convolution layer is located at the head end of the feature extraction network, the convolution layer can be determined to be the first convolution layer. It can be understood that, assuming that point Q in the target image corresponds to point P in the mask image, if the mask value of point P in the mask image is 0, then when point Q in the target image is bitwise multiplied with point P in the mask image, the feature value of the point in the input feature map obtained will also be 0, that is, at this time, the input feature map cannot reflect the feature of the point, which is equivalent to shielding the point in the input feature map; if the mask value of point P in the mask image is 1, then when point Q in the target image is bitwise multiplied with point P in the mask image, the feature value of the point in the input feature map obtained will remain unchanged, so that the convolution layer can focus on the point and extract the feature of the point.

[0053] (2) When the convolution layer is the second convolution layer, the output feature map of the previous convolution layer corresponding to the second convolution layer is bitwise multiplied with the mask image to obtain the input feature map corresponding to the second convolution layer. When the convolution layer is not located at the head end of the feature extraction network, it can be determined that the convolution layer is the second convolution layer. Considering that the region of interest may have deviations during the feature extraction process, the embodiment of the present invention performs bitwise multiplication of the output feature map corresponding to each convolution layer with the mask image, thereby correcting the region of interest in each output feature map, improving the accuracy of the region of interest in the output feature map, and thus improving the accuracy of the feature extraction of the convolution layer to a certain extent.

[0054] Step 2, determining the features of the region of interest in the target image based on the output feature map of the feature extraction network. In one embodiment, the region of interest includes at least one target object, and the region of interest can be understood as the region where the target object is located in the target image. To facilitate understanding of the above step 2, an embodiment of the present invention also provides an implementation method for determining the features of the region of interest in the target image based on the output feature map of the feature extraction network, see the following steps 2.1 to 2.3:

[0055] Step 2.1, determine the target feature points of each target object in the target image. In one embodiment, the target image can be subjected to object detection to obtain each target object in the target image, and for each target object, the center point of the target object is determined, and the center point of the target object is used as the target feature point of the target object. In a specific implementation, the target image can be subjected to object detection by a variety of target detection methods such as R-CNN (Regions with CNN features) or Fast R-CNN (Fast Regions with CNN features) to determine the target objects contained in the target image, wherein the target objects may include people, animals or objects, etc. After determining the target objects contained in the target image, for each target object, the contour lines of each target object can be determined respectively, and for irregular contour lines such as people, the minimum rotation rectangle circumscribed by the contour can be determined, and the center point of the minimum rotation rectangle is used as the center point of the target object. It should be noted that the above embodiment is only an exemplary embodiment for determining the center point of a target object. In practical applications, other methods for determining the center point of a target object can be used, and the embodiment of the present invention does not limit this.

[0056] Step 2.2, obtain the features corresponding to the target feature points of each target object in the output feature map of the feature extraction network. Among them, the feature map of the center point of the target object in the output feature map of the feature extraction network can reflect the characteristics of the target object. Therefore, the embodiment of the present invention captures the features of the center point of the target object in the output feature map of the feature extraction network as the features corresponding to the target object. Optionally, the output feature map of the feature extraction network can be the output feature map of the convolutional layer at the end of the feature extraction network. In one embodiment, the output feature map can be represented by a matrix, and each element in the matrix represents the feature value of each point in the output feature map. Therefore, the features corresponding to the target object can be obtained by capturing the feature value corresponding to the center point of the target object.

[0057] In a specific implementation, for each target feature point of the target object, the output feature map of the feature extraction network is bitwise multiplied with the target feature point of the target object to obtain the feature corresponding to the target feature point of the target object. For example, the center point of each target object is represented in matrix form, that is, the element value corresponding to the center point of the target object is set to 1, and the remaining element values ​​are set to 0. By bitwise multiplying the center point of the target object in matrix form with the output feature map in matrix form, the feature value corresponding to the center point of each target object can be obtained, and the feature value corresponding to the center point of each target object is used as the feature of the target object.

[0058] Step 2.3, the features corresponding to the target feature points of each target object are determined as the features of the region of interest in the target image.

[0059] In summary, the above-mentioned feature extraction method provided by the embodiment of the present invention performs feature extraction on the target image based on the mask image, eliminating the cutout operation before feature extraction, thereby reducing the speed loss caused by memory movement caused by cutout in the fine-grained feature extraction process, thereby improving the feature extraction efficiency. In addition, by forcing the feature extraction network to pay attention to the region of interest in the target image through the mask image, the accuracy of feature extraction can be effectively improved, and the influence of surrounding environment pixels on feature extraction during the feature extraction process can also be avoided to a certain extent.

[0060] Embodiment three:

[0061] On the basis of the above-mentioned embodiment, this embodiment provides a specific example of applying the above-mentioned feature extraction method. First, the embodiment of the present invention provides a feature extraction network, the feature extraction network includes multiple convolutional layers. The embodiment of the present invention takes the feature extraction network including three convolutional layers as an example, and provides the following example: Figure 3 A structural schematic diagram of a feature extraction network is shown, wherein the input of the feature extraction network is a target image, a mask image and an object center point, and the output of the feature extraction network is a feature map corresponding to the target image. The oblique line area represents the input feature map corresponding to each convolutional layer, wherein the feature map of the target image includes object features of each target object in the target image.

[0062] exist Figure 3 Based on this, the embodiment of the present invention provides another feature extraction method, see Figure 4 FIG. 4 is a flow chart of another feature extraction method shown in FIG. 4 , wherein the method mainly includes the following steps S402 to S406:

[0063] Step S402, obtaining the original image, mask and object center point. The original image is also the target image, the mask is also the mask image, and the object center point is also the object center point. In one embodiment, target detection can be performed on the original image to determine the target object contained in the original image, and then the mask and object center point are determined according to the area where the target object is located in the original image.

[0064] Step S404, using the Attention mechanism, the convolution layer in the feature extraction network extracts features from the original image based on the object mask. The convolution layer can adopt a 3*3 convolution structure. Since the convolution layer in the feature extraction network has translation invariance, the Mask and the output feature map are bitwise multiplied to avoid the influence of the surrounding environment during the feature extraction process. In the specific implementation, the result of the bitwise multiplication of the original image and the mask is used as the input feature map of the first convolution layer, and the first convolution layer extracts features from the input feature map to obtain the output feature map of the first convolution layer; the result of the bitwise multiplication of the output feature map of the first convolution layer and the mask is used as the input feature map of the second convolution layer, and the second convolution layer extracts features from the input feature map to obtain the output feature map of the second convolution layer, and so on, until the output feature map of the convolution layer at the end of the feature extraction network is obtained.

[0065] Step S406, capture the object features corresponding to the center point of the object in the output feature map of the convolutional layer at the end of the feature extraction network to obtain the features of the original image. Each point on the output feature map can reflect the features of the target object. Therefore, by taking out the feature map of the center point of the object on the output feature map, the feature extraction of the object can be achieved, that is, the feature corresponding to the center point of the object in the output feature map is used as the feature of the target object corresponding to the center point of the object, such as Figure 3 As shown, it illustrates the extraction of features corresponding to the center points of objects in the output feature map to obtain object features corresponding to the center points of each object. For details, please refer to steps 2.1 to 2.3 provided in the aforementioned embodiment, and the embodiments of the present invention will not be repeated here.

[0066] In summary, the above-mentioned feature extraction method provided by the embodiment of the present invention first obtains the original image, mask and object center point, and then uses the Attention mechanism to extract features of the original image based on the object mask through the convolution layer in the feature extraction network, and captures the object features corresponding to the object center point in the output feature map of the convolution layer at the end of the feature extraction network to obtain the features of the original image. The above-mentioned method can extract features of the original image more quickly and accurately based on the mask and the object center point. The use of the mask can not only save the cutout operation before feature extraction and reduce the speed loss caused by the cutout operation, but also make the feature extraction network focus on the area of ​​interest in the target image, thereby effectively improving the accuracy of feature extraction.

[0067] Embodiment 4:

[0068] For the feature extraction method provided in the second embodiment, the present invention provides a feature extraction device, see Figure 5The structural diagram of a feature extraction device shown in FIG. 1 mainly includes the following parts:

[0069] The target image acquisition module 502 is used to acquire the target image whose features are to be extracted.

[0070] The mask image acquisition module 504 is used to acquire a mask image corresponding to the target image; wherein the mask image corresponds to the region of interest in the target image.

[0071] The feature extraction module 506 is used to input the target image and the mask image into the feature extraction network, and extract features of the target image based on the mask image through the feature extraction network.

[0072] The above-mentioned feature extraction device provided by the embodiment of the present invention uses a mask image to mark the region of interest in the target image, so that the feature extraction network can focus on the region of interest in the target image and perform feature extraction on the region of interest. Compared with the method of first cutting out the image and then extracting features in the prior art, the embodiment of the present invention can omit the cutting out process, thereby saving the time spent on cutting out before feature extraction, and is no longer restricted by the cutting out efficiency, thereby effectively improving the feature extraction efficiency.

[0073] In one embodiment, the feature extraction network includes a plurality of sequentially connected convolutional layers; the feature extraction module 506 is further used to: for each convolutional layer in the feature extraction network, determine the input feature map corresponding to the convolutional layer, and perform feature extraction on the input feature map through the convolutional layer to obtain the output feature map corresponding to the convolutional layer; wherein the input feature map corresponding to the first convolutional layer in the feature extraction network is related to the target image and the mask image, and the input feature map corresponding to the second convolutional layer in the feature extraction network is related to the output feature map of the previous convolutional layer and the mask image; and determine the features of the region of interest in the target image according to the output feature map of the feature extraction network.

[0074] In one embodiment, the feature extraction module 506 is also used for: when the convolution layer is the first convolution layer, bitwise multiplying the target image with the mask image to obtain an input feature map corresponding to the first convolution layer; when the convolution layer is the second convolution layer, bitwise multiplying the output feature map of the previous convolution layer corresponding to the second convolution layer with the mask image to obtain an input feature map corresponding to the second convolution layer.

[0075] In one embodiment, the region of interest includes at least one target object; the feature extraction module 506 is also used to: determine the target feature points of each target object in the target image; obtain the features corresponding to the target feature points of each target object in the output feature map of the feature extraction network; and determine the features corresponding to the target feature points of each target object as the features of the region of interest in the target image.

[0076] In one embodiment, the feature extraction module 506 is further used to: for each target feature point of the target object, perform bitwise multiplication of the output feature map of the feature extraction network with the target feature point of the target object to obtain the feature corresponding to the target feature point of the target object.

[0077] In one implementation, the feature extraction module 506 is further configured to: perform object detection on the target image to obtain each target object in the target image; and determine a target feature point of each target object.

[0078] In one embodiment, the mask image acquisition module 504 is further used to: establish an initial image according to the size of the target image; wherein the pixels in the initial image are of a first value; determine a region of interest in the target image; and adjust the pixels in the region of the initial image corresponding to the region of interest of the target image from the first value to the second value to obtain a mask image corresponding to the target image.

[0079] The implementation principle and technical effects of the device provided in this embodiment are the same as those of the aforementioned embodiments. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0080] The computer program products of the feature extraction method, device and electronic device provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0081] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0082] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0083] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0084] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A feature extraction method, characterized in that: include: Obtain a target image to be feature extracted; Acquire a mask image corresponding to the target image; wherein the mask image corresponds to a region of interest in the target image, and the region of interest is used to characterize a region where a target object is located in the target image; Inputting the target image and the mask image into a feature extraction network, and extracting features of the target image based on the mask image through the feature extraction network, wherein the input of the feature extraction network is the target image, the mask image and the center point of the object, and the output of the feature extraction network is a feature map corresponding to the target image; The region of interest includes at least one target object; and the step of determining the features of the region of interest in the target image according to the output feature map of the feature extraction network comprises: Determine the target feature points of each target object in the target image; obtain the features corresponding to the target feature points of each target object in the output feature map of the feature extraction network; determine the features corresponding to the target feature points of each target object as the features of the region of interest in the target image; The step of determining the target feature points of each target object in the target image comprises: performing object detection on the target image to obtain each target object in the target image, and for each target object, determining the center point of the target object, and using the center point of the target object as the target feature point of the target object; The step of obtaining the features corresponding to the target feature points of each target object in the output feature map of the feature extraction network includes: the output feature map is represented by a matrix, each element in the matrix represents the feature value of each point in the output feature map, and the features corresponding to the target feature points of the target object are obtained by capturing the feature value corresponding to the center point of the target object.

2. The method according to claim 1, characterized in that The feature extraction network includes a plurality of convolutional layers connected in sequence; The step of extracting features from the target image based on the mask image through the feature extraction network includes: For each of the convolutional layers in the feature extraction network, determine an input feature map corresponding to the convolutional layer, and perform feature extraction on the input feature map through the convolutional layer to obtain an output feature map corresponding to the convolutional layer; wherein the input feature map corresponding to the first convolutional layer in the feature extraction network is related to the target image and the mask image, and the input feature map corresponding to the second convolutional layer in the feature extraction network is related to the output feature map of the previous convolutional layer and the mask image; The features of the region of interest in the target image are determined according to the output feature map of the feature extraction network.

3. The method according to claim 2, characterized in that The step of determining the input feature map corresponding to the convolutional layer includes: When the convolution layer is the first convolution layer, the target image and the mask image are bitwise multiplied to obtain an input feature map corresponding to the first convolution layer; When the convolution layer is the second convolution layer, the output feature map of the previous convolution layer of the second convolution layer is bitwise multiplied with the mask image to obtain an input feature map corresponding to the second convolution layer.

4. The method according to claim 1, characterized in that: The step of obtaining features corresponding to target feature points of each target object in the output feature map of the feature extraction network comprises: For each target feature point of the target object, the output feature map of the feature extraction network is bitwise multiplied with the target feature point of the target object to obtain the feature corresponding to the target feature point of the target object.

5. The method according to claim 1, characterized in that The step of determining target feature points of each target object in the target image comprises: Performing object detection on the target image to obtain each target object in the target image; For each of the target objects, a target feature point of the target object is determined.

6. The method according to claim 1, characterized in that The step of acquiring the mask image corresponding to the target image comprises: Establishing an initial image according to the size of the target image; wherein the pixels in the initial image are first values; Determining a region of interest in the target image; The pixels in the area of ​​the initial image corresponding to the area of ​​interest of the target image are adjusted from the first value to a second value to obtain a mask image corresponding to the target image.

7. A feature extraction device, characterized in that: include: A target image acquisition module is used to acquire a target image to be extracted features; A mask image acquisition module, used to acquire a mask image corresponding to the target image; wherein the mask image corresponds to a region of interest in the target image, and the region of interest is used to characterize a region where a target object is located in the target image; A feature extraction module, used for inputting the target image and the mask image into a feature extraction network, and performing feature extraction on the target image based on the mask image through the feature extraction network, wherein the input of the feature extraction network is the target image, the mask image and the center point of the object, and the output of the feature extraction network is a feature map corresponding to the target image; The region of interest includes at least one target object; the feature extraction module is specifically used for: Determine the target feature points of each target object in the target image; obtain the features corresponding to the target feature points of each target object in the output feature map of the feature extraction network; determine the features corresponding to the target feature points of each target object as the features of the region of interest in the target image; The step of determining the target feature points of each target object in the target image comprises: performing object detection on the target image to obtain each target object in the target image, and for each target object, determining the center point of the target object, and using the center point of the target object as the target feature point of the target object; The step of obtaining the features corresponding to the target feature points of each target object in the output feature map of the feature extraction network includes: the output feature map is represented by a matrix, each element in the matrix represents the feature value of each point in the output feature map, and the features corresponding to the target feature points of the target object are obtained by capturing the feature value corresponding to the center point of the target object.

8. An electronic device, characterized in that: including a processor and a memory; A computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 6 is executed.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are executed.