Image classification method, device, equipment and storage medium

Abstract

The present invention discloses an image classification method, device, equipment and storage medium, and belongs to the field of image processing technology. The image classification method of the present invention comprises obtaining an image training set and a reference feature map; constructing an attention mechanism module according to the reference feature map; constructing an image classification model according to the attention mechanism module and a preset convolution module; training the image classification model using the image training set to obtain a trained image classification model; classifying and processing the target image according to the trained image classification model; wherein the attention mechanism module is an attention mechanism module based on local attention. This image classification method can more accurately identify the feature information of the image and improve the classification accuracy.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image classification method, device, equipment and storage medium. Background Art

[0002] At present, when classifying images, the classification of images is often achieved by identifying and analyzing the feature information of the images. However, the feature information of the images cannot be accurately identified, which affects the classification accuracy. Therefore, how to provide an image classification method to more accurately identify the feature information of the images and improve the classification accuracy has become an urgent problem to be solved. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes an image classification method, which can more accurately identify the feature information of the image and improve the classification accuracy.

[0004] The present invention also provides an image classification device having the above image classification method.

[0005] The present invention also provides an image classification device having the above image classification method.

[0006] The present invention also provides a computer-readable storage medium.

[0007] The image classification method according to the first aspect of the present invention includes:

[0008] Obtain image training set and reference feature map;

[0009] Constructing an attention mechanism module according to the reference feature map;

[0010] Constructing an image classification model according to the attention mechanism module and the preset convolution module;

[0011] Using the image training set to train the image classification model to obtain a trained image classification model;

[0012] Classify the target image according to the trained image classification model;

[0013] Among them, the attention mechanism module is an attention mechanism module based on local attention.

[0014] The image classification method according to the embodiment of the present invention has at least the following beneficial effects: this image classification method obtains an image training set and a reference feature map, and uses the reference feature map to construct an attention mechanism module based on local attention. Then, an image classification model is constructed according to the attention mechanism module based on local attention and a preset convolution module. The image classification model is trained using the image training set to obtain a trained image classification model, and finally, the target image is classified and processed according to the trained image classification model. This method can more accurately identify the local features of the image by adopting the attention mechanism module based on local attention, thereby improving the classification accuracy.

[0015] According to some embodiments of the present invention, constructing an attention mechanism module according to the reference feature map includes:

[0016] Inputting the reference feature map into the attention mechanism module, segmenting the reference feature map to obtain a local feature map;

[0017] Performing channel addition processing on the local feature map to obtain a fused feature map;

[0018] Compressing the fused feature map to obtain a global feature expression value of each local feature map;

[0019] According to the global feature expression value, obtaining the regional feature parameters of each local feature map;

[0020] Determining the relative weight of each local feature map according to the regional feature parameters;

[0021] Obtaining a weighted local feature map according to the relative weight and the local feature map;

[0022] All the weighted local feature maps are concatenated to generate a target feature map.

[0023] According to some embodiments of the present invention, the step of inputting the reference feature map into the attention mechanism module and performing segmentation processing on the reference feature map to obtain a local feature map includes:

[0024] The reference feature map is segmented according to a preset segmentation ratio and a preset segmentation direction to obtain a local feature map.

[0025] According to some embodiments of the present invention, compressing the fused feature map to obtain a global feature expression value of each local feature map includes:

[0026] At least one of global uniform pooling, global maximum pooling, and global variance pooling is used to compress the spatial domain information of the fused feature map to obtain a global feature expression value of each local feature map.

[0027] According to some embodiments of the present invention, obtaining the regional feature parameters of each local feature map according to the global feature expression value includes:

[0028] Encoding the global feature expression value according to at least one convolutional layer and at least one Relu function to obtain the global feature expression value in an encoded form;

[0029] Reconstructing the channel relationship between the local feature maps according to at least two convolutional layers to obtain a new channel relationship;

[0030] Feature extraction is performed on the global feature expression value in the encoded form according to the new channel relationship to obtain regional feature parameters of each local feature map.

[0031] According to some embodiments of the present invention, determining the relative weight of each local feature map according to the regional feature parameter includes:

[0032] The regional feature parameters are assigned values ​​according to a preset assignment function to obtain the relative weight of each local feature map.

[0033] According to some embodiments of the present invention, obtaining a weighted local feature map according to the relative weight and the local feature map includes:

[0034] Channel product processing is performed on each of the local feature maps and the relative weight corresponding to the local feature map to obtain a weighted local feature map.

[0035] An image classification device according to a second aspect of the present invention includes:

[0036] An acquisition module, used to acquire image training sets and reference feature maps;

[0037] An attention mechanism module construction module, used to construct an attention mechanism module according to the reference feature map;

[0038] An image classification model construction module, used to construct an image classification model according to the attention mechanism module and the preset convolution module;

[0039] An image classification model training module is used to train the image classification model using the image training set to obtain a trained image classification model;

[0040] An image classification module is used to classify the target image according to the trained image classification model;

[0041] Among them, the attention mechanism module is an attention mechanism module based on local attention.

[0042] The image classification device according to the embodiment of the present invention has at least the following beneficial effects: the image classification device obtains the image training set and the reference feature map through the acquisition module, and the attention mechanism module construction module uses the reference feature map to construct the attention mechanism module based on local attention. Then the image classification model construction module constructs the image classification model according to the attention mechanism module based on local attention and the preset convolution module. The image classification model training module trains the image classification model using the image training set to obtain the trained image classification model, and finally the image classification module classifies the target image according to the trained image classification model. The device can more accurately identify the local features of the image by adopting the attention mechanism module based on local attention, thereby improving the classification accuracy.

[0043] An image classification device according to a third aspect of the present invention includes:

[0044] at least one processor, and

[0045] a memory communicatively connected to the at least one processor; wherein,

[0046] The memory stores instructions, and the instructions are executed by the at least one processor so that the at least one processor implements the image classification method as described in the embodiment of the first aspect when executing the instructions.

[0047] The image classification device according to the embodiment of the present invention has at least the following beneficial effects: the image classification device adopts the above-mentioned image classification method, obtains the image training set and the reference feature map, and uses the reference feature map to construct an attention mechanism module based on local attention. Then, an image classification model is constructed according to the attention mechanism module based on local attention and a preset convolution module. The image classification model is trained using the image training set to obtain the trained image classification model, and finally the target image is classified and processed according to the trained image classification model. This method can more accurately identify the local features of the image by using the attention mechanism module based on local attention, thereby improving the classification accuracy.

[0048] According to the computer-readable storage medium of the fourth aspect embodiment of the present invention, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the image classification method as described in the first aspect embodiment.

[0049] The computer-readable storage medium according to the embodiment of the present invention has at least the following beneficial effects: the computer-readable storage medium executes the above-mentioned image classification method, obtains the image training set and the reference feature map, and uses the reference feature map to construct an attention mechanism module based on local attention. Then, an image classification model is constructed according to the attention mechanism module based on local attention and a preset convolution module. The image classification model is trained using the image training set to obtain a trained image classification model, and finally the target image is classified and processed according to the trained image classification model. This method can more accurately identify the local features of the image by adopting the attention mechanism module based on local attention, thereby improving the classification accuracy.

[0050] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments, wherein:

[0052] Figure 1 is a flow chart of an image classification method according to an embodiment of the present invention;

[0053] Figure 2 for Figure 1 Flow chart of step S102;

[0054] Figure 3 for Figure 1 Flow chart of step S204;

[0055] Figure 4 Schematic diagram of the structure of an image classification device according to an embodiment of the present invention.

[0056] Attached figure numbers: 401, acquisition module; 402, attention mechanism module construction module; 403, image classification model construction module; 404, image classification model training module; 405, image classification module. DETAILED DESCRIPTION

[0057] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0058] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0059] In the description of the present invention, "several" means more than one, "many" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0060] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0061] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0062] First, refer to Figure 1 , the image classification method of the embodiment of the present invention includes:

[0063] S101, obtaining an image training set and a reference feature map;

[0064] S102, constructing an attention mechanism module according to the reference feature map;

[0065] S103, constructing an image classification model according to the attention mechanism module and the preset convolution module;

[0066] S104, training the image classification model using the image training set to obtain a trained image classification model;

[0067] S105, classifying the target image according to the trained image classification model;

[0068] Among them, the attention mechanism module is an attention mechanism module based on local attention.

[0069] When classifying an image, first obtain an image training set and a reference feature map. Among them, the image training set can be composed of some pre-acquired image data, and the reference feature map R can be a series of pre-set feature maps. The size of this type of feature map is C×H×W, where C, H, and W refer to the number of channels, height, and width of the reference feature map, respectively. Then, use the reference feature map to construct an attention mechanism module based on local attention. This attention mechanism module based on local attention can more accurately identify the local features of the image. For example, by assigning weights to different local features of the image through the attention mechanism module based on local attention, the difference between the local features of the image can be made more significant, thereby obtaining the required image information. Then, an image classification model is constructed based on the attention mechanism module based on local attention and a preset convolution module. The image classification model is trained using the image training set to obtain a trained image classification model, and finally the target image is classified according to the trained image classification model. This method can more accurately identify the local features of the image and improve the classification accuracy by adopting the attention mechanism module based on local attention. Since in image classification, the target distribution and the distribution of target salient features of the image are regional, and the attention mechanism module in traditional technology can often only identify the global feature information of the image, the attention mechanism module of the present application is compared with the attention mechanism module in traditional technology. The attention mechanism module in the present application can solve the problem that traditional technology cannot effectively identify local features of the image. The attention mechanism module based on local attention can assign weights to different local features of the image, which can make the differences between the local features of the image more significant, thereby improving the model performance of the image classification model.

[0070] It should be noted that, according to actual needs, the above-mentioned attention mechanism module based on local attention can also be used together with other convolution modules to form other neural network models to achieve other computer vision tasks such as image segmentation, target detection or target positioning, which are not limited to the image classification tasks of this application. In addition, the attention mechanism module based on local attention of this application can also be embedded in various convolutional neural networks as the main module, so that the attention mechanism module based on local attention can be deployed to servers, workstations, supercomputing centers, cloud computing centers, personal computers, mobile phones, edge devices and medical equipment through various convolutional neural networks, and has good applicability.

[0071] Reference Figure 2 In some embodiments, step S102 includes:

[0072] S201, inputting the reference feature map into the attention mechanism module, segmenting the reference feature map to obtain a local feature map;

[0073] S202, performing channel addition processing on the local feature map to obtain a fused feature map;

[0074] S203, compressing the fused feature map to obtain a global feature expression value of each local feature map;

[0075] S204, obtaining regional feature parameters of each local feature map according to the global feature expression value;

[0076] S205, determining the relative weight of each local feature map according to the regional feature parameters;

[0077] S206, obtaining a weighted local feature map according to the relative weight and the local feature map;

[0078] S207, concatenating all weighted local feature maps to generate a target feature map.

[0079] When constructing the attention mechanism module according to the reference feature map, the reference feature map is first input into the attention mechanism module, and the reference feature map is segmented to obtain a local feature map. The segmentation of the reference feature map may be to segment the reference feature map according to a preset segmentation ratio and a preset segmentation direction. For example, the reference feature map is equally divided in the height direction to obtain two local feature maps of the same size, wherein the same size here means that the number of channels, width, and height of the two local feature maps are the same (that is, the size is C×H×W / 2). After the reference feature map is segmented to obtain the local feature map, the local feature map is subjected to channel addition processing to obtain a fused feature map. For example, the local feature map and local feature maps Perform channel addition processing to obtain the fused feature map U. Specifically, Where U(R C×W×H / 2) is the feature map after addition and fusion. It should be noted that, in some other embodiments, the local feature map can also be subjected to channel subtraction processing or splicing processing, etc. to obtain a fused feature map, which is not limited to this. Then, the fused feature map is compressed to obtain the global feature expression value of each local feature map. According to the global feature expression value, the regional feature parameters of each local feature map are obtained. Specifically, the channel relationship between each local feature map can be constructed according to multiple convolutional layers and ReLu functions. After the channel relationship is constructed, the regional feature parameters of each local feature map are obtained through the new channel relationship. Then, according to the regional feature parameters, each local feature map is assigned a value to determine the relative weight of each local feature map. By performing a product operation on each local feature map and the relative weight corresponding to the local feature map, a weighted local feature map can be obtained. Finally, all weighted local feature maps are spliced ​​to generate a target feature map. Through the above process, it is possible to construct an attention mechanism module based on local attention. By assigning weights to different local features of the image through the attention mechanism module based on local attention, the differences between the local features of the image can be made more significant, thereby obtaining the required image information and improving the accuracy of image classification.

[0080] In some embodiments, step S201 includes:

[0081] The reference feature map is segmented according to a preset segmentation ratio and a preset segmentation direction to obtain a local feature map.

[0082] When the reference feature map is segmented, the reference feature map can be segmented according to a preset segmentation ratio and a preset segmentation direction. For example, the preset segmentation ratio can be 1:1; it can also be 1:2:1; the preset segmentation direction can be the height direction (i.e., the y-axis direction) or the width direction (i.e., the x-axis direction), without limitation. For example, the reference feature map is equally divided in the height direction to obtain two local feature maps of the same size, where the same size here means that the number of channels, width, and height of the two local feature maps are the same (i.e., the size is C×H×W / 2).

[0083] In some embodiments, step S203 includes:

[0084] At least one of global uniform pooling, global maximum pooling, and global variance pooling is used to compress the spatial domain information of the fused feature map to obtain the global feature expression value of each local feature map.

[0085] When compressing the fused feature map, at least one of global uniform pooling, global maximum pooling, and global variance pooling can be used to compress the spatial information of the fused feature map. For example, the global average pooling (GAP) method is used to compress the fused feature map to obtain the global feature expression value z of each local feature map, where Assuming that the number of local feature maps in the entire convolutional layer is C, the global mean expression value of each local feature map is C×1×1. The process of compressing the spatial information of the fused feature map using global maximum pooling and global variance pooling is similar to the operation of global uniform pooling and will not be described in detail.

[0086] Reference Figure 3 In some embodiments, step S204 includes:

[0087] S301, encoding the global feature expression value according to at least one convolutional layer and at least one ReLU function to obtain the global feature expression value in an encoded form;

[0088] S302, reconstructing the channel relationship between the local feature maps according to at least two convolutional layers to obtain a new channel relationship;

[0089] S303, performing feature extraction on the global feature expression value in the encoded form according to the new channel relationship to obtain regional feature parameters of each local feature map.

[0090] When obtaining the regional feature parameters of each local feature map, the global feature expression value can be encoded according to at least one convolution layer and at least one Relu function to obtain the global feature expression value in encoded form. For example, a 1x1 convolution layer and a ReLu function are used to construct the channel relationship between the local feature maps to encode the global feature expression value to obtain the global feature expression value in encoded form, and then the channel relationship between the local feature maps is reconstructed according to at least two convolution layers to obtain a new channel relationship. For example, two 1x1 convolution layers are used to complete the channel relationship reconstruction. The regional feature parameters of each local feature map are extracted respectively through the new channel relationship (for example, after dividing the reference feature map into equal parts, the feature expression information A and B of different image regions are obtained through this step, where the feature vector is expressed as C×1×1). In this way, the regional feature parameters of each local feature map can be obtained more conveniently, simplifying the construction process.

[0091] In some embodiments, step S205 includes:

[0092] The regional feature parameters are assigned according to a preset assignment function to obtain the relative weight of each local feature map.

[0093] After obtaining the regional feature parameters of each local feature map, it is necessary to assign weights to different local features of the image through an attention mechanism module based on local attention. Specifically, the regional feature parameters can be assigned according to a preset assignment function to obtain the relative weight of each local feature map. It should be noted that the preset assignment function can be a softmax function that can perform a gating operator, or other functions, but is not limited to this. For example, the regional feature parameters of the local feature maps obtained by the aforementioned equal division are assigned respectively through the softmax function to determine the relative weights a and b of the two local feature maps, where, Among them A c ,B c are the eigenvalue weights after the cth channel reconstructs the local area of ​​the image. In this way, the difference between the local features of the image can be made more significant, so as to obtain the required image information.

[0094] In some embodiments, step S206 includes:

[0095] Channel product processing is performed on each local feature map and the relative weight corresponding to the local feature map to obtain a weighted local feature map.

[0096] When constructing an attention mechanism module based on local attention by referring to feature maps, it is also necessary to perform channel product processing on each local feature map and the relative weight corresponding to the local feature map, that is, to perform channel product processing on each local feature map and the relative weight corresponding to the local feature map, where Represents the channel product operation, thereby obtaining the weighted upper and lower feature maps and Finally, the local feature maps of the upper and lower parts are spliced ​​along the height direction and restored to the dimension C×H×W of the input reference feature map, realizing the construction of the attention mechanism module based on local attention. By assigning weights to different local features of the image through the attention mechanism module based on local attention, the difference between the local features of the image can be made more significant, thereby obtaining the required image information and improving the model performance of the image classification model. Classifying images through this image classification model can also improve the accuracy of image classification.

[0097] Second, refer to Figure 4 , the image classification device of the embodiment of the present invention comprises:

[0098] An acquisition module 401 is used to acquire an image training set and a reference feature map;

[0099] An attention mechanism module construction module 402 is used to construct an attention mechanism module according to a reference feature map;

[0100] An image classification model construction module 403 is used to construct an image classification model according to the attention mechanism module and the preset convolution module;

[0101] An image classification model training module 404 is used to train the image classification model using the image training set to obtain a trained image classification model;

[0102] An image classification module 405 is used to classify the target image according to the trained image classification model;

[0103] Among them, the attention mechanism module is an attention mechanism module based on local attention.

[0104] In a third aspect, an image classification device according to an embodiment of the present invention comprises at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions, and the instructions are executed by the at least one processor, so that when the at least one processor executes the instructions, an image classification method according to the embodiment of the first aspect is implemented.

[0105] In a fourth aspect, the present invention further provides a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the image classification method of the embodiment of the first aspect.

[0106] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in the relevant technical field without departing from the purpose of the present invention. In addition, the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

Claims

1. An image classification method, characterized in that: include: Obtain image training set and reference feature map; Constructing an attention mechanism module according to the reference feature map; Constructing an image classification model according to the attention mechanism module and the preset convolution module; Using the image training set to train the image classification model to obtain a trained image classification model; Classify the target image according to the trained image classification model; Wherein, the attention mechanism module is an attention mechanism module based on local attention; The step of constructing an attention mechanism module according to the reference feature map comprises: Inputting the reference feature map into the attention mechanism module, segmenting the reference feature map to obtain a local feature map; Performing channel addition processing on the local feature map to obtain a fused feature map; Compressing the fused feature map to obtain a global feature expression value of each local feature map; According to the global feature expression value, obtaining the regional feature parameters of each local feature map; Determining the relative weight of each local feature map according to the regional feature parameters; Obtaining a weighted local feature map according to the relative weight and the local feature map; All the weighted local feature maps are concatenated to generate a target feature map.

2. The image classification method according to claim 1, characterized in that: The step of inputting the reference feature map into the attention mechanism module and performing segmentation processing on the reference feature map to obtain a local feature map includes: The reference feature map is segmented according to a preset segmentation ratio and a preset segmentation direction to obtain a local feature map.

3. The image classification method according to claim 1, characterized in that: The compressing process of the fused feature map to obtain the global feature expression value of each local feature map includes: At least one of global uniform pooling, global maximum pooling, and global variance pooling is used to compress the spatial domain information of the fused feature map to obtain a global feature expression value of each local feature map.

4. The image classification method according to claim 1, characterized in that: The step of obtaining the regional feature parameters of each local feature map according to the global feature expression value includes: Encoding the global feature expression value according to at least one convolutional layer and at least one Relu function to obtain the global feature expression value in an encoded form; Reconstructing the channel relationship between the local feature maps according to at least two convolutional layers to obtain a new channel relationship; Feature extraction is performed on the global feature expression value in the encoded form according to the new channel relationship to obtain regional feature parameters of each local feature map.

5. The image classification method according to claim 1, characterized in that: Determining the relative weight of each local feature map according to the regional feature parameters includes: The regional feature parameters are assigned values ​​according to a preset assignment function to obtain the relative weight of each local feature map.

6. The image classification method according to claim 1, characterized in that: The step of obtaining a weighted local feature map according to the relative weight and the local feature map comprises: Channel product processing is performed on each of the local feature maps and the relative weight corresponding to the local feature map to obtain a weighted local feature map.

7. An image classification device, characterized in that include: An acquisition module, used to acquire image training sets and reference feature maps; An attention mechanism module construction module, used to construct an attention mechanism module according to the reference feature map; An image classification model construction module, used to construct an image classification model according to the attention mechanism module and the preset convolution module; An image classification model training module is used to train the image classification model using the image training set to obtain a trained image classification model; An image classification module is used to classify the target image according to the trained image classification model; Wherein, the attention mechanism module is an attention mechanism module based on local attention; The step of constructing an attention mechanism module according to the reference feature map comprises: Inputting the reference feature map into the attention mechanism module, segmenting the reference feature map to obtain a local feature map; Performing channel addition processing on the local feature map to obtain a fused feature map; Compressing the fused feature map to obtain a global feature expression value of each local feature map; According to the global feature expression value, obtaining the regional feature parameters of each local feature map; Determining the relative weight of each local feature map according to the regional feature parameters; Obtaining a weighted local feature map according to the relative weight and the local feature map; All the weighted local feature maps are concatenated to generate a target feature map.

8. An image classification device, characterized in that include: at least one processor, and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions, and the instructions are executed by the at least one processor, so that the at least one processor implements the image classification method according to any one of claims 1 to 6 when executing the instructions.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the image classification method according to any one of claims 1 to 6.

Images