Image classification model based on coordinate attention mechanism

Abstract

The invention relates to an image classification model based on a coordinate attention mechanism, and the model mainly comprises the following steps: inputting image data, and carrying out the image coding; the attention module processes the image feature information in the tensor form; image feature tensors are subjected to weight updating through normalization and nonlinear operation, image data are supplemented and classified and diagnosed through image convolution and deconvolution models in a YOLOV7 algorithm, training of an image classifier is completed, and an image classification model is generated. The provided attention module can push coding information into channel attention and manufacturing, so that excessive calculation overhead is avoided, and the defect of insufficient calculation power is overcome; a channel attention mechanism is divided into two parallel one-dimensional feature coding processes, spatial context information is effectively integrated into a generated attention map to capture direction sense and position sensitive information, the attention mechanism is more easily inserted into a traditional mobile network, and a target area can be more accurately positioned and recognized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence image classification, and in particular to the field of an image classification model based on a coordinate attention mechanism. Background Art

[0002] The attention mechanism can calculate the similarity between the query, key, and value of the same input sequence to determine the relative importance of each thing, especially for those inputs that contain a lot of redundant information; for image classification, the use of the attention mechanism can improve the generalization ability of the model, perform better when processing different types of images, and improve the robustness of the model. The attention mechanism can help the model automatically identify and focus on the most relevant features without manual intervention, so it is widely used in deep learning image classification. However, since the attention mechanism usually involves matrix multiplication and function calculation, it requires a lot of computing resources, which leads to the problem of insufficient computing power when the attention mechanism processes large amounts of data. Summary of the invention

[0003] In view of this, the present invention provides an image classification model based on a coordinate attention mechanism, which effectively solves the technical problem of insufficient computing power when using the attention mechanism during image classification.

[0004] To achieve the above object, the present invention provides an image classification model based on a coordinate attention mechanism, comprising the following steps:

[0005] S1. Input image data, perform image encoding, and store image feature data information in the form of tensors;

[0006] S2, the attention module processes the image feature information in the form of tensors, calculates the channel attention through two-dimensional global pooling, and pushes the position encoding information into the channel attention module through coordinate information embedding and coordinate attention generation to obtain the processed image feature tensor;

[0007] S3, the image feature tensor is normalized and nonlinearly divided into two groups for convolution and activation respectively, the attention weight of each channel is calculated through the neural network, the attention weight is multiplied by the channel feature to realize the channel attention calculation, the image feature tensor and the processed feature tensor are weighted and the result is output;

[0008] S4. Use the image convolution and deconvolution models in the YOLOV7 algorithm to supplement and classify image data, complete the training of the image classifier and generate an image classification model.

[0009] As a further improvement of the above solution:

[0010] Preferably, the input vector image data is converted into an RGB three-channel tensor by encoding.

[0011] Preferably, the two-dimensional global pooling is performed by performing global average pooling on each channel in the spatial dimension to obtain a global feature representation, and the attention module receives the image feature tensor As input, it outputs a transformed tensor Y=[y1,y2,...y C ], where C is the number of channels of the image tensor, H is the height, and W is the width. The attention module includes two steps: squeezing and excitation; the squeezing is to compress the information of each channel into a single value. For a given tensor X, the squeezing step calculation formula for the cth channel is as follows:

[0012]

[0013] Among them, z c Represents the output information associated with output channel c, i represents, j represents;

[0014] The incentive is used to fully capture the channel dependency and is calculated as follows:

[0015]

[0016] in, is the tensor obtained after excitation, σ is an s-type function, · represents channel multiplication, is the result generated by the transformation function, ReLU is a nonlinear activation function, and T1 and T2 are two linear transformations.

[0017] Preferably, accurate position information is used to represent channel relationships and long-term dependencies based on the attention module, including coordinate information embedding and coordinate attention generation;

[0018] The embedding of coordinate information comprises the following steps:

[0019] S301, calculate the z by global pooling c The expression is decomposed into two one-dimensional feature encoding tensors;

[0020] S302. For a given input X, use two different pooling kernels (H, 1) or (1, W) on each channel to encode each channel in horizontal and vertical coordinates;

[0021] S303, the output of the cth channel at height h can be expressed as follows:

[0022]

[0023] S304, the output of the cth channel at a width of w can be expressed as follows:

[0024]

[0025] The coordinate attention generation comprises the following steps:

[0026] S311, when given by and When the aggregate feature map is generated, all the aggregate feature maps are concatenated and sent to a shared 1×1 convolution transformation function named F1 for further processing, resulting in:

[0027] f=δ(F1([z h ,z w ]))

[0028] S312, using a concatenation operation [·,·] to connect the aggregated feature graphs in a spatial dimension;

[0029] S313, using a nonlinear activation function δ to process the connected feature map;

[0030] S314, using an intermediate feature map Encode spatial information and adjust the block size by controlling the reduction ratio r;

[0031] S315, split the feature map f into two independent tensors along the spatial dimension and

[0032] S316, use two 1×1 convolution transformations F h and F w f h and f w Transformed into a tensor with the same number of channels as the input X, we get:

[0033] g h =σ(F h (f h ))

[0034] g w =σ(F w (f w ))

[0035] S317, expand the output g h and g w Used as attention weight, output coordinate attention block Y, the specific form is:

[0036]

[0037] Among them, yc (i,j) represents.

[0038] Preferably, the steps of supplementing and classifying image data using the image convolution and deconvolution models in the YOLOV7 algorithm are as follows: open the YOLOV7 yaml configuration file, add a CA layer before the output layer, determine the number of inputs and outputs, modify the yaml configuration file to use the image convolution and deconvolution model to achieve image data supplementation and classification diagnosis, use the original data and supplementary data to jointly complete the training and testing of the image classification model, and output the image classification model.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. The attention module provided by the present invention utilizes the input position information weight to distinguish the importance of different positions and effectively utilizes the input spatial information, thereby reducing the number of parameters that the model needs to process, thereby reducing the computational complexity and solving the problem of insufficient computing power of the image classification model based on the attention mechanism.

[0041] 2. Channel attention is calculated through two-dimensional global pooling, which improves model performance while having relatively low computational cost. At the same time, in order to avoid the loss of position information that may be caused by two-dimensional global pooling, the channel attention mechanism is divided into two parallel one-dimensional feature encoding processes, which effectively integrates spatial context information into the produced attention map, captures directional and position-sensitive information, and makes the coordinate attention mechanism flexible and lightweight, and easy to insert into existing classic mobile networks to improve feature representation performance.

[0042] 3. Push the position encoding information into the channel attention mechanism to help the mobile network focus on a large range, avoid excessive computational overhead, and reduce the risk of insufficient model computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the process of the present invention;

[0044] Figure 2 It is the flow chart of YOLOV7 algorithm of the present invention;

[0045] Figure 3 Schematic diagram comparing the coordinate attention mechanism proposed in the present invention with the traditional attention mechanism. DETAILED DESCRIPTION

[0046] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0047] The present invention provides the following technical solutions:

[0048] An image classification model based on coordinate attention mechanism includes the following steps:

[0049] S1. Input image data, perform image encoding, and store image feature data information in the form of tensors;

[0050] S2, the attention module processes the image feature information in the form of tensors, calculates the channel attention through two-dimensional global pooling, and pushes the position encoding information into the channel attention module through coordinate information embedding and coordinate attention generation to obtain the processed image feature tensor;

[0051] S3, the image feature tensor is normalized and nonlinearly divided into two groups for convolution and activation respectively, the attention weight of each channel is calculated through the neural network, the attention weight is multiplied by the channel feature to realize the channel attention calculation, the image feature tensor and the processed feature tensor are weighted and the result is output;

[0052] S4. Use the image convolution and deconvolution models in the YOLOV7 algorithm to supplement and classify image data, complete the training of the image classifier and generate an image classification model.

[0053] First, read the image data in JPG format, convert the image data into three-dimensional NumPy array information through Python, and then encode it into RGB three-channel tensor form.

[0054] Two-dimensional global pooling is to obtain global feature representation by performing global average pooling on each channel in the spatial dimension. The attention module is a computing unit used to improve the ability of mobile network to learn feature expression. The attention module receives the image feature tensor As input, it outputs a transformed tensor Y=[y1,y2,...y C ] is used to enhance the representation ability of X, where C is the number of channels of the image tensor, H is the height, and W is the width; traditional standard convolution is difficult to simulate the relationship between channels, so it is necessary to explicitly construct the dependency between channels to improve the model's perception of information channels, which play an important role in the final classification decision. At the same time, global averaging can help the model obtain global information and compensate for the shortcomings of convolution.

[0055] The attention module includes two steps, squeeze and excitation, which are used to adaptively re-teach the global and channel relationships; squeeze is to compress the information of each channel into a single value to facilitate the merging of global information and the extraction of overall features. For a given tensor X, the squeeze step calculation formula for the cth channel is as follows:

[0056]

[0057] Among them, z c Represents the output information associated with output channel c, i represents, j represents;

[0058] The incentive is used to fully capture the channel dependency and is calculated as follows:

[0059]

[0060] in, is the tensor obtained after excitation, σ is an s-type function, · represents channel multiplication, is the result generated by the transformation function, ReLU is a nonlinear activation function, and T1 and T2 are two linear transformations.

[0061] Global pooling is mainly used to compress and encode input data in the channel dimension. However, global pooling ignores spatial information and cannot save the position information of input data. Therefore, precise position information is used to represent channel relationships and long-term dependencies based on the attention module, including coordinate information embedding and coordinate attention generation.

[0062] Coordinate information embedding includes the following steps:

[0063] S301, calculate the z by global pooling c The expression is decomposed into two one-dimensional feature encoding tensors;

[0064] S302. For a given input X, use two different pooling kernels (H, 1) or (1, W) on each channel to encode each channel in horizontal and vertical coordinates;

[0065] S303, the output of the cth channel at height h can be expressed as follows:

[0066]

[0067] S304, the output of the cth channel at a width of w can be expressed as follows:

[0068]

[0069] S303 and S304 describe two different transformation modes in the attention mechanism, which focus on different spatial directions to generate feature maps with direction perception, and can provide a global receptive field, which can help the attention module capture long-term dependencies at longer distances and maintain precise positional relationships at the same time, so that the network can locate the target of interest more accurately.

[0070] For applications in mobile environments, it is necessary to consider cost and complexity issues and make full use of direction encoding information to highlight. In addition, it is also necessary to effectively capture the relationship between channels; coordinate attention generation includes the following steps:

[0071] S311, when given by and When the aggregate feature maps are generated, all the aggregate feature maps are connected together and sent to a shared 1×1 convolution transformation function called F1 for further processing, resulting in:

[0072] f=δ(F1([z h ,z w ]))

[0073] S312, using a concatenation operation [·,·] to connect the aggregated feature graphs in a spatial dimension;

[0074] S313, using a nonlinear activation function δ to process the connected feature map;

[0075] S314, using an intermediate feature map Encode spatial information and adjust the block size by controlling the reduction ratio r;

[0076] S315, split the feature map f into two independent tensors along the spatial dimension and

[0077] S316, use two 1×1 convolution transformations F h and F w f h and f w Transformed into a tensor with the same number of channels as the input X, we get:

[0078] g h =σ(F h (f h ))

[0079] g w =σ(F w (f w ))

[0080] σ is an s-shaped function, and usually the number of channels f is reduced by appropriately reducing the ratio r, thereby reducing the cost of model complexity;

[0081] S317, expand the output g h and g w Used as attention weight, output coordinate attention block Y, the specific form is:

[0082]

[0083] Among them, y c (i,j) indicates;

[0084] The coordinate attention module applies attention in both horizontal and vertical directions to the input tensor simultaneously. Each element in each attention map indicates whether the object of interest exists in the corresponding row and column. Through this encoding method, the position of the object of interest can be located more accurately, which helps the entire model to better identify tasks.

[0085] The steps of using the image convolution and deconvolution models in the YOLOV7 algorithm to supplement and classify image data are as follows: open the YOLOV7 yaml configuration file, add a CA layer before the output layer, determine the number of inputs and outputs, modify the yaml configuration file to use the image convolution and deconvolution model to supplement and classify image data, use the original data and supplementary data to complete the training and testing of the image classification model, and output the image classification model.

[0086] The algorithm flow of the coordinate attention mechanism is as follows:

[0087]

[0088]

[0089] In the biological aeration tank image classification model, the running data Q represents the overall biological aeration tank data set, x represents a single biological aeration tank feature data, g represents a counter, C represents the number of image channels, H represents the image height, and W represents the image width; the algorithm flow 2) to 9) steps belong to the data preprocessing process, which is to process the h image feature vectors and generate the parameters used for training; then, the industrial data is convolutionally trained according to the image data processing method, and the data enhancement module is used to supplement the data to improve the final state diagnosis effect. The advantage of the state diagnosis scheme based on the coordinate attention mechanism is that it can enhance the model's sensitivity to state information.

[0090] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. An image classification model based on coordinate attention mechanism, characterized in that: The following steps are involved: S1. Input image data, perform image encoding, and store image feature data information in the form of tensors; S2, the attention module processes the image feature information in the form of tensors, calculates the channel attention through two-dimensional global pooling, and pushes the position encoding information into the channel attention module through coordinate information embedding and coordinate attention generation to obtain the processed image feature tensor; S3, the image feature tensor is normalized and nonlinearly divided into two groups for convolution and activation respectively, the attention weight of each channel is calculated through the neural network, the attention weight is multiplied by the channel feature to realize the channel attention calculation, the image feature tensor and the processed feature tensor are weighted and the result is output; S4. Use the image convolution and deconvolution models in the YOLOV7 algorithm to supplement and classify image data, complete the training of the image classifier and generate an image classification model.

2. The image classification model based on coordinate attention mechanism according to claim 1, characterized in that: The input vector image data is converted into an RGB three-channel tensor through encoding.

3. The image classification model based on coordinate attention mechanism according to claim 2, characterized in that: The two-dimensional global pooling is performed by performing global average pooling on each channel in the spatial dimension to obtain a global feature representation. The attention module receives the image feature tensor As input, it outputs a transformed tensor Y=[y1,y2,...y C ], where C is the number of channels of the image tensor, H is the height, and W is the width. The attention module includes two steps: squeezing and excitation; the squeezing is to compress the information of each channel into a single value. For a given tensor X, the squeezing step calculation formula for the cth channel is as follows: Among them, z c Represents the output information associated with output channel c, i represents, j represents; The incentive is used to fully capture the channel dependency and is calculated as follows: in, is the tensor obtained after excitation, σ is an s-type function, · represents channel multiplication, is the result generated by the transformation function, ReLU is a nonlinear activation function, and T1 and T2 are two linear transformations.

4. The image classification model based on coordinate attention mechanism according to claim 3, characterized in that: Based on the attention module, accurate position information is used to represent channel relationships and long-term dependencies, including coordinate information embedding and coordinate attention generation; The coordinate information embedding comprises the following steps: S301, calculate the z by global pooling c The expression is decomposed into two one-dimensional feature encoding tensors; S302, for a given input X, use two different pooling kernels (H, 1) or (1, W) on each channel to encode each channel in horizontal and vertical coordinates; S303, the output of the cth channel at height h can be expressed as follows: S304, the output of the cth channel at a width of w can be expressed as follows: The coordinate attention generation comprises the following steps: S311, when given by and When the aggregate feature map is generated, all the aggregate feature maps are concatenated and sent to a shared 1×1 convolution transformation function named F1 for further processing, resulting in: S312, using a concatenation operation [·,·] to connect the aggregated feature graphs in a spatial dimension; S313, using a nonlinear activation function δ to process the connected feature map; S314, using an intermediate feature map Encode spatial information and adjust the block size by controlling the reduction ratio r; S315, split the feature map f into two independent tensors along the spatial dimension and S316, use two 1×1 convolution transformations F h and F w f h and f w Transformed into a tensor with the same number of channels as the input X, we get: g h =σ(F h (f h )) g w =σ(F w (f w )) S317, expand the output g h and g w Used as attention weight, output coordinate attention block Y, the specific form is: Among them, y c (i,j) represents.

5. The image classification model based on coordinate attention mechanism according to claim 4, characterized in that: The steps of using the image convolution and deconvolution models in the YOLOV7 algorithm to supplement and classify image data are as follows: open the YOLOV7 yaml configuration file, add a CA layer before the output layer, determine the number of inputs and outputs, modify the yaml configuration file to use the image convolution and deconvolution model to supplement and classify image data, use the original data and supplementary data to complete the training and testing of the image classification model, and output the image classification model.

Images