Image classification method based on separable convolution block and spatial reduction attention mechanism

Abstract

The present application relates to a kind of image classification methods based on separable convolution block and spatial reduction attention mechanism, belong to image classification field.Crossover depth separable convolution and improved spatial reduction attention mechanism are added to PVT model, reduce model training time, and while reducing attention calculation, the original information of feature map is not lost basically at the same time amount of calculation.Crossover depth separable convolution block embedding and spatial reduction attention mechanism based network model is built, including block embedding module, linear projection module, position information embedding module and spatial reduction attention mechanism module;Image classification is carried out to improve the calculation rate when image classification and preserve original boundary information, so as to achieve the overall improvement effect.The present application effectively improves the problems such as huge model calculation and image boundary information loss, reduces the amount of calculation of model and improves the model classification performance.

Description

Technical Field

[0001] This invention belongs to the field of image classification and relates to an image classification method based on separable convolutional blocks and spatially reduced attention mechanism. Background Technology

[0002] Image classification, a key problem in computer vision, aims to enable computers to automatically and accurately categorize input images, thereby achieving understanding and analysis of image content. Traditional image classification methods typically rely on manually designed feature extraction algorithms, such as edge detection and color histograms. However, these methods have limited effectiveness for complex and varied image content. In recent years, the rise of deep learning technology has changed this landscape, particularly the widespread application of convolutional neural networks (CNNs), which allows computers to automatically learn high-level feature representations in images. This deep learning approach has achieved great success in image classification, performing exceptionally well not only in natural image classification but also finding widespread application in fields such as medical image analysis, autonomous driving, and security monitoring. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide an image classification method based on separable convolutional blocks and a spatially reduced attention mechanism. By improving the block embedding method and the attention mechanism, the model can reduce the amount of computation while maintaining high image classification performance. This alleviates the huge computational burden of image classification to a certain extent, making the image classification task easier to handle.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] An image classification method based on separable convolutional blocks and spatially reduced attention mechanism includes the following steps:

[0006] S1: Obtain the original dataset for image classification and divide it into a training sample set and a test sample set;

[0007] S2: Preprocess the above image classification training sample set to construct the training dataset;

[0008] S3: Build an image classification network model based on cross-depth separable convolutional block embedding and spatial reduction attention mechanism, including a feature extraction module and a spatial reduction attention mechanism module. Features are extracted through cross-depth separable convolutional block embedding and image information is extracted through spatial reduction attention mechanism.

[0009] S4: Using the training dataset described above, the image is divided into blocks by the described cross-depth separable convolutional block embeddings and trained through a spatial reduction attention mechanism to obtain the corresponding image classification model;

[0010] S5: Using the image classification model obtained in S4, perform inference and prediction on various types of images in the test sample set to obtain the final classification result.

[0011] Optionally, in S3, building an image classification network model based on cross-depth separable convolutional block embedding and spatially reduced attention mechanism specifically involves:

[0012] S311: For the input image, the image is divided into fixed-size patches using a cross-depth separable convolution operation, which reduces the computational cost of each convolution operation.

[0013] S312: Spatial Reduction Attention Mechanism Calculation: After the image undergoes cross-depth separable convolutional block embedding, it is followed by spatial reduction and normalization operations. Then, positional information is embedded into the data after these operations to facilitate image sequence searching; the spatial reduction attention mechanism is then calculated. For the input information:

[0014] SRA(Q,K,V)=Concat(head0,...head Ni W o

[0015] For input information with dimension C i The data is divided into N i There are 1 head, and the dimensions of each head are 1. W o To reduce the spatial projection parameters, the input data is mapped to a new feature space;

[0016]

[0017] After the head partitioning of the multi-head attention mechanism is completed, for the j-th head... j The difference between SR(Q) and SR(K) in computation and the traditional multi-head attention mechanism is that for the spatial reduction transformation of sequence Q and key K, SR(Q) and SR(K) are spatial scale downsampling operations on sequence Q and key K respectively. as well as These are the projection matrices of Q, K, and V, respectively; Attention(·) is the calculation of the attention mechanism, which... and The result of the multiplication is normalized, and then the result is multiplied by... Multiply the corresponding positions and sum them to get the head. j ;

[0018] Spatial scale downsampling operation SR(·):

[0019] SR(x) = Norm(Reshape(x,R) iW S )

[0020] R i The spatial downsampling factor is denoted by x, which is the input data for downsampling. During the calculation, x will be... Downsampling significantly reduces computational cost, thus speeding up calculations; the Reshape(·) operation reshapes the size of x for easier subsequent calculations; W S SR(·) is a spatial reduction projection operation, whose main purpose is to reduce the dimensionality of the input sequence. Norm(·) refers to Layer Normalization, which, specifically, calculates the mean and standard deviation of the feature dimension for a given input sample, and then normalizes all features so that their mean is 0 and their standard deviation is 1. The advantage of doing this is that it allows the network to maintain a consistent distribution characteristic for each input sample, reduces coupling between samples, and helps improve the generalization performance of the model. This is the detailed calculation process of SR(·).

[0021] Optionally, in S3, the specific construction process of the network model is as follows:

[0022] S321: Cross-depth separable convolution for feature extraction: The input image is divided into blocks by different channels through a cross-depth separable convolution operation;

[0023] S322: Location information embedding: First, the image block obtained in S321 is changed in dimension by Linear() operation, and then the image block is normalized by Norm() operation. The previous results are input into the CPE module for location information embedding, which facilitates the subsequent search of sequence information.

[0024] S323: Spatial Reduction Attention Mechanism Operation: For image patches with embedded location information, they are first backed up, then normalized by Norm(), and then enter the SRA() spatial reduction attention mechanism module. The calculated result is then compared with the backup data to perform a residual operation to preserve detail information and obtain the corresponding feature map.

[0025] S324: Forward propagation process: The output of S323 is first backed up, then subjected to a Norm() operation, and then put into the forward propagation module. The output result is summed with the backup data to obtain the final output result; this completes one stage of the network.

[0026] Optionally, the network model includes four types: Large, medium, small, and tiny.

[0027] The beneficial effects of this invention are as follows: This invention effectively improves the problems of huge model computation and loss of image boundary information, reduces the model computation and improves the model classification performance.

[0028] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0029] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0030] Figure 1 This is a schematic diagram of the overall architecture of the network model;

[0031] Figure 2 This is a schematic diagram of the stage module;

[0032] Figure 3 A schematic diagram of the attention mechanism provided by the present invention;

[0033] Figure 4 This is a schematic diagram of a cross-depth separable convolution provided by the present invention;

[0034] Figure 5 This is a schematic diagram of the cross-convolution process provided by the present invention. Detailed Implementation

[0035] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0036] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures, and should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0037] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0038] Step 1: Obtain the original dataset for image classification and divide it into a training sample set and a test sample set;

[0039] Step 2: Preprocess the above image classification training sample set to construct the training dataset;

[0040] like Figure 1 The diagram shown is the overall architecture of the network model in this example. The specific setup process is as follows:

[0041] The network construction consists of four stages. Each stage includes a feature extraction module and a spatial reduction attention mechanism module. The feature map output from the first stage is fed into the second stage to output the feature map of the second stage. Then it is fed into the third stage to output the feature map of the third stage. Finally, it is fed into the fourth stage to output the final classification result, thus completing the classification.

[0042] like Figure 2 As shown, this is the backbone module of the network model. The figure shows the contents of each stage module. For the input feature map, it first goes through a cross-depth separable convolution operation to divide the feature map into blocks. Then, a linear projection operation is performed on the block feature map to project it to a dimension that is easy to calculate. After that, a normalization operation is performed. The result of the normalization operation is input into the position encoding module to encode the position of each block to protect its sequence order.

[0043] X = CPE(PE(X) in ))

[0044] The results here are copied, and the original information is normalized again. The results are then input into the linear attention module to calculate attention, and the calculation results are compared with the copied data using a residual operation.

[0045] Y = SRA(Norm(X)) + X

[0046] Continue copying the results, normalize and forward propagate the original data, then perform residual operations with the copied data to obtain the output results. At this point, the stage module calculation is complete.

[0047] Z = FF(Norm(Y)) + Y

[0048] Step 3.1.2. Figure 3 As shown, this is the spatial reduction attention mechanism module. The spatial reduction attention mechanism calculation involves the following steps: After the image undergoes cross-depth separable convolutional block embedding, linear operations and normalization operations are performed. Then, positional information is embedded into the data after these operations to facilitate image sequence retrieval. After these operations are completed, the spatial reduction attention mechanism is calculated.

[0049] SRA(Q,K,V)=Concat(head0,...head Ni W o

[0050] For input information with dimension C i The data is divided into N i There are 1 head, and the dimensions of each head are 1. W o The linear projection parameters map the input data to a new feature space.

[0051] For input information with dimension C i The data is divided into N i There are 1 head, and the dimensions of each head are 1. W o The linear projection parameters map the input data to a new feature space.

[0052]

[0053] After the head partitioning of the multi-head attention mechanism is completed, for the j-th head... j The difference between this and the traditional multi-head attention mechanism in computation is that for the linear transformation of sequence Q and key K, SR(Q) and SR(K) are spatial scale downsampling operations on sequence Q and key K, respectively. as well as These are the projection matrices of Q, K, and V, respectively. Attention(·) is the calculation of the attention mechanism, which... and The result of the multiplication is normalized, and then the result is multiplied by... Multiply the corresponding positions and sum them to get the head. j ;

[0054] SR(x) = Norm(Reshape(x,R) i W S )

[0055] R i The spatial downsampling factor is denoted by x, which is the input data for downsampling. During the calculation, x will be... Downsampling significantly reduces computational cost, thus speeding up the calculation. The Reshape(·) operation reshapes the size of x to facilitate subsequent calculations. W S SR(·) is a linear projection operation, primarily used to reduce the dimensionality of the input sequence. Norm(·) refers to Layer Normalization. Specifically, for a given input sample, Layer Normalization calculates the mean and standard deviation of each feature dimension, then normalizes all features so that their mean is 0 and their standard deviation is 1. This ensures the network maintains a consistent distribution across all input samples, reducing coupling between samples and improving the model's generalization performance. The above describes the detailed calculation process of SR(·).

[0056] like Figure 4 The diagram shows a cross-depth separable convolution in this example. Cross-depth separable convolution consists of two steps: depthwise convolution and pointwise convolution. In the first step, for the input image, the number of convolution kernels is allocated according to its channels, that is, the number of channels is equal to the number of convolution kernels. Since this operation is performed independently for each channel of the input layer, it does not effectively utilize the feature information of different channels at the same spatial location. Therefore, pointwise convolution is needed to combine these feature maps to generate new feature maps.

[0057] Pointwise convolution is a convolution kernel with a size of 1x1 and the number of channels is the same as the number of channels in the output of the previous layer. For the first layer, the number of channels is the dimension of the image. The pointwise convolution operation will weight and combine the feature maps of the previous step in the channel direction to generate new feature maps.

[0058] like Figure 5 The diagram shown is a schematic of cross-convolution in this example. In order to extract the tile information without losing the boundary information between tiles, cross-convolution is used.

[0059] Depthwise separable convolutions can significantly reduce computation compared to normal convolutions. In this model, this greatly reduces the computational cost of segmenting the input image. A comparison of the computational cost of the two types of convolutions is as follows;

[0060] Next, we will analyze the computational performance of ordinary convolution and DSC. Assume the size of the input feature map is D. k ×Dk ×M, the kernel size is D f ×D f The number of convolutions is N, which is ×M. Assuming that a convolution operation is performed once for each point in the corresponding feature map spatial location, then a single convolution requires D convolutions. k ×D k ×D f ×D f ×M calculations. This is because the feature map space contains a total dimension of D. k ×D k There are 10 points, and the computational cost of performing a convolution operation on each point is the same as the size of the convolution kernel, i.e., D. f ×D f Therefore, for a single convolution, the total computational cost is ×M, which is:

[0061] (1)D k ×D k ×D f ×D f ×M

[0062] Therefore, for N convolutions, the total computational cost is:

[0063] (2)D k ×D k ×D f ×D f ×M×N

[0064] A similar analysis of DSC shows that the total computational amount of Depthwise Convolution is:

[0065] (3)D k ×D k ×D f ×D f ×M

[0066] The total computational cost of Pointwise Convolution is:

[0067] (4)M×N×D k ×D k

[0068] Therefore, the total calculation amount for DSC is:

[0069] (5)D k ×D k ×D f ×D f ×M+M×N×D k ×D k

[0070] Therefore, compared to ordinary convolution, the computational cost of DSC is:

[0071] (6)

[0072] As can be seen from the above, the computational efficiency of DSC is far superior to that of ordinary convolution.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An image classification method based on separable convolutional blocks and spatially reduced attention mechanism, characterized in that: The method includes the following steps: S1: Obtain the original dataset for image classification and divide it into a training sample set and a test sample set; S2: Preprocess the above image classification training sample set to construct the training dataset; S3: Build an image classification network model based on cross-depth separable convolutional block embedding and spatial reduction attention mechanism, including a feature extraction module and a spatial reduction attention mechanism module. Features are extracted through cross-depth separable convolutional block embedding and image information is extracted through spatial reduction attention mechanism. In step S3, the image classification network model based on cross-depth separable convolutional block embedding and spatial reduction attention mechanism is constructed as follows: S311: For the input image, the image is divided into fixed-size patches using a cross-depth separable convolution operation, which reduces the computational cost of each convolution operation. S312: Spatial Reduction Attention Mechanism Calculation: After the image undergoes cross-depth separable convolutional block embedding, it is followed by spatial reduction and normalization operations. Then, positional information is embedded into the data after these operations to facilitate image sequence searching; the spatial reduction attention mechanism is then calculated. For the input information: SRA(Q,K,V)=Concat(head0,...head Ni )W o For input information with dimension C i The data is divided into N i There are 1 head, and the dimensions of each head are 1. W o To reduce the spatial projection parameters, the input data is mapped to a new feature space; After the head partitioning of the multi-head attention mechanism is completed, for the j-th head... j The difference between SR(Q) and SR(K) in computation and the traditional multi-head attention mechanism is that for the spatial reduction transformation of sequence Q and key K, SR(Q) and SR(K) are spatial scale downsampling operations on sequence Q and key K respectively. as well as These are the projection matrices of Q, K, and V, respectively; Attention(·) is the calculation of the attention mechanism, which... and The result of the multiplication is normalized, and then the result is multiplied by... Multiply the corresponding positions and sum them to get the head. j ; Spatial scale downsampling operation SR(·): SR(x)=Norm(Reshape(x,R i )W S ) R i The spatial downsampling factor is denoted by x, which is the input data for downsampling. During the calculation, x will be... Downsampling significantly reduces computational cost, thus speeding up calculations; the Reshape(·) operation reshapes the size of x for easier subsequent calculations; W S SR(·) is a spatial reduction projection operation, whose main purpose is to reduce the dimensionality of the input sequence. Norm(·) refers to Layer Normalization, which, specifically, for an input sample, calculates the mean and standard deviation of the feature dimension, and then normalizes all features so that their mean is 0 and their standard deviation is 1. The advantage of doing this is that it allows the network to maintain a consistent distribution characteristic for each input sample, reduces coupling between samples, and helps improve the generalization performance of the model. This is the detailed calculation process of SR(·). S4: Using the training dataset described above, the image is divided into blocks by the described cross-depth separable convolutional block embeddings and trained through a spatial reduction attention mechanism to obtain the corresponding image classification model; S5: Using the image classification model obtained in S4, perform inference and prediction on various types of images in the test sample set to obtain the final classification result.

2. The image classification method based on separable convolutional blocks and spatially reduced attention mechanism according to claim 1, characterized in that: In S3, the specific construction process of the network model is as follows: S321: Cross-depth separable convolution for feature extraction: The input image is divided into blocks by different channels through a cross-depth separable convolution operation; S322: Location information embedding: First, the image block obtained in S321 is changed in dimension by Linear() operation, and then the image block is normalized by Norm() operation. The previous results are input into the CPE module for location information embedding, which facilitates the subsequent search of sequence information. S323: Spatial Reduction Attention Mechanism Operation: For image patches with embedded location information, they are first backed up, then normalized by Norm(), and then enter the SRA() spatial reduction attention mechanism module. The calculated result is then compared with the backup data to perform a residual operation to preserve detail information and obtain the corresponding feature map. S324: Forward propagation process: The output of S323 is first backed up, then subjected to a Norm() operation, and then put into the forward propagation module. The output result is summed with the backup data to obtain the final output result; this completes one stage of the network.

3. The image classification method based on separable convolutional blocks and spatially reduced attention mechanism according to claim 1, characterized in that: The network models include four types: Large, Medium, Small, and Tiny.

Images