An image classification method and system based on lightweight de-redundant convolution and improved transformer

Abstract

The application discloses an image classification method and system based on a light de-redundancy convolution and an improved Transform, and belongs to the technical field of image processing.The application comprises the following steps: acquiring a classification image dataset; dividing the classification image dataset into a training set and a test set and performing data enhancement; constructing an image classification model of the light de-redundancy convolution and the improved Transform and adopting a training mode of transfer learning, using the training set to optimize parameters to obtain an optimal model; and loading the trained optimal model, inputting an image to be classified to obtain a classification result.The application performs feature screening on images with similar backgrounds and small target areas by constructing a light de-redundancy convolution, removes redundant features, and obtains robust feature representation; meanwhile, a double-flow mode combining a convolutional neural network and an improved Transform is adopted to strengthen the inductive bias ability and long-distance dependence capturing ability of the model, effectively improving the quality of the final result of image classification, and having excellent characteristics of strong applicability and high classification performance.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to an image classification method and system based on lightweight deredundant convolution and an improved Transformer. Background Technology

[0002] With the continuous development of artificial intelligence technology, image classification plays an important role in various application scenarios, such as autonomous driving, medical diagnosis, and security monitoring. While traditional convolutional neural networks perform well in image classification tasks, they primarily rely on local convolution operations, thus performing poorly in capturing the dependencies between distant pixels in an image. Existing image classification algorithms also struggle to effectively extract features from data with similar backgrounds and small target regions, resulting in poor classification performance.

[0003] A search revealed patent number CN118247561A, published on June 25, 2024, entitled "A Method and Apparatus for Image Classification of New Energy Battery Packs Based on Multi-Scale Attention Mechanism." This application improves upon traditional CNN-based image classification methods by using a CNN+Transformer architecture with a multi-scale attention mechanism, enhancing the image classification accuracy of new energy battery packs when the feature differences between categories are too small. The MSNET network can extract local information at different scales and integrate local structural information into global contextual information, improving feature recognition capabilities and alleviating the problem of insufficient global contextual information. Furthermore, to address the issues of large parameter count and high computational complexity, this application proposes introducing depthwise separable convolution into the MSNET network, performing self-attention in the channel dimension rather than the spatial dimension, significantly reducing convolution parameters and computational load.

[0004] The aforementioned applications focus on solving the problems of small differences in target regions, insufficient global context information, and large number of network parameters. These are similar to the problems that this patent aims to solve. However, the aforementioned applications mainly solve these problems by constructing a CNN+Transformer architecture based on a multi-scale attention mechanism, which differs from the solution design of this patent. Summary of the Invention

[0005] 1. The technical problem that the invention aims to solve

[0006] In view of the problems existing in the prior art, such as the inability of traditional classification algorithms to handle images with similar backgrounds and small target regions well, and the poor performance of classification results obtained by ignoring global contextual information of the image, this invention provides an image classification method and system based on lightweight deredundant convolution and an improved Transformer. The classification method provided by this invention can effectively improve the quality of the final image classification results.

[0007] 2. Technical Solution

[0008] To achieve the above objectives, the technical solution provided by the present invention is as follows:

[0009] The present invention discloses an image classification method based on lightweight deredundancy convolution and an improved Transformer, the steps of which are as follows:

[0010] Step 101: Obtain a classification image dataset, which must contain at least two classes of data;

[0011] Step 102: Divide the classified image data into training and testing sets, and perform data augmentation processing;

[0012] Step 103: Construct a lightweight, redundancy-removing convolutional and improved Transformer image classification model, and optimize and train the image classification model;

[0013] Step 104: Load and save the trained optimal model, input the image to be classified, and obtain the classification result of the image data.

[0014] Furthermore, the data augmentation operations used in step 102 include random rotation, random cropping, random flipping, and color transformation.

[0015] Furthermore, the image classification model constructed in step 103 includes four stages, which are connected in series. Stage 1 includes a feature extraction head Stem, a lightweight de-redundancy convolutional module SC Block, and an improved Transformer module LTA Block. Stages 2, 3, and 4 have the same structure, each including a downsampling module DS Block, a lightweight de-redundancy convolutional module SC Block, and an improved Transformer module LTA Block.

[0016] Furthermore, after the data is input into the image classification model, it is first subjected to preliminary feature extraction by the feature extraction head Stem, and then fed into the lightweight deredundancy convolution module SC Block and the improved Transformer module LTA Block for parallel processing, forming a dual-stream feature extraction channel before feature fusion.

[0017] Furthermore, the data processed by Stage 1 is sequentially processed by Stage 2, Stage 3, and Stage 4. Stage 2, Stage 3, and Stage 4 repeat the same processing procedure, specifically: first, the data is downsampled by the downsampling module DS Block, and then subjected to dual-stream feature extraction by the lightweight redundancy removal convolution module SC Block and the improved Transformer module LTA Block to extract deep abstract features.

[0018] Furthermore, the feature extraction head Stem consists of convolutional and normalization layers with a kernel size of 4 and a stride of 4.

[0019] Furthermore, the data processing procedure of the lightweight de-redundancy convolution module SC Block is as follows: the data first enters the de-redundancy convolution SCS, then passes through the depthwise separable convolution DWC with a kernel size of 7, a stride of 1, and padding of 3, and the layer normalization LN, then through the convolution Conv with a kernel size of 1 and a stride of 1, and the convolution Conv with a kernel size of 7, a stride of 1, and padding of 3, and the layer normalization LN, and finally inputs the Drop Path to prevent overfitting.

[0020] Furthermore, the data processing procedure of the improved Transformer module LTA Block is as follows: the input features are first subjected to convolutions with kernel sizes of 3 and 1 for preliminary feature extraction, then the global context information is modeled by the Transformer module for global feature learning, and then the number of channels and the feature map size are adjusted by convolutions with kernel sizes of 1 and 3 in sequence. Finally, the edge features are strengthened by the local channel attention interaction mechanism.

[0021] Furthermore, step 103 employs transfer learning to accelerate model convergence and uses a label-smoothed cross-entropy loss function for loss calculation.

[0022] This invention provides an image classification system based on lightweight deredundant convolution and an improved Transformer. The system performs image classification using the aforementioned method, including:

[0023] The data acquisition module is used to acquire categorized image datasets;

[0024] The data processing module is used to divide the image data into training and testing sets according to a specified ratio, and to perform data augmentation operations to preprocess the image data;

[0025] The model training module uses the training set to train the parameters of the predefined model, and obtains and saves the optimal model weights.

[0026] The output module is used to call the trained optimal model, input the image to be classified, and obtain the final high-precision classification result.

[0027] 3. Beneficial effects

[0028] Compared with existing known technologies, the technical solution provided by this invention has the following significant advantages:

[0029] The present invention provides an image classification method based on lightweight deredundant convolution and an improved Transformer. First, data augmentation methods are used to enable the model to better learn the essential features of the data during training, thereby performing better when faced with test data that was not used in training. This helps to reduce overfitting and improve the model's generalization ability.

[0030] Next, by proposing a lightweight deredundancy convolution module, redundancy is simultaneously filtered at both the spatial and channel scales, and excessive information loss is avoided through residual connections. This solves the problem of interference with classification results caused by small target regions and similar backgrounds in images, making the extracted features more representative.

[0031] Finally, the proposed improved Transformer module enhances the model's attention to global contextual information through a local-global-local approach. The self-attention mechanism effectively captures the relationships between distant elements in the sequence, while channel attention enables the model to better distinguish target edge features. After initial downsampling, the data is processed in parallel by the improved Transformer module and a lightweight de-redundancy convolution module, and then fused to form a dual-stream feature extraction channel. This approach extracts features more fully and improves the model's robustness. It effectively improves the quality of the final image classification results, exhibiting excellent characteristics of strong applicability and high classification performance. Attached Figure Description

[0032] Figure 1 This is a flowchart of the image classification method based on lightweight deredundancy convolution and improved Transformer in Embodiment 1 of the present invention;

[0033] Figure 2 This is a schematic diagram of the model structure of the image classification method based on lightweight de-redundant convolution and improved Transformer in Embodiment 1 of the present invention;

[0034] Figure 3 This is a schematic diagram of the improved Transformer (LTA Block) structure in the image classification method based on lightweight deredundancy convolution and improved Transformer in Embodiment 1 of the present invention;

[0035] Figure 4This is a schematic diagram of the image classification system based on lightweight deredundancy convolution and improved Transformer in Embodiment 2 of the present invention. Detailed Implementation

[0036] To further understand the content of this invention, a detailed description of the invention will be provided in conjunction with the accompanying drawings and embodiments.

[0037] Example 1

[0038] like Figure 1 As shown, this embodiment provides an image classification method based on lightweight deredundancy convolution and an improved Transformer, including the following steps:

[0039] Step 101: Obtain the classification image dataset, which contains at least two classes of data.

[0040] Step 102: Divide the classification image data into training and test sets, and perform data augmentation. First, divide the dataset into training and test sets in an 8:2 ratio. Then, perform data augmentation operations on the training set, such as random rotation, random cropping, random flipping, and color transformation, to reduce the risk of model overfitting.

[0041] Step 103: Design and build a lightweight, de-redundant convolutional and improved Transformer image classification model, use transfer learning for training, and optimize the parameters using the training set to obtain the optimal model.

[0042] like Figure 2 As shown, the model classification designed and constructed in this invention includes four stages. Stage 1 consists of a feature extraction head (Stem), a lightweight redundancy-removing convolutional module (SC Block), and an improved Transformer module (LTA Block). Stages 2, 3, and 4 each consist of a downsampling module (DS Block), a lightweight redundancy-removing convolutional module (SC Block), and an improved Transformer module (LTAB Block). After the data is input into the network, it is first processed by the feature extraction head (Stem), which consists of convolutional layers with a kernel size of 4 and a stride of 4, and a normalization layer, to perform preliminary feature extraction and reduce computational cost. Then, it is fed into the lightweight redundancy-removing convolutional module (SC Block) and the improved Transformer module (LTA Block) for parallel processing, forming a dual-stream feature extraction channel, which is then fused. The data then passes through the serially connected Stages 2, 3, and 4, repeating the downsampling process performed by the downsampling module (DS Block) and the dual-stream feature extraction process performed by the lightweight redundancy-removing convolutional module (SC Block) and the improved Transformer module (LTA Block) to extract deep abstract features. Finally, after normalization and linear mapping by the classification layer, the classification category is output.

[0043] The initial layers capture simple features, such as edges and corners, while deeper layers combine these simple features to form more complex abstract representations. This layer-by-layer processing enables the network to deeply understand the structure and patterns of the data. In this multi-layered structure, the lightweight de-redundant convolutional module (SC Block) improves the efficiency of extracting effective features and avoids redundant information affecting the final classification accuracy. The improved Transformer module (LTA Block) uses a local-global-local approach to effectively extract global information from small target regions. This multi-layered structure helps the model capture deep semantic features of images, effectively improving image classification performance. Figure 2 , Figure 2 In this context, K represents the kernel size, S represents the stride of the convolution operation, and P represents the padding during the convolution process.

[0044] The data transmission process via the Lightweight Redundancy Removal Convolutional Module (SC Block) is as follows: First, the data enters the Redundancy Removal Convolutional Module (SCS), then passes through a Depthwise Separable Convolutional Module (DWC) with a kernel size of 7, a stride of 1, and padding of 3, followed by Layer Normalization (LN). Next, it passes through a Convolutional Module (Conv) with a kernel size of 1 and a stride of 1, the GELU activation function, and another Convolutional Module (Conv) with a kernel size of 7, a stride of 1, and padding of 3, followed by Layer Normalization (LN). Finally, it is input into the Drop Path to prevent overfitting. The SCS convolution removes redundancy for images with small target regions and similar backgrounds, obtaining representative features. Compared to ordinary convolution, the Depthwise Separable Convolutional Module (DWC) has fewer parameters, and its convolutional feature extraction performance is basically on par with traditional convolution, effectively improving the model's inference speed and reducing computational requirements.

[0045] like Figure 3 As shown, the improved Transformer module LTA Block consists of two convolutional layers (Conv) with kernel sizes of 3 and 1, a Transformer for global feature learning, and a local channel attention interaction mechanism. Input features are first extracted through convolutions with kernel sizes of 3 and 1. Then, the Transformer module for global feature learning models global contextual information. Next, convolutions with kernel sizes of 1 and 3 are used to adjust the number of channels and feature map size. Finally, the local channel attention interaction mechanism enhances edge features. This local-global-local approach fully extracts image feature information while possessing the inductive bias of convolution and the global modeling capability of the Transformer. The Transformer's self-attention mechanism can be represented as:

[0046]

[0047] Where Q, K, and V are three sub-features obtained by performing a cubic linear mapping on the input features, and they satisfy... l represents the length of the input feature and the three sub-features (Q, K, V). d q ,d k d v These represent the widths of the three sub-features. Q and K are divided into multiple parts, and their similarity is calculated using a dot product to determine the weight V.

[0048] In the local channel attention interaction mechanism, the local channel interaction mechanism can be represented as:

[0049]

[0050] in, wi Let σ represent the i-th component of the channel weight, σ be the Sigmoid function, and ω be the sigmoid function. j Let y represent the learnable weight factors in j dimensions. j Let j represent the j-th channel of the feature map, and k represent the range of local channel interactions.

[0051] This invention employs transfer learning to accelerate the model convergence process and uses a label-smoothed cross-entropy loss function (Loss). ls ) Perform loss calculation, Loss ls It can be represented as:

[0052]

[0053] Where ε represents a small positive hyperparameter, K represents the number of image classifications, and q i This represents the predicted label output by the model.

[0054] Step 104: Load and save the trained optimal model, input the image to be classified, and obtain the classification result of the image data.

[0055] Example 2

[0056] like Figure 4 As shown, this embodiment provides an image classification system based on lightweight deredundant convolution and an improved Transformer. The image classification process utilizes the aforementioned method, including:

[0057] Data acquisition module 401 is used to acquire a classification image dataset;

[0058] Data processing module 402 is used to divide image data into training set and test set according to a specified ratio and perform data augmentation operations such as random rotation, random cropping, random flipping, and color transformation.

[0059] The model training module 403 uses the training set to train the parameters of the predefined model, and obtains and saves the optimal model weights.

[0060] Output module 404 is used to call the trained optimal model and input the image to be classified to obtain the final high-precision classification result.

[0061] Example 3

[0062] Based on the image classification method based on lightweight de-redundant convolution and improved Transformer provided by this invention, the specific implementation process of this embodiment is as follows:

[0063] In this embodiment, image features are learned end-to-end using a deep learning model, eliminating the need for manual feature design.

[0064] The dataset used consists of lung CT images obtained from multiple medical institutions. To ensure the accuracy and reliability of the experimental results, the data was cleaned and preprocessed, and multiple experiments were conducted to calculate the average value. The entire dataset includes lung CT images of different stages and types, enabling the trained model to have better generalization performance. The dataset used in this embodiment is open-source and available at: https: / / www.kaggle.com / datasets / mohamedhanyyy / chest-ctscan-images.

[0065] Experimental environment:

[0066] To ensure sufficient and efficient model training, a high-performance computing platform was used for model parameter inference. The experiment used a Linux operating system (64-bit), paired with an AMD Ryzen 7 5800H@3.2GHz CPU and 16GB of memory.

[0067] To handle the massive computations required for image recognition tasks, an NVIDIA RTX V100 GPU was used, significantly improving the speed of model training and inference.

[0068] The software version used was Python 3.8, and the deep learning framework used was PyTorch 1.11.1; this experimental environment provided a solid foundation for the development and optimization of the image classification model, ensuring the smooth progress of the project.

[0069] Table 1 below shows the performance comparison between the image classification method based on lightweight deredundancy convolution and improved Transformer proposed in this embodiment and other existing state-of-the-art models.

[0070] Table 1 Performance Comparison of Each Model

[0071] Model Name Accuracy (%) Accuracy (%) Recall rate (%) F1 score (%) ResNet34 96.98 96.97 97.14 97.03 MobileVit 97.49 97.55 97.62 97.56 CMT 91.46 92.49 92.52 92.12 ConvNeXt 97.65 97.79 97.54 97.64 Our model 98.99 98.86 99.15 99.00

[0072] As shown in Table 1, the image classification method proposed in this embodiment achieves accuracy of 98.99%, precision of 98.86%, recall of 99.15%, and F1 score of 99.00%, respectively. This surpasses four other existing state-of-the-art models. This indicates that the model of this invention has higher classification accuracy when handling image classification tasks and is more suitable for practical applications.

[0073] In summary, the image classification method based on lightweight de-redundant convolution and improved Transformer in the above embodiments performs feature filtering on lung CT images with similar backgrounds and small target regions by constructing lightweight de-redundant convolution, removing redundant features and obtaining robust feature representations. At the same time, the dual-stream mode combining convolutional neural networks and improved Transformer enhances the model's inductive bias ability and long-distance dependency capture ability, effectively improving the quality of the final lung CT image classification results. It has excellent characteristics of strong applicability and high classification performance.

[0074] The present invention and its embodiments have been described above illustratively. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention, and are not actually limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar methods and embodiments without departing from the spirit of the present invention, such designs should fall within the protection scope of the present invention.

Claims

1. An image classification method based on lightweight de-redundant convolution and improved Transformer, characterized in that, The steps are: Step 101: Obtain a classification image data set, which contains at least two types of data; Step 102: Divide the classification image data into a training set and a test set, and perform data enhancement processing; Step 103: Construct a lightweight de-redundancy convolution and improved Transformer image classification model, and optimize the training of the image classification model; the image classification model includes four stages, which are connected in series, wherein: Stage 1 includes a feature extraction head Stem, a lightweight de-redundancy convolution module SC Block and an improved Transformer module LTA Block; Stage 2, Stage 3 and Stage 4 have the same structure, and each includes a downsampling module DS Block, a lightweight de-redundancy convolution module SC Block and an improved Transformer module LTA Block; After the data is input into the image classification model, it is first preliminarily extracted by the feature extraction head Stem, then input into the lightweight de-redundancy convolution module SC Block and the improved Transformer module LTA Block for parallel processing, and then the double-flow feature extraction channel is formed. Feature fusion is performed; The data processing process of the lightweight de-redundancy convolution module SC Block is: the data first enters the de-redundancy convolution SCS, then passes through the depth separable convolution DSC with a convolution kernel size of 7, a step distance of 1 and a padding of 3, and then passes through the convolution Conv with a convolution kernel size of 1 and a step distance of 1, and the convolution Conv with a convolution kernel size of 7, a step distance of 1 and a padding of 3, and the layer normalization LN, and finally input into the Drop Path to prevent overfitting; The data processing process of the improved Transformer module LTA Block is: the input features first pass through the convolution with a convolution kernel size of 3 and 1 for preliminary feature extraction, then pass through the Transformer module for global context information modeling, and then pass through the convolution with a convolution kernel size of 1 and the convolution with a convolution kernel size of 3 to adjust the channel number and feature map size, and finally pass through the local channel attention interaction mechanism to strengthen the edge features; Step 104: Load and save the trained optimal model, and input the image to be classified to obtain the classification result of the image data.

2. The image classification method based on lightweight de-redundancy convolution and improved Transformer according to claim 1, characterized in that: The data enhancement operation adopted in step 102 includes random rotation, random cropping, random flipping and color transformation.

3. The image classification method based on lightweight de-redundancy convolution and improved Transformer according to claim 2, characterized in that: After the data processed by Stage 1 enters Stage 2, Stage 3 and Stage 4 in turn for further processing, Stage 2, Stage 3 and Stage 4 repeat the same processing process, which is: first, the downsampling module DS Block is used for downsampling, and then the lightweight de-redundancy convolution module SC Block and the improved Transformer module LTA Block are used for double-flow feature extraction to extract deep abstract features.

4. The image classification method based on lightweight de-redundancy convolution and improved Transformer according to claim 1, characterized in that: The feature extraction head Stem is composed of a convolution with a convolution kernel size of 4 and a step distance of 4 and a normalization layer.

5. The image classification method based on lightweight de-redundancy convolution and improved Transformer according to claim 1, characterized in that: The step 103 adopts a training mode of transfer learning to accelerate model convergence and adopts a cross-entropy loss function with label smoothing to perform loss calculation.

6. An image classification system based on lightweight de-redundant convolution and improved Transformer, characterized in that: The method of any one of claims 1-5 is used for image classification, comprising: a data acquisition module configured to acquire a classification image dataset; a data processing module configured to divide the image data into a training set and a test set according to a specified proportion, and perform a data enhancement operation to pre-process the image data; a model training module configured to perform parameter training on a defined model using the training set to obtain and save optimal model weights; an output module configured to call the trained optimal model, input an image to be classified, and obtain a final high-precision classification result.

Images