Image classification method and system based on lightweight redundancy removal convolution and improved Transform

Abstract

The invention discloses an image classification method and system based on lightweight redundant convolution removal and improved Transform, and belongs to the technical field of image processing, and the method comprises the steps: obtaining a classification image data set; dividing the classified image data set into a training set and a test set, and performing data enhancement; constructing a lightweight redundant convolution removal and Transform improvement image classification model, adopting a transfer learning training mode, and using a training set to optimize parameters to obtain an optimal model; and loading the trained optimal model, and inputting an image to be classified to obtain a classification result. According to the method, features of images with similar backgrounds and small target areas are screened by constructing lightweight redundancy removal convolution, redundant features are removed, robust feature representation is obtained, the inductive bias capability and the long-distance dependence capture capability of a model are enhanced by adopting a double-flow mode combining a convolutional neural network and an improved Transform, and the robustness of the model is improved. The method effectively improves the quality of a final image classification result, and has the excellent characteristics of high applicability and high classification performance.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image classification method and system based on lightweight de-redundant convolution and improved Transformer. Background Art

[0002] With the continuous development of artificial intelligence technology, image classification plays an important role in multiple application scenarios, such as autonomous driving, medical diagnosis, security monitoring, etc. Although traditional convolutional neural networks perform well in image classification tasks, they mainly rely on local convolution operations, so they perform poorly in capturing the dependencies between distant pixels in the image. Existing image classification algorithms are also unable to effectively extract features from data with similar image backgrounds and small target areas, resulting in poor performance of the classification results obtained by running the classification algorithm.

[0003] After searching, the patent number is CN118247561A, the patent publication date is June 25, 2024, and the patent name is: A new energy battery pack image classification method and device based on a multi-scale attention mechanism. This application is based on the CNN+Transformer architecture of the multi-scale attention mechanism, which improves the traditional CNN-based image classification method and improves the image classification accuracy of new energy battery packs when the feature differences between categories are too small; the MSNET network can extract local information at different scales, integrate local structural information into global context information, and improve feature recognition capabilities to alleviate the problem of insufficient global context information. At the same time, in order to alleviate the problems of large number of parameters and high computational complexity, the application proposes to introduce deep separable convolution in the MSNET network, perform self-attention in the channel dimension rather than the spatial dimension, and greatly reduce the convolution parameters and computational complexity.

[0004] The above-mentioned application focuses on solving the problems of too small differences in target areas, insufficient global context information, and a large number of network parameters, which are similar to the problems to be solved by this patent. However, the above-mentioned application mainly solves the problems by constructing a CNN+Transformer architecture based on a multi-scale attention mechanism, which differs from this patent in terms of solution design. Summary of the invention

[0005] 1. Technical problem to be solved by the invention

[0006] In view of the problems in the prior art that traditional classification algorithms cannot handle the characteristics of images with similar backgrounds and small target areas well and ignore the global context information of the image, resulting in poor performance of the classification results obtained by running traditional classification algorithms, the present invention provides an image classification method and system based on lightweight de-redundant convolution and improved Transformer. The classification method provided by the present invention can effectively improve the quality of the final results of image classification.

[0007] 2. Technical solution

[0008] In order to achieve the above object, the technical solution provided by the present invention is:

[0009] The present invention provides an image classification method based on lightweight de-redundant convolution and improved Transformer, the steps of which are as follows:

[0010] Step 101: Obtain a classified image data set, which contains at least two types of data;

[0011] Step 102: Divide the classified image data into a training set and a test set, and perform data enhancement processing;

[0012] Step 103: Build a lightweight de-redundant convolution and improved Transformer image classification model, and optimize the image classification model for training;

[0013] Step 104: Load and save the trained optimal model, input the image to be classified to obtain the classification result of the image data.

[0014] Furthermore, the data enhancement operations adopted in step 102 include random rotation, random cropping, random flipping and color transformation.

[0015] Furthermore, the image classification model constructed in step 103 includes 4 stages, and the 4 stages are connected in series, wherein: Stage 1 includes a feature extraction head Stem, a lightweight de-redundant convolution module SC Block and an improved Transformer module LTA Block; Stage 2, Stage 3 and Stage 4 have the same structure, and all include a downsampling module DS Block, a lightweight de-redundant convolution module SC Block and an improved Transformer module LTA Block.

[0016] Furthermore, after the data is input into the image classification model, the feature extraction head Stem first performs preliminary feature extraction, and then passes it to the lightweight de-redundant convolution module SC Block and the improved Transformer module LTA Block for parallel processing to form a dual-stream feature extraction channel for feature fusion.

[0017] Furthermore, the data processed by Stage 1 enters Stage 2, Stage 3 and Stage 4 for further processing in turn. Stage 2, Stage 3 and Stage 4 repeat the same processing process, specifically: first, down-sampling is performed through the down-sampling module DS Block, and then dual-stream feature extraction is performed through the lightweight de-redundant convolution module SC Block and the improved Transformer module LTA Block to extract deep abstract features.

[0018] Furthermore, the feature extraction head Stem consists of a convolution and normalization layer with a convolution kernel size of 4 and a stride of 4.

[0019] Furthermore, the data processing process of the lightweight de-redundant convolution module SC Block is as follows: the data first enters the de-redundant convolution SCS, and then passes through the depth-separable convolution DWC and layer normalization LN with a convolution kernel size of 7, a step size of 1, and a padding of 3, and then passes through the convolution Conv with a convolution kernel size of 1 and a step size of 1, and the convolution Conv with a convolution kernel size of 7, a step size of 1, and a padding of 3, layer normalization LN, and finally enters the Drop Path to prevent overfitting.

[0020] Furthermore, the data processing process of the improved Transformer module LTA Block is as follows: the input features are first subjected to preliminary feature extraction by convolutions with kernel sizes of 3 and 1 respectively, and then the global context information is modeled by the Transformer module of global feature learning, and then the number of channels and the size of the feature map are adjusted by convolutions with kernel size of 1 and convolution kernel size of 3 in turn, and finally the edge features are enhanced through the local channel attention interaction mechanism.

[0021] Furthermore, step 103 adopts a transfer learning training method to accelerate model convergence and uses a label-smoothed cross entropy loss function to calculate the loss.

[0022] The present invention provides an image classification system based on lightweight de-redundant convolution and improved Transformer, which uses the above method to perform image classification, including:

[0023] A data acquisition module, used to acquire a classification image data set;

[0024] A data processing module is used to divide the image data into a training set and a test set according to a specified ratio, and perform data enhancement operations to pre-process the image data;

[0025] The model training module uses the training set to train the parameters of the defined model to obtain and save the optimal model weights;

[0026] The output module is used to call the trained optimal model and input the image to be classified to obtain the final high-precision classification result.

[0027] 3. Beneficial effects

[0028] Compared with the existing known technologies, the technical solution provided by the present invention has the following significant effects:

[0029] The present invention discloses an image classification method based on lightweight de-redundant convolution and improved Transformer. Firstly, a data enhancement method is used to enable the model to better learn the essential characteristics of the data during the training process, so that it performs better when facing test data that has not participated in the training. This helps to reduce overfitting and improve the generalization ability of the model.

[0030] Then, the proposed lightweight de-redundant convolution module is used to perform redundant screening at both the spatial scale and the channel scale, and excessive information loss is avoided through residual connections, which solves the interference of small target areas and similar backgrounds on the classification results in the image, making the extracted features more representative.

[0031] Finally, the proposed improved Transformer module is used to improve the model's attention to global context information in a local-global-local manner. The self-attention mechanism can effectively capture the relationship between distant elements in the sequence, and the channel attention can enable the model to better distinguish the edge features of the target. After preliminary downsampling, the data is processed in parallel by the improved Transformer module and the lightweight de-redundant convolution module and then fused to form a two-stream feature extraction channel. This method can more fully extract features and improve the robustness of the model. It effectively improves the quality of the final result of image classification and has the excellent characteristics of strong applicability and high classification performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flowchart of an image classification method based on lightweight de-redundant convolution and improved Transformer in Embodiment 1 of the present invention;

[0033] Figure 2 It is a schematic diagram of the model structure of the image classification method based on lightweight de-redundant convolution and improved Transformer in Example 1 of the present invention;

[0034] Figure 3 It is a structural schematic diagram of an improved Transformer (LTA Block) in an image classification method based on lightweight de-redundant convolution and improved Transformer in Example 1 of the present invention;

[0035] Figure 4It is a structural diagram of an image classification system based on lightweight de-redundant convolution and improved Transformer in Example 2 of the present invention. DETAILED DESCRIPTION

[0036] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and embodiments.

[0037] Example 1

[0038] like Figure 1 As shown, this embodiment provides an image classification method based on lightweight de-redundant convolution and improved Transformer, including the following steps:

[0039] Step 101: Obtain a classified image dataset, where the classified dataset contains at least two types of data.

[0040] Step 102: Divide the classified image data into a training set and a test set, and perform data enhancement. First, divide the data set into a training set and a test set in a ratio of 8:2, and then perform data enhancement operations such as random rotation, random cropping, random flipping, and color transformation on the training set to reduce the risk of model overfitting.

[0041] Step 103: Design and build a lightweight de-redundant convolution and improved Transformer image classification model, adopt the transfer learning training method, and use the training set to optimize the parameters to obtain the optimal model.

[0042] like Figure 2 As shown, the model classification designed and constructed by the present invention includes 4 stages. Stage 1 is composed of a feature extraction head Stem, a lightweight de-redundant convolution module SC Block and an improved Transformer module LTA Block. Stages 2, 3, and 4 are all composed of a downsampling module DS Block, a lightweight de-redundant convolution module SC Block and an improved Transformer module LTABlock. After the data is input into the network, it is firstly subjected to preliminary feature extraction and reduced computational complexity by a feature extraction head Stem consisting of a convolution kernel size of 4 and a step size of 4 and a normalization layer, and then is passed into a lightweight de-redundant convolution module SC Block and an improved Transformer module LTA Block for parallel processing to form a dual-stream feature extraction channel for fusion. Then, after passing through the serially connected Stages 2, 3, and 4, the downsampling is repeated through the downsampling module DS Block, and the dual-stream feature extraction process is repeated through the lightweight de-redundant convolution module SC Block and the improved Transformer module LTA Block to extract deep abstract features. Finally, the classification category is output after normalization and linear mapping at the classification layer.

[0043] The initial layers are responsible for capturing simple features, such as edges and corners, while deeper layers combine these simple features to form more complex abstract representations. This layer-by-layer processing enables the network to deeply understand the structure and patterns of the data. In the multi-layer structure, the lightweight de-redundant convolution module SC Block can improve the efficiency of extracting effective features and avoid redundant information affecting the final classification accuracy. The improved Transformer module LTA Block can lock on to small target areas and effectively extract global information from the data through a local-global-local approach. This multi-layer structure helps the model capture the deep semantic features of the image and can effectively improve the image classification effect. Combined with Figure 2 , Figure 2 Here K represents the size of the convolution kernel, S represents the stride of the convolution operation, and P represents the padding during the convolution process.

[0044] The data transmission process through the lightweight de-redundant convolution module SC Block is as follows: first enter the de-redundant convolution SCS, then pass through the depth-separable convolution DWC with a convolution kernel size of 7, a stride of 1, and a padding of 3, and layer normalization LN, then pass through the convolution Conv with a convolution kernel size of 1 and a stride of 1, the GELU activation function, and the convolution Conv with a convolution kernel size of 7, a stride of 1, and a padding of 3, layer normalization LN, and finally enter the Drop Path to prevent overfitting. SCS convolution removes redundancy based on the characteristics of small target areas and similar backgrounds in images to obtain representative features. Compared with ordinary convolution, the depth-separable convolution DWC has a smaller number of parameters, and the convolution feature extraction performance is basically the same as that of traditional convolution, which can effectively improve the inference speed of the model and reduce computing power requirements.

[0045] like Figure 3 As shown in the figure, the improved Transformer module LTA Block consists of two convolution Convs with convolution kernel sizes of 3 and 1, a Transformer for global feature learning, and a local channel attention interaction mechanism. The input features are first subjected to preliminary feature extraction through convolutions with convolution kernel sizes of 3 and 1, followed by global context information modeling by the Transformer module for global feature learning, and then the number of channels and feature map size are adjusted through convolutions with convolution kernel size of 1 and convolution kernel size of 3, and finally the edge features are enhanced through the local channel attention interaction mechanism. This local-global-local approach can fully extract the feature information of the image, while having the inductive bias of convolution and the global modeling capability of Transformer. The self-attention mechanism of Transformer can be expressed as:

[0046]

[0047] Among them, Q, K, and V are three sub-features obtained by performing three linear mappings on the input features, which satisfy l represents the length of the input feature and the three sub-features (Q, K, V). q ,d k , d v Represents the width of the three sub-features respectively. Q and K will be divided into multiple parts, and the similarity is calculated by dot product to determine the weight V.

[0048] In the local channel attention interaction mechanism, the local channel interaction mechanism can be expressed as:

[0049]

[0050] in, wi represents the i-th component of the channel weight, σ is the Sigmoid function, ω j represents the learnable weight factor of j dimensions, y j represents the jth channel of the feature map, and k represents the range of local channel interaction.

[0051] The present invention adopts the transfer learning method to accelerate the model convergence process and adopts the label smoothing cross entropy loss function (Loss ls ) to calculate the loss, Loss ls It can be expressed as:

[0052]

[0053] Among them, ε represents a small positive hyperparameter, K represents the type of image classification, and q i Represents the predicted label of the model output.

[0054] Step 104: Load and save the trained optimal model, input the image to be classified to obtain the classification result of the image data.

[0055] Example 2

[0056] like Figure 4 As shown, this embodiment provides an image classification system based on lightweight de-redundant convolution and improved Transformer, and uses the above method to perform image classification work, including:

[0057] A data acquisition module 401 is used to acquire a classification image data set;

[0058] The data processing module 402 is used to divide the image data into a training set and a test set according to a specified ratio and perform data enhancement operations such as random rotation, random cropping, random flipping, and color transformation;

[0059] Model training module 403, using the training set to perform parameter training on the defined model, and obtain and save the optimal model weights;

[0060] The output module 404 is used to call the trained optimal model and input the image to be classified to obtain the final high-precision classification result.

[0061] Example 3

[0062] Based on the image classification method based on lightweight de-redundant convolution and improved Transformer provided by the present invention, this embodiment provides a specific implementation process as follows:

[0063] In this embodiment, a deep learning model is used to learn image features end-to-end without the need for manual feature design.

[0064] The dataset used is lung CT images obtained from multiple medical institutions. In order to ensure the accuracy and reliability of the experimental results, the data was cleaned and preprocessed, and multiple experiments were performed to obtain the average value. The entire dataset includes lung CT images of different stages and types, so that the trained model has better generalization performance. The dataset used in this embodiment is open source at: https: / / www.kaggle.com / datasets / mohamedhanyyy / chest-ctscan-images.

[0065] Experimental environment:

[0066] In order to ensure that the model training is sufficient and efficient, a high-performance computing platform is used for model parameter inference. The experiment uses the Linux operating system (64-bit), with an AMD Ryzen 7 5800H@3.2GHz CPU and 16GB of memory.

[0067] To handle the massive amounts of computation required for image recognition tasks, NVIDIA RTX V100 GPUs were used to greatly speed up model training and reasoning.

[0068] The software version uses Python 3.8 and the deep learning framework uses PyTorch 1.11.1; such an experimental environment provides a solid foundation for the development and tuning of the image classification model and ensures the smooth progress of the project.

[0069] Table 1 below shows the performance comparison between the image classification method based on lightweight de-redundant convolution and improved Transformer proposed in this embodiment and other existing advanced models.

[0070] Table 1 Performance comparison of each model

[0071] Model Name Accuracy (%) Accuracy (%) Recall rate (%) F1 score (%) ResNet34 96.98 96.97 97.14 97.03 MobileVit 97.49 97.55 97.62 97.56 CMT 91.46 92.49 92.52 92.12 ConvNeXt 97.65 97.79 97.54 97.64 Our Model 98.99 98.86 99.15 99.00

[0072] As can be seen from Table 1, the image classification method proposed in this embodiment has achieved 98.99%, 98.86%, 99.15% and 99.00% in accuracy, precision, recall and F1 score respectively, which exceeds the other four existing advanced models. This shows that the model of the present invention has a higher classification accuracy when processing image classification tasks and is more suitable for use in practical applications.

[0073] In summary, the image classification method based on lightweight de-redundant convolution and improved Transformer in the above embodiment performs feature screening on lung CT images with similar backgrounds and small target areas by constructing lightweight de-redundant convolution, removes redundant features, and obtains robust feature representation. At the same time, a dual-stream mode combining a convolutional neural network and an improved Transformer is adopted to enhance the model's inductive bias ability and long-distance dependency capture ability, thereby effectively improving the quality of the final result of lung CT image classification, and having the excellent characteristics of strong applicability and high classification performance.

[0074] The above is a schematic description of the present invention and its implementation methods, which is not restrictive. The drawings show only one implementation method of the present invention and are not limited thereto. Therefore, if a person skilled in the art is inspired by it and designs methods and embodiments similar to the technical solution without creativity without departing from the purpose of the invention, they should all fall within the scope of protection of the present invention.

Claims

1. An image classification method based on lightweight de-redundant convolution and improved Transformer, characterized in that: The steps are: Step 101: Obtain a classified image data set, which contains at least two types of data; Step 102: Divide the classified image data into a training set and a test set, and perform data enhancement processing; Step 103: Build a lightweight de-redundant convolution and improved Transformer image classification model, and optimize the image classification model for training; Step 104: Load and save the trained optimal model, input the image to be classified to obtain the classification result of the image data.

2. The image classification method based on lightweight de-redundant convolution and improved Transformer according to claim 1, characterized in that: The data enhancement operations used in step 102 include random rotation, random cropping, random flipping and color transformation.

3. The image classification method based on lightweight de-redundant convolution and improved Transformer according to claim 1, characterized in that: The image classification model constructed in step 103 includes 4 stages, which are connected in series, wherein: Stage 1 includes a feature extraction head Stem, a lightweight de-redundant convolution module SC Block and an improved Transformer module LTABlock; Stage 2, Stage 3 and Stage 4 have the same structure, and all include a downsampling module DS Block, a lightweight de-redundant convolution module SC Block and an improved Transformer module LTA Block.

4. The image classification method based on lightweight de-redundant convolution and improved Transformer according to claim 3, characterized in that: After the data is input into the image classification model, the feature extraction head Stem first performs preliminary feature extraction, and then passes it to the lightweight de-redundant convolution module SC Block and the improved Transformer module LTA Block for parallel processing to form a dual-stream feature extraction channel for feature fusion.

5. The image classification method based on lightweight de-redundant convolution and improved Transformer according to claim 4, characterized in that: The data processed by Stage 1 enters Stage 2, Stage 3 and Stage 4 for further processing. Stage 2, Stage 3 and Stage 4 repeat the same processing process, specifically: first, down-sampling is performed through the down-sampling module DS Block, and then dual-stream feature extraction is performed through the lightweight de-redundant convolution module SC Block and the improved Transformer module LTA Block to extract deep abstract features.

6. The image classification method based on lightweight de-redundant convolution and improved Transformer according to claim 3, characterized in that: The feature extraction head Stem consists of a convolution and normalization layer with a convolution kernel size of 4 and a stride of 4.

7. The image classification method based on lightweight de-redundant convolution and improved Transformer according to claim 3, characterized in that: The data processing process of the lightweight de-redundant convolution module SC Block is as follows: the data first enters the de-redundant convolution SCS, then passes through the depth-separable convolution DWC with a convolution kernel size of 7, a step size of 1, and a padding of 3, and the layer normalization LN, then passes through the convolution Conv with a convolution kernel size of 1 and a step size of 1, and the convolution Conv with a convolution kernel size of 7, a step size of 1, and a padding of 3, the layer normalization LN, and finally enters the Drop Path to prevent overfitting.

8. The image classification method based on lightweight de-redundant convolution and improved Transformer according to claim 3, characterized in that: The data processing process of the improved Transformer module LTA Block is as follows: the input features are first subjected to preliminary feature extraction by convolutions with kernel sizes of 3 and 1, respectively, and then the global context information is modeled by the Transformer module of global feature learning, and then the number of channels and the size of the feature map are adjusted by convolutions with kernel sizes of 1 and 3, and finally the edge features are enhanced by the local channel attention interaction mechanism.

9. The image classification method based on lightweight de-redundant convolution and improved Transformer according to claim 3, characterized in that: Step 103 uses transfer learning training to accelerate model convergence and uses label-smoothed cross entropy loss function to calculate loss.

10. An image classification system based on lightweight de-redundant convolution and improved Transformer, characterized by: The method according to any one of claims 1 to 9 is used to classify images, comprising: A data acquisition module, used to acquire a classification image data set; A data processing module is used to divide the image data into a training set and a test set according to a specified ratio, and perform data enhancement operations to pre-process the image data; The model training module uses the training set to train the parameters of the defined model to obtain and save the optimal model weights; The output module is used to call the trained optimal model and input the image to be classified to obtain the final high-precision classification result.

Images