General framework construction method for small data set color image classification
By performing channel decomposition and Haar transformation on small datasets, a BLOCK module with multi-stage self-attention learning is built, which solves the overfitting problem of the visual Transformer framework on small datasets, improves image classification accuracy and reduces the computational complexity , achieves similar performance to CNN framework.
Patent Information
- Application Number
- CN202411516164.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2024-12-27
AI Technical Summary
The visual Transformer framework is prone to overfitting when processing small-scale data sets, declining classification performance, and is not good at capturing local information of the image, resulting in low classification accuracy on small data sets.
Using a general framework method of small dataset training, the sample images are preprocessed, channel decomposition and Haar transformation are carried out, and the BLOCK module based on Swin transformer Block is built to perform multi-stage self-attention learning, blocking and exhibition in and between windows. Flat feature maps to improve feature extraction capabilities.
On a small dataset, the image classification accuracy is higher than that of the existing Transformer framework, achieving the same performance as the CNN framework, while reducing the amount of parameters and calculations.
Smart Images
Figure CN119206368A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image classification, and in particular relates to a general framework construction method for classifying color images of small data sets. Background Art
[0002] Image classification is to divide the input image into different categories, which is a basic task in the field of computer vision. Traditional image classification methods use the color, texture, and edge of the image as image research features. These features form a feature space through feature vectors and obtain the corresponding category space through artificially designed feature matching, and the classification accuracy is low. The image classification framework based on deep learning uses convolutional neural network CNN to automatically learn local features and deep features of the image to achieve end-to-end training. This method achieves state-of-the-art performance in image classification tasks, and the classification accuracy is improved from about 50% of traditional methods to nearly 90%. However, despite the great success of the CNN framework, its ability to process sequence data is weak and it cannot make good use of the spatial and temporal information of the image sequence. To solve this problem, the visual Transformer framework came into being. It regards images as sequence data, uses the self-attention mechanism to capture the global information of the image, and learns the long-distance dependencies between different regions in the image. This enables the visual Transformer framework to show excellent classification performance with the support of training on large-scale datasets.
[0003] However, due to the large number of parameters of the visual Transformer framework and its limited ability to capture local information of images, it is very easy to overfit when processing small-scale data sets, resulting in a decline in classification performance and its accuracy is not higher than that of the CNN framework. Summary of the invention
[0004] The purpose of the present invention is to provide a general framework construction method for small data set color image classification, which solves the problem that the visual Transformer framework has a large number of parameters and is not good at capturing local information of the image. The method uses a small data set for training and achieves higher image classification accuracy than the existing Transformer framework under small data set training, and reaches the accuracy of image classification methods based on the CNN framework.
[0005] The technical solution adopted by the present invention is a general framework construction method for small data set color image classification, specifically: Step 1, preprocessing sample image set; Step 2, performing channel decomposition processing on the images in the sample image set obtained after preprocessing in step 1; Step 3, performing a layer of Haar transform on each channel feature map obtained in step 2; Step 4, cascading the feature maps obtained in step 3 into one feature map; Step 5: Build a BLOCK module, divide the feature map obtained in step 4 into blocks, flatten it into column vectors, and input it into the BLOCK module for Stage 1 processing; Step 6: The feature map of the Stage 1 obtained in Step 5 is first cascaded, then segmented, and then scaled and flattened into vectors and input into the BLOCK module for Stage 2 processing; In step 7, the feature map of Stage 2 obtained in step 6 is first cascaded, then segmented, and then scaled and flattened into vectors and input into the BLOCK module for Stage 3 processing.
[0006] The present invention is also characterized in that: The specific process of step 1 is: Step 1.1, select n images of size The images are used as training sample sets ; Step 1.2: Set the training sample set obtained in step 1.1 The i Put an image on the CPU, , use the memory-contiguous num array to store the image; Step 1.3, for i The image data is enhanced for the first image. i The image data enhancement processing methods for the image are random geometric transformation, brightness adjustment, random noise addition and random cropping. Among them, the random geometric transformation includes random rotation, random inversion, scaling and translation, and the image size is kept .
[0007] Among them, the rotation range of random rotation is ; The scaling degree of random scaling is ; The degree of translation is ; Brightness parameter is ; The noise coefficient of random noise is .
[0008] The specific process of step 2 is: The first i The sample image is decomposed into R, G, and B channels to obtain three channel feature maps of R, G, and B. The sizes of the three channel feature maps are .
[0009] The specific process of step 3 is: Step 3.1: Perform a Haar transform on the R channel feature map obtained in step 2 to obtain the approximate feature size of the R channel. The feature map of the R channel has a horizontal size of The detail feature map of the R channel has a vertical dimension of The size of the detail feature map and the diagonal direction of the R channel is Detailed feature map; Step 3.2: Perform a Haar transform on the G channel feature map obtained in step 2 to obtain the approximate feature size of the G channel. The feature map of G channel has a horizontal size of The detail feature map of G channel has a vertical dimension of The size of the detail feature map and the diagonal direction of the G channel is Detailed feature map; Step 3.3, perform a layer of Haar transform on the B channel feature map obtained in step 2, and obtain the approximate feature size of the B channel as The feature map of the B channel has a horizontal size of The detail feature map of the B channel has a vertical dimension of The size of the detail feature map of and the diagonal direction of the B channel is After the previous steps, 12 feature maps are finally obtained.
[0010] The specific process of step 4 is: cascade the 12 feature maps obtained in step 3 into a feature map with a size of .
[0011] The specific process of step 5 is: Step 5.1, build the BLOCK module. The BLOCK module structure is based on the structure of Swin transformer Block, which includes the intra-window self-attention learning module (W-MSA) and the inter-window self-attention learning module (SW-MSA). The intra-window self-attention learning module (W-MSA) is composed of a normalization layer (Layer Norm, LN), a window multi-headed self-attention (Window Multi-Headed Self-Attention, W-MSA) learning layer, an LN layer and a multi-layer perceptron (Multilayer Perceptron, MLP) layer in sequence; the inter-window self-attention learning module (SW-MSA) is composed of a normalization layer (LN), a shifted window multi-headed self-attention (Shifted Window Multi-Headed Self-Attention, SW-MSA) learning, an LN layer and a multi-layer perceptron (Multilayer Perceptron, MLP) layer in sequence; Step 5.2: split the feature map obtained in step 4 into windows, with each window size being , a total of non-overlapping windows; Step 5.3: Perform the feature map obtained in step 4 Scale block, each window has Feature blocks; Step 5.4, the size obtained in step 5.3 is The feature block is transformed into a size of One-dimensional vector of Feature blocks, converted to size The vector matrix of ; Step 5.5: Substitute the The vector matrix is input into the BLOCK module to perform self-attention learning within and between windows of Stage 1. It is iterated twice to train the learning parameters of Stage 1 and output the learning parameters of Stage 1. Scale feature map.
[0012] The specific process of step 6 is: Step 6.1: Output the Stage 1 obtained in step 5 The scale feature maps are cascaded and then the feature maps are segmented; Step 6.2: perform window segmentation on the feature map obtained in step 6.1, with each window size being , a total of non-overlapping windows; Step 6.3: perform the following operations on the feature map obtained in step 6.1: Scale blocks, so that each window divided in step 6.2 has Feature blocks; Step 6.4, the scale size obtained in step 6.3 is The feature block is transformed into a size of One-dimensional vector of Feature blocks, converted to size The vector matrix of ; Step 6.5, change the size obtained in step 6.4 to The vector matrix is input into the BLOCK module to perform self-attention learning within and between windows of Stage2. It is iterated 4 times to train the learning parameters of Stage2 and output the learning parameters of Stage2. Scale feature map.
[0013] The specific process of step 7 is: Step 7.1: Output the framework Stage2 obtained in step 6 Scale feature map, cascade, and then segment feature map; Step 7.2: perform window segmentation on the feature map obtained in step 7.1, with each window size being , a total of non-overlapping windows; Step 7.3: perform the following operations on the feature map obtained in step 7.1: Scale blocks, so that each window divided in step 7.2 has Feature blocks; Step 7.4, the scale size obtained in step 7.3 is The feature block is transformed into a size of One-dimensional vector of Feature blocks, converted to size The vector matrix of ; Step 7.5, change the size obtained in step 7.4 to The vector matrix is input into the BLOCK module to perform self-attention learning in the Stage3 window, iterate twice, train the learning parameters of Stage3, and output the feature vector; Step 7.6: Repeat steps 1.2 to 7.5 to obtain The classification results of images until The training ends when , and a general framework for color image classification in small datasets is obtained.
[0014] The beneficial effects of the present invention are: The method of the present invention can use the existing public small data set of color images to assist in training and establish a model for image classification, without the need for large-scale, diversified and high-quality annotated data sets; using the method of the present invention, under the premise that the number of parameters and the amount of calculation are significantly lower than those of CNN, Vision Transformer framework and SwinTransformer framework, the classification accuracy is 14 points higher than that of the Vision Transformer framework, 7 points higher than that of the SwinTransformer framework, and is on par with that of the CNN framework. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 This is a flow chart of a general framework construction method for small data set color image classification according to Embodiment 1 of the present invention; Figure 2 The Haar transform process in Embodiment 1 of the present invention; Figure 3 Schematic diagram of the structure of the BLOCK model in Example 1 of the present invention; Figure 4 This is a schematic diagram of the feature segmentation process of Example 1 of the present invention; Figure 5 This is a comparison chart of image classification performance between the framework obtained by the method of the present invention and the prior art. DETAILED DESCRIPTION
[0016] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0017] The present invention provides a general framework construction method for small data set color image classification, which is specifically implemented according to the following steps: Step 1, preprocessing sample image set; The specific process of step 1 is: Step 1.1, select n images of size The images are used as training sample sets ; Step 1.2: Set the training sample set obtained in step 1.1 The i Put an image on the CPU, , use the memory-contiguous num array to store the image; Step 1.3, for i The image data is enhanced for the first image. i The image data enhancement processing methods for the image are random geometric transformation, brightness adjustment, random noise addition and random cropping. Among them, the random geometric transformation includes random rotation, random inversion, scaling and translation, and the image size is kept Among them, the rotation range of random rotation is ; The random scaling is ; The degree of translation is ; Brightness parameter is ; The noise coefficient of random noise is .
[0018] Step 2: perform channel decomposition on the image in the sample image set obtained after preprocessing in step 1, that is, i The size is The image is processed by channel decomposition; The specific process of step 2 is: The first i The sample image is decomposed into R, G, and B channels to obtain three channel feature maps of R, G, and B. The sizes of the three channel feature maps are .
[0019] Step 3, perform a layer of Haar transform on each channel feature map obtained in step 2, and obtain 12 feature maps in total; The specific process of step 3 is: Step 3.1: Perform a Haar transform on the R channel feature map obtained in step 2 to obtain the approximate feature size of the R channel. The feature map of the R channel has a horizontal size of The detail feature map of the R channel has a vertical dimension of The size of the detail feature map and the diagonal direction of the R channel is Detailed feature map; Step 3.2: Perform a Haar transform on the G channel feature map obtained in step 2 to obtain the approximate feature size of the G channel. The feature map of G channel has a horizontal size of The detail feature map of G channel has a vertical dimension of The size of the detail feature map and the diagonal direction of the G channel is Detailed feature map; Step 3.3, perform a layer of Haar transform on the B channel feature map obtained in step 2, and obtain the approximate feature size of the B channel as The feature map of the B channel has a horizontal size of The detail feature map of the B channel has a vertical dimension of The size of the detail feature map of and the diagonal direction of the B channel is After the previous steps, 12 feature maps are finally obtained.
[0020] Step 4, concatenate the 12 feature maps obtained in step 3 into one feature map; The specific process of step 4 is: The 12 feature maps obtained in step 3 are cascaded into one feature map, whose size is .
[0021] Step 5: Build the BLOCK module, divide the feature map obtained in step 4 into blocks, flatten it into column vectors, and input it into the BLOCK module to perform self-attention learning within and between windows in the Stage 1 stage, obtain the learning parameters of Stage 1, and output the Stage 1 stage. Scale feature map; The specific process of step 5 is: Step 5.1, build the BLOCK module. The BLOCK module structure is based on the structure of Swin transformer Block, which includes the intra-window self-attention learning module (W-MSA) and the inter-window self-attention learning module (SW-MSA). The intra-window self-attention learning module (W-MSA) is composed of a normalization layer (Layer Norm, LN), a window multi-headed self-attention (Window Multi-Headed Self-Attention, W-MSA) learning layer, an LN layer and a multi-layer perceptron (Multilayer Perceptron, MLP) layer in sequence; the inter-window self-attention learning module (SW-MSA) is composed of a normalization layer (LN), a shifted window multi-headed self-attention (Shifted Window Multi-Headed Self-Attention, SW-MSA) learning, an LN layer and a multi-layer perceptron (Multilayer Perceptron, MLP) layer in sequence; Step 5.2: split the feature map obtained in step 4 into windows, with each window size being , a total of non-overlapping windows; Step 5.3: Perform the feature map obtained in step 4 Scale block, each window has Feature blocks; Step 5.4, the size obtained in step 5.3 is The feature block is transformed into a size of One-dimensional vector of Feature blocks, converted to size The vector matrix of ; Step 5.5: Substitute the The vector matrix is input into the BLOCK module to perform self-attention learning within and between windows of Stage 1. It is iterated twice to train the learning parameters of Stage 1 and output the learning parameters of Stage 1. Scale feature map.
[0022] Step 6: The feature map of the Stage 1 obtained in Step 5 is first cascaded, then segmented, and then scaled and flattened into vectors. The vectors are input into the BLOCK module to perform self-attention learning within and between windows of Stage 2 to obtain the learning parameters of Stage 2 and output the Stage 2. Scale feature map; The specific process of step 6 is: Step 6.1: Output the Stage 1 obtained in step 5 The scale feature maps are cascaded and then the feature maps are segmented; Step 6.2: perform window segmentation on the feature map obtained in step 6.1, with each window size being , a total of non-overlapping windows; Step 6.3: perform the following operations on the feature map obtained in step 6.1: Scale blocks, so that each window divided in step 6.2 has Feature blocks; Step 6.4, the scale size obtained in step 6.3 is The feature block is transformed into a size of One-dimensional vector of Feature blocks, converted to size The vector matrix of ; Step 6.5, change the size obtained in step 6.4 to The vector matrix is input into the BLOCK module to perform self-attention learning within and between windows of Stage2. It is iterated 4 times to train the learning parameters of Stage2 and output the learning parameters of Stage2. Scale feature map.
[0023] In step 7, the feature map of Stage 2 obtained in step 6 is first cascaded, then segmented, and then scaled and flattened into vectors. The map is input into the BLOCK module, and self-attention learning is performed within the Stage 3 window to obtain the learning parameters of Stage 3. The feature vector is output to complete the construction of the framework for small data set image classification.
[0024] The specific process of step 7 is: Step 7.1: Output the framework Stage2 obtained in step 6 Scale feature map, cascade, and then segment feature map; Step 7.2: perform window segmentation on the feature map obtained in step 7.1, with each window size being , a total of non-overlapping windows; Step 7.3: perform the following operations on the feature map obtained in step 7.1: Scale blocks, so that each window divided in step 7.2 has Feature blocks; Step 7.4, the scale size obtained in step 7.3 is The feature block is transformed into a size of One-dimensional vector of Feature blocks, converted to size The vector matrix of ; Step 7.5, change the size obtained in step 7.4 to The vector matrix is input into the BLOCK module to perform self-attention learning in the Stage3 window, iterate twice, train the learning parameters of Stage3, and output the feature vector; Step 7.6: Repeat steps 1.2 to 7.5 to obtain The classification results of images until The training ends when , and a general framework for color image classification in small datasets is obtained.
[0025] Based on the above, the general framework for color image classification of small datasets is obtained.
[0026] Example 1 Figure 1 A flowchart of a general framework construction method for small data set color image classification according to an embodiment of the present invention is shown, and the method includes: Step 1, obtain a sample image set and perform image enhancement; In this step, this embodiment uses the CIFAR 10 dataset, which has 10 common categories and includes 50,000 training images and 10,000 test images.
[0027] During the training process, the image size of the CIFAR 10 dataset was adjusted to 224×224. In addition, appropriate data augmentation was performed on the training images in order to effectively learn the image features under various affine transformations. The data augmentation methods include random cropping, random horizontal flipping, and normalization.
[0028] The specific process of step 1 is: Step 1.1: Select 50,000 images of size 224×224×3 as the training sample set ; Step 1.2: Set the training sample set obtained in step 1.1 The i Put an image on the CPU, , use the memory-contiguous num array to store the image; Step 1.3, for i The image data is enhanced for the first image. i The image data enhancement processing methods for the image are random geometric transformation, brightness adjustment, random noise addition and random cropping. Among them, the random geometric transformation includes random rotation, random inversion, scaling and translation, and the image size is kept Among them, the rotation range of random rotation is ; The random scaling is ; The degree of translation is ; Brightness parameter is ; The noise coefficient of random noise is .
[0029] Step 2: The enhanced i The sample image is decomposed into R, G, and B channels to obtain three channel feature maps of R, G, and B. The sizes of the three channel feature maps are .
[0030] Step 3: Perform a layer of Haar transform on each channel feature map obtained in step 2 to obtain 12 feature maps. The specific process of step 3 is as follows: Step 3.1: Perform a Haar transform on the R channel feature map obtained in step 2 to obtain the approximate feature size of the R channel. The feature map of the R channel has a horizontal size of The detail feature map of the R channel has a vertical dimension of The size of the detail feature map and the diagonal direction of the R channel is Detailed feature map; Step 3.2: Perform a Haar transform on the G channel feature map obtained in step 2 to obtain the approximate feature size of the G channel. The feature map of G channel has a horizontal size of The detail feature map of G channel has a vertical dimension of The size of the detail feature map and the diagonal direction of the G channel is Detailed feature map; Step 3.3, perform a layer of Haar transform on the B channel feature map obtained in step S2, and obtain the approximate feature size of the B channel as The feature map of the B channel has a horizontal size of The detail feature map of the B channel has a vertical dimension of The size of the detail feature map of and the diagonal direction of the B channel is Detailed feature maps, and finally 12 feature maps are obtained.
[0031] In this embodiment, Figure 2 A layer of Haar transform process is shown: the feature maps are input into Haar high-pass and low-pass filters respectively, decomposed in the row direction, and the low-pass feature map and high-pass feature map in the row direction are obtained; the two are then input into Haar high-pass and low-pass filters respectively, decomposed in the column direction, and four feature maps are output, namely: the feature map of approximate features (dA), the detail feature map in the horizontal direction (dH), the detail feature map in the vertical direction (dV) and the detail feature map in the diagonal direction (dD).
[0032] Step 4: Concatenate the 12 feature maps obtained in step 3 into one feature map with a size of ; Step 5: Divide the feature map obtained in step S4 into blocks, flatten them into column vectors, input them into the BLOCK module, perform self-attention learning within and between windows of Stage 1, iterate twice, obtain the learning parameters of Stage 1, and output the learning parameters of Stage 1. Scale feature map. The specific steps include: Step 5.1, build the BLOCK module, Figure 3 The schematic diagram of the BLOCK module structure is shown. The BLOCK module structure is based on the Swin transformer Block structure, which includes an intra-window self-attention learning module (W-MSA) and an inter-window self-attention learning module (SW-MSA). Among them, the intra-window self-attention learning module (W-MSA) is composed of a normalization layer (Layer Norm, LN), a window multi-headed self-attention (Window Multi-Headed Self-Attention, W-MSA) learning layer, an LN layer, and a multi-layer perceptron (Multilayer Perceptron, MLP) series of operations; the inter-window self-attention learning module (SW-MSA) is composed of a normalization layer (LN), a shifted window multi-headed self-attention (Shifted Window Multi-Headed Self-Attention, SW-MSA) learning, an LN layer, and a multi-layer perceptron (MLP) series of operations; Step 5.2: split the feature map obtained in step 4 into windows, with each window size being , a total of non-overlapping windows; Step 5.3: The feature map obtained in step 4 is Scale block, each window has Feature blocks; Step 5.4, the size obtained in step 5.3 is The feature block is transformed into a size of One-dimensional vector of Feature blocks, converted to size The vector matrix of ; Step 5.5: Substitute the The vector matrix is input into the BLOCK module to perform self-attention learning within and between windows of Stage 1. It is iterated twice to train the learning parameters of Stage 1 and output the learning parameters of Stage 1. Scale feature map.
[0033] Step 6: The feature map of Stage 1 obtained in step 5 is first cascaded, then segmented, and then scaled and flattened into vectors. The block module is input to perform self-attention learning within and between windows of Stage 2. It is iterated 4 times to obtain the learning parameters of Stage 2 and output the Stage 2 phase. Scale feature map. The specific steps include: Step 6.1: Convert the value obtained in step 5 to Cascade the feature maps of different scales and then segment the feature maps; Step 6.2: split the feature map obtained in step 6.1 into windows, with each window size being , a total of non-overlapping windows; Step 6.3: perform the following operations on the feature map obtained in step 6.1: Scale blocks, so that each window divided in step 6.2 has Feature blocks; Step 6.4, the scale size obtained in step 6.3 is The feature block is transformed into a size of One-dimensional vector of Feature blocks, converted to size The vector matrix of ; Step 6.5, change the size obtained in step 6.4 to The vector matrix is input into the BLOCK module to perform self-attention learning within and between windows of Stage2. It is iterated 4 times to train the learning parameters of Stage2 and output the learning parameters of Stage2. Scale feature map.
[0034] In this embodiment, Figure 4 shows the feature map segmentation process. First, use a A window of size 2k is slid on the feature map with a step size of 2k (k is the scale of the feature map before segmentation); each window is divided into 4 regions, and the same region of each window is classified as a sub-image, so that the entire feature map is divided into 4 sub-images; then, the 4 sub-images are cascaded and normalized in the depth direction, and then the 4 sub-images are converted into 2 in the depth direction using full connection. After the feature map is segmented, the length and width of the original feature map are halved and the depth is doubled.
[0035] Step 7: The feature map of Stage 2 obtained in step 6 is first cascaded, then segmented, and then scaled and flattened into vectors, and then input into the BLOCK module to perform self-attention learning in the Stage 3 window, iterate twice, obtain the learning parameters of Stage 3, and output the feature vector of Stage 3. The specific steps include: Step 7.1: Convert the value obtained in step 6 to The scaled feature maps are cascaded and then the feature maps are segmented; Step 7.2: perform window segmentation on the feature map obtained in step 7.1, with each window size being , a total of non-overlapping windows; Step 7.3: perform the segmentation feature map obtained in step 7.1 on Scale blocks, so that each window divided in step 7.2 has Feature blocks; Step 7.4, the scale size obtained in step 7.3 is The feature block is transformed into a size of One-dimensional vector of Feature blocks, converted to size The vector matrix of ; Step 7.5, change the size obtained in step 7.4 to The vector matrix is input into the BLOCK module to perform self-attention learning in the Stage3 window of the present invention, iterate twice, train the learning parameters of Stage3, and output the feature vector of Stage3.
[0036] Step 7.6: Repeat steps 1.2 to 7.5 to obtain The classification results of images until The training ends when , and a general framework for color image classification in small datasets is obtained.
[0037] In order to further verify the technical effect of the framework obtained by the method of the present invention, the number of training samples was adjusted from small to large, and the image classification performance of the framework obtained by the method of the present invention was compared with that of the prior art. The specific control test conditions and control scheme are as follows: The experiment uses a 3090 graphics card for training, and the batch size is set to 128. Software environment configuration: Pytorch library and Pytorch visual model library (timm); the initial learning rate is 0.01, the minimum learning rate is set to 2e-4, and the weight decay is set to 5e-4. The SGD optimizer is used with a momentum of 0.9. The learning rate is adjusted using cosine annealing, and Tmax is set to 60. A total of 300 epochs are trained.
[0038] The number of images in the training sample set was expanded from 25,000 to 50,000. Figure 5 The comparison chart of the classification performance between the framework obtained by the method of the present invention and the prior art is shown. Figure 5 It can be seen that in the case of a small training set, the image classification performance of the framework obtained by the method of the present invention is significantly improved compared with the ViT and Swin transformer frameworks, which verifies the feasibility of the framework method of the present invention.
[0039] In order to further verify the feasibility of the framework method of the present invention, the experiment selected parameters of specific image classification methods based on CNN, ViT and Swintransformer as the framework, including three indicators: parameter quantity (Params (M)), computational complexity (FLOPs (G)) and evaluation accuracy (Top-1 Acc. (%)). These methods are ResNet5, ViT-B, CrossViT-B, DeiT-B and Swin-B.
[0040] It can be seen from Table 1 that the number of parameters and the amount of calculation in the embodiment of the present invention are significantly lower than those in the specific methods based on the CNN, Vision Transformer and Swin Transformer frameworks, and a higher classification accuracy is achieved.
[0041] Table 1 Experimental results of various methods on CIFAR 10 dataset
[0042] Example 2 A general framework construction method for small data set color image classification, specifically: Step 1, preprocessing sample image set; Step 2, performing channel decomposition processing on the images in the sample image set obtained after preprocessing in step 1; Step 3, performing a layer of Haar transform on each channel feature map obtained in step 2; Step 4, cascading the feature maps obtained in step 3 into one feature map; Step 5: Build a BLOCK module, divide the feature map obtained in step 4 into blocks, flatten it into column vectors, and input it into the BLOCK module for Stage 1 processing; Step 6: The feature map of the Stage 1 obtained in Step 5 is first cascaded, then segmented, and then scaled and flattened into vectors and input into the BLOCK module for Stage 2 processing; In step 7, the feature map of Stage 2 obtained in step 6 is first cascaded, then segmented, and then scaled and flattened into vectors and input into the BLOCK module for Stage 3 processing.
[0043] Example 3 A general framework construction method for small data set color image classification, specifically: Step 1, preprocessing sample image set; Step 1.3, for i The image data is enhanced for the first image. i The image data enhancement processing methods for the image are random geometric transformation, brightness adjustment, random noise addition and random cropping. Among them, the random geometric transformation includes random rotation, random inversion, scaling and translation, and the image size is kept Among them, the rotation range of random rotation is ; The random scaling is ; The degree of translation is ; Brightness parameter is ; The noise coefficient of random noise is .
[0044] Step 2, performing channel decomposition processing on the images in the sample image set obtained after preprocessing in step 1; Step 3, performing a layer of Haar transform on each channel feature map obtained in step 2; Step 4, cascading the feature maps obtained in step 3 into one feature map; Step 5: Build a BLOCK module, divide the feature map obtained in step 4 into blocks, flatten it into column vectors, and input it into the BLOCK module for Stage 1 processing; Step 6: The feature map of the Stage 1 obtained in Step 5 is first cascaded, then segmented, and then scaled and flattened into vectors and input into the BLOCK module for Stage 2 processing; In step 7, the feature map of Stage 2 obtained in step 6 is first cascaded, then segmented, and then scaled and flattened into vectors and input into the BLOCK module for Stage 3 processing.
[0045] Step 1.1: Select n images of size N×M×3 as the training sample set ; Step 1.2: Set the training sample set obtained in step 1.1 The i Put an image on the CPU, , use the memory-contiguous array num to store the image.
Claims
1. A general framework construction method for small data set color image classification, characterized in that: Specifically: Step 1, preprocessing sample image set; Step 2, performing channel decomposition processing on the images in the sample image set obtained after preprocessing in step 1; Step 3, performing a layer of Haar transform on each channel feature map obtained in step 2; Step 4, cascading the feature maps obtained in step 3 into one feature map; Step 5: Build a BLOCK module, divide the feature map obtained in step 4 into blocks, flatten it into column vectors, and input it into the BLOCK module for Stage 1 processing; Step 6: The feature map of the Stage 1 obtained in Step 5 is first cascaded, then segmented, and then scaled and flattened into vectors and input into the BLOCK module for Stage 2 processing; In step 7, the feature map of Stage 2 obtained in step 6 is first cascaded, then segmented, and then scaled and flattened into vectors and input into the BLOCK module for Stage 3 processing.
2. The general framework construction method for small data set color image classification according to claim 1 is characterized in that: The specific process of step 1 is: Step 1.1, select n images of size The images are used as training sample sets ; Step 1.2: Set the training sample set obtained in step 1.1 The i Put an image on the CPU, , use the memory-contiguous num array to store the image; Step 1.3, for i The image data is enhanced for the first image. i The image data enhancement processing methods for the image are random geometric transformation, brightness adjustment, random noise addition and random cropping. Among them, the random geometric transformation includes random rotation, random inversion, scaling and translation, and the image size is kept ; Among them, the rotation range of random rotation is ; The scaling degree of random scaling is ; The degree of translation is ; Brightness parameter is ; The noise coefficient of random noise is .
3. The general framework construction method for small data set color image classification according to claim 1, characterized in that: The specific process of step 2 is: The first i The sample image is decomposed into R, G, and B channels to obtain three channel feature maps of R, G, and B. The sizes of the three channel feature maps are .
4. The general framework construction method for small data set color image classification according to claim 3 is characterized in that: The specific process of step 3 is: Step 3.1: Perform a Haar transform on the R channel feature map obtained in step 2 to obtain the approximate feature size of the R channel. The feature map of the R channel has a horizontal size of The detail feature map of the R channel has a vertical dimension of The size of the detail feature map and the diagonal direction of the R channel is Detailed feature map; Step 3.2: Perform a Haar transform on the G channel feature map obtained in step 2 to obtain the approximate feature size of the G channel. The feature map of G channel has a horizontal size of The detail feature map of G channel has a vertical dimension of The size of the detail feature map and the diagonal direction of the G channel is Detailed feature map; Step 3.3, perform a layer of Haar transform on the B channel feature map obtained in step 2, and obtain the approximate feature size of the B channel as The feature map of the B channel has a horizontal size of The detail feature map of the B channel has a vertical dimension of The size of the detail feature map of and the diagonal direction of the B channel is After the previous steps, 12 feature maps are finally obtained.
5. The general framework construction method for small data set color image classification according to claim 4 is characterized in that: The specific process of step 4 is: cascade the 12 feature maps obtained in step 3 into a feature map with a size of .
6. The general framework construction method for small data set color image classification according to claim 5, characterized in that: The specific process of step 5 is: Step 5.1, build a BLOCK module, including an intra-window self-attention learning module and an inter-window self-attention learning module, wherein the intra-window self-attention learning module is sequentially composed of a normalization layer, a window multi-head self-attention learning layer, an LN layer, and a multi-layer perception layer; the inter-window self-attention learning module is sequentially composed of a normalization layer, a moving window multi-head self-attention learning, an LN layer, and a multi-layer perception layer; Step 5.2: split the feature map obtained in step 4 into windows, with each window size being , a total of non-overlapping windows; Step 5.3: Perform the feature map obtained in step 4 Scale block, each window has Feature blocks; Step 5.4, the size obtained in step 5.3 is The feature block is transformed into a size of One-dimensional vector of Feature blocks, converted to size The vector matrix of ; Step 5.5: Substitute the The vector matrix is input into the BLOCK module to perform self-attention learning within and between windows of Stage 1. It is iterated twice to train the learning parameters of Stage 1 and output the learning parameters of Stage 1. Scale feature map.
7. The general framework construction method for small data set color image classification according to claim 6, characterized in that: The specific process of step 6 is: Step 6.1: Output the Stage 1 obtained in step 5 The scale feature maps are cascaded and then the feature maps are segmented; Step 6.2: perform window segmentation on the feature map obtained in step 6.1, with each window size being , a total of non-overlapping windows; Step 6.3: perform the following operations on the feature map obtained in step 6.1: Scale blocks, so that each window divided in step 6.2 has Feature blocks; Step 6.4, the scale size obtained in step 6.3 is The feature block is transformed into a size of One-dimensional vector of Feature blocks, converted to size The vector matrix of ; Step 6.5, change the size obtained in step 6.4 to The vector matrix is input into the BLOCK module to perform self-attention learning within and between windows of Stage2. It is iterated 4 times to train the learning parameters of Stage2 and output the learning parameters of Stage2. Scale feature map.
8. The general framework construction method for small data set color image classification according to claim 7, characterized in that: The specific process of step 7 is: Step 7.1: Output the framework Stage2 obtained in step 6 Scale feature map, cascade, and then segment feature map; Step 7.2: perform window segmentation on the feature map obtained in step 7.1, with each window size being , a total of non-overlapping windows; Step 7.3: perform the following operations on the feature map obtained in step 7.1: Scale blocks, so that each window divided in step 7.2 has Feature blocks; Step 7.4, the scale size obtained in step 7.3 is The feature block is transformed into a size of One-dimensional vector of Feature blocks, converted to size The vector matrix of ; Step 7.5, change the size obtained in step 7.4 to The vector matrix is input into the BLOCK module to perform self-attention learning in the Stage3 window, iterate twice, train the learning parameters of Stage3, and output the feature vector; Step 7.6: Repeat steps 1.2 to 7.5 to obtain The classification results of images until The training ends when , and a general framework for color image classification in small datasets is obtained.