Image classification width learning system and training method thereof and image classification method

Abstract

The present invention discloses an image classification width learning system and its training method and image classification method. The image classification width learning system includes an input layer, a feature extraction layer, a downsampling module, a convolution module, a flattening layer, an enhancement block and an output layer; the feature extraction layer includes a plurality of feature blocks connected in sequence, each feature block includes a large value ratio attention block and an encoder connected to the output of the large value ratio attention block, and the introduced large value ratio attention technology learns the relationship between channels, realizes the use of discriminative features to measure the importance of a given channel, and can improve the accuracy of image classification by the image classification width learning system. The training method of the image classification width learning system combines the ideas of self-supervision and cross-domain learning, and improves the generalization ability of the model by iteratively adding training samples selected from an unlabeled test set, whose pseudo labels are the previous prediction results of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to a width learning system for image classification and a training method thereof and an image classification method. Background Art

[0002] Image classification is one of the fundamental core problems in image understanding, which is the process of classifying images into different categories based on their features and specific rules. In recent years, many methods have been proposed to perform this task. Among these methods, deep learning methods, such as Deep Convolutional Neural Network (DCNN), have been widely used for image classification tasks due to their advantages in classification accuracy. Popular DCNNs include VGG16, ResNet50, SENet, etc. Due to their excellent ability to learn high-level features, DCNNs are widely used for image classification and have achieved excellent results on complex datasets such as CIFAR-100. However, due to the presence of many hidden layers in DCNNs and the large number of parameters that need to be updated during training, the training process of DCNNs is very time-consuming. In addition, some studies have revealed that the test time (also called inference time in some cases) of DCNNs is also quite long. However, we know that short test time is a desirable feature for machine learning models because it can achieve faster predictions, which is critical in many real-world applications.

[0003] To overcome the long training and testing time of DCNN, the Broad Learning System (BLS) was proposed as an alternative to the deep structure. BLS is a flat structure with only one hidden layer. The weights and biases used to form feature nodes and enhancement nodes are randomly generated in BLS and will not change after generation, and the output weights in BLS are calculated by ridge regression without iteration. Although BLS is designed to run on CPU instead of GPU, BLS is very fast to train and has competitive testing speed. However, compared with state-of-the-art DCNNs (such as ResNet50), BLS's feature learning ability is generally weaker because it has only one hidden layer and most weights and biases are randomly generated. Therefore, there is still a gap between the accuracy of BLS and state-of-the-art DCNNs in image classification, especially when both methods are applied to complex image datasets such as CIFAR-100.

[0004] In order to further improve the classification performance of BLS, many new BLS variants have been proposed, such as EFBLS, CNNBLS, K-means-BLS, and MFBLS. We divide these models into two categories: CNN-based BLS variants and non-CNN-based BLS variants. For most non-CNN-based BLS variants, traditional feature engineering is a necessary and critical step. Features need to be manually selected and extracted based on domain knowledge, and the accuracy of the model depends largely on the results of feature engineering. Therefore, this is a time-consuming, technical, and specific process for each dataset.

[0005] Unlike non-CNN-based BLS variants, CNN-based BLS variants do not require feature engineering due to their built-in CNN. Only the original RGB images need to be used for end-to-end learning. Generally, the role of CNN in BLS is to improve feature learning capabilities. In previous studies, people have tried to introduce new features, such as CNN features, to improve the performance of BLS models. For example, in CNNBLS, convolution and maximum pooling are used to extract image features, and then principal component analysis (PCA) is used to reduce the dimension of the features, and the results are used as the final CNN features. Although the above CNN-based BLS variants do improve classification accuracy, the improvement is limited, especially for complex image datasets. The reason may be that the gradient descent process is not applied to the entire structure, so these models cannot fully learn the features in the dataset. In addition, due to the introduction of CNN and other modules, the testing time of CNN-based BLS variants is inevitably longer than that of basic BLS. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide an image classification width learning system and its training method and image classification method to solve the technical problem of improving the accuracy of BLS for image classification and allowing BLS to maintain a low training / testing time.

[0007] The image classification width learning system of the present invention comprises an input layer, a feature extraction layer, a downsampling module, a convolution module, a flattening layer, an enhancement block and an output layer;

[0008] The feature extraction layer includes a number of feature blocks connected in sequence, each feature block includes a large-value ratio attention block and an encoder connected to the output of the large-value ratio attention block, the input layer inputs the image to the large-value ratio attention block of the first feature block, and the output of the large-value ratio attention block in each feature block is used as the input of the large-value ratio attention block in the next feature block; except for the last feature block, the encoder module of each feature block outputs the extracted features to a downsampling module respectively; the output of each downsampling module is connected to a convolution module, and the output of each convolution module and the output of the last feature block are respectively connected to the flattening layer; the flattening layer converts the scale of the output features of each convolution module into a whole. The flattened layer is flattened to the same size as the output feature of the last feature block, the nodes in the flattened layer are feature nodes, and the feature nodes from the same feature block in the flattened layer are classified as a feature node group; the output of the flattened layer is connected to the enhancement block, and the enhancement block includes an element-by-element addition layer and an enhancement node layer. The element-by-element addition layer adds the feature nodes of the corresponding order in each feature node group in the flattened layer and connects them to the enhancement nodes in the enhancement layer, and each enhancement node in the enhancement layer is independently connected to the output layer, and the enhancement layer randomly selects a fixed number of enhancement nodes and the connection of the output layer does not work; the feature nodes of the flattened layer are also connected to the output layer, and the weights connecting the feature nodes and the output layer are set to zero;

[0009] The large-valued ratio attention block includes several Conv-BN sequences, a block-level loss layer connecting adjacent Conv-BN sequences, a GPP layer connected to the last Conv-BN sequence, an average pooling layer with the product of the output of the GPP layer and the output of the last Conv-BN sequence as input, and a random loss layer with the output of the average pooling layer as input. The Conv-BN sequence is composed of a convolutional layer and a batch normalization layer connected after the convolutional layer; the GPP layer is defined as follows:

[0010]

[0011] Where Z is the input of GPP, H and W are the height and width of input Z, and z i ∈Z, t is the preset threshold, and [·] represents the indicator function.

[0012] Furthermore, the encoder includes an average pooling layer, a first Transformer module, a second Transformer module and a reshaping layer connected in sequence, and the Transformer module consists of two layer normalization layers, a multi-head attention layer and a feed-forward layer.

[0013] The training method of the image classification width learning system in the present invention comprises the following steps:

[0014] 1) Create a training set: D orig=(X tr ,Y tr ), where the training sample set s is the number of samples, d is the dimension of the samples; represents the label, where c is the number of categories;

[0015] make is the test set, the test set is unlabeled, and Each sample belongs to one of c categories;

[0016] 2) Using the training samples in the training set to train the image classification width learning system;

[0017] 3) Apply the trained image classification width learning system to the test set X te , to obtain the predicted label set and its confidence score set

[0018] 4) From P te(j) Select samples whose confidence score is not less than the threshold τ from X te Form a sample set and mark it as pseudo label of

[0019] 5) Merge into X tr ,Will Merge into Y tr , to update the training set;

[0020] 6) Use the updated training set to train the image classification width learning system for one epoch;

[0021] 7) Determine whether the training meets the early stopping condition: verify that the loss does not improve for 10 consecutive times. If the early stopping condition is met, interrupt the training and save the best model encountered during the training process; otherwise, repeat steps 3)-7).

[0022] Furthermore, during the training process, all weights in the image classification width learning system are optimized using the AdamW algorithm.

[0023] The present invention also discloses an image classification method, which comprises inputting an image to be classified into an optimal image classification width learning system obtained by the above training method, and obtaining an image classification result by the optimal image classification width learning system.

[0024] Beneficial effects of the present invention:

[0025] 1. The image classification width learning system of the present invention proposes to use feature blocks (FB) to extract image features. The feature blocks introduce a technology called Proportion of LargeValues ​​Attention (PLVA) to learn the relationship between channels. By using this technology, it is possible to use discriminative features to measure the importance of a given channel, which can improve the accuracy of image classification by the image classification width learning system.

[0026] 2. The image classification width learning system of the present invention also applies an encoder module in its feature block to enhance the image classification width learning system's ability to capture global features.

[0027] 3. The image classification width learning system in the present invention also proposes an Enhancement Block (EB). In this simple but effective module, the outputs of feature nodes from different groups in the flattening layer are added by element-by-element addition and passed through the enhancement layer. This module acts as an enhancement mapping, which can better learn the input features and avoid underfitting on complex data sets.

[0028] 4. The image classification width learning system in the present invention has both BLS characteristics and gradient descent characteristics. On the one hand, the cascaded FB and EB in the system act as feature mapping and enhancement mapping respectively, thus giving the model the advantages of BLS; on the other hand, the AdamW algorithm is applied to the entire image classification width learning system, and the gradient descent process occurs in the entire model, and all weights and biases of the network can be updated to the optimal value.

[0029] 5. The image classification width learning system in the present invention achieves 97.04%, 94.10% and 75.64% classification accuracy on the SVHN, CIFAR-10 and CIFAR-100 datasets, respectively, which is better than all compared BLS variants. The time complexity of the proposed model is very low and has the shortest test time.

[0030] 6. The training method of the image classification width learning system in the present invention defines a training cycle as an epoch. In each cycle, the model is only used to generate pseudo labels; the "refined" training cycle is helpful for the training of the model.

[0031] 7. The training method of the image classification width learning system of the present invention adds the test set samples with confidence scores higher than the threshold τ to D in each cycle. orig This method can avoid duplication of samples and labels in the training set.

[0032] 8. The training method of the image classification width learning system in the present invention combines the ideas of self-supervision and cross-domain learning. By iteratively adding training samples selected from unlabeled test sets, whose pseudo labels are the previous prediction results of the model, this training method improves the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 The architecture of the width learning system for image classification. The gray nodes in the figure represent units that are temporarily discarded.

[0034] Figure 2 is the structure of PLVA, where C refers to the number of channels.

[0035] Figure 3 is the structure of the encoder, where C refers to the number of channels.

[0036] Figure 4 The figure shows the effect of different numbers of FBs on the performance of the image classification width learning system. There are 3 Conv-BN sequences in each FB. Among them, (a) shows the change in the accuracy of STTL-BLS when the number of FBs changes; (b) and (c) respectively show the changes in the training time and test time of STTL-BLS when the number of FBs changes.

[0037] Figure 5 The figure shows the impact of different numbers of Conv-BN sequences on model performance in PLVA. In the model evaluation, four FBs were added. Among them, (a) shows the change in accuracy of STTL-BLS when the number of Conv-BN changes; (b) and (c) respectively show the changes in training time and test time of STTL-BLS when the number of Conv-BN changes.

[0038] Figure 6 This is a graph showing the impact of EB on the number of training rounds during early stopping.

[0039] Figure 7 This is a graph showing the relationship between the number of iterations and the loss.

[0040] Figure 8 is the impact of the threshold τ on the classification accuracy of the model for different data sets. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0042] like Figure 1 As shown, the image classification width learning system in this embodiment includes an input layer, a feature extraction layer, a downsampling module, a convolution module, a flattening layer, an enhancement block (Enhancement Block, EB) and an output layer.

[0043] The feature extraction layer includes several feature blocks (FB) connected in sequence, each of which includes a proportion of large values ​​attention block (PLVA) and an encoder connected to the output of the proportion of large values ​​attention block. The input layer inputs the image to the proportion of large values ​​attention block of the first feature block, and the output of the proportion of large values ​​attention block in each feature block is used as the input of the proportion of large values ​​attention block in the next feature block. Except for the last feature block, the encoder module of each feature block outputs the extracted features to a downsampling module for connection. Since each FB will halve its input image and output it to the next FB, the number of FBs connected in series in the system is limited, which can prevent the training time from being too long.

[0044] The output of each downsampling module is connected to a convolution module, and the output of each convolution module and the output of the last feature block are connected to the flattening layer respectively. The flattening layer flattens the size of the output features of each convolution module to the same size as the output features of the last feature block. The nodes in the flattening layer are feature nodes, and the feature nodes from the same feature block in the flattening layer are classified as a feature node group. The output of the flattening layer is connected to the enhancement block, which includes an element-by-element addition layer and an enhancement node layer. The element-by-element addition layer adds the feature nodes of corresponding order in each feature node group in the flattening layer and connects them to the enhancement nodes in the enhancement layer. Each enhancement node in the enhancement layer is independently connected to the output layer, and the enhancement layer randomly selects a fixed number of enhancement nodes and the connection with the output layer does not work. The feature nodes of the flattening layer are also connected to the output layer, and the weights connecting the feature nodes and the output layer are set to zero; this simplifies the structure of the model, thereby reducing the computational cost.

[0045] like Figure 2As shown, the large-value ratio attention block in this embodiment includes several Conv-BN sequences, a block-level loss layer connecting adjacent Conv-BN sequences, a GPP layer connected to the last Conv-BN sequence, an average pooling layer with the product of the output of the GPP layer and the output of the last Conv-BN sequence as input, and a random loss layer with the output of the average pooling layer as input. The input and output of the large-value ratio attention block are both 3D tensors. The Conv-BN sequence consists of a convolutional layer and a batch normalization (Batch Normalization, BN) layer connected after the convolutional layer. In a specific implementation, all convolutional layers in the same PLVA can use the same settings: step size = 1, zero padding and ReLU activation; the number of filters of all convolutional layers in the same PLVA is the same, and the size of the convolution kernel is set to 3×3 to ensure stable performance; of course, in different embodiments, the parameter settings can also be adjusted as needed. The BN layer is used to adjust the distribution of data. In addition to the connection function, the block-level loss layer can also prevent overfitting, and the hyperparameter of the block-level loss layer is the loss rate p_b. The GPP layer is defined as follows:

[0046]

[0047] Where Z is the input of GPP, H and W are the height and width of input Z, and z i ∈Z, t is a preset threshold, and [·] represents the indicator function. GPP is a very different form of pooling because it uses the proportion of large values ​​(PLV), which describes the proportion of large values ​​in the input. It actually enables PLVA to use discriminative features to weigh the importance of a given channel by emphasizing useful features and suppressing less useful features in the feature map, and is therefore crucial to the excellent accuracy of the model.

[0048] Since the average pooling layer added in PLVA halves the height and width of the input, the number of FBs in series in the image classification width learning system is limited, which can prevent the training time from being too long. In the specific implementation, the key hyperparameters of the average pooling layer can be set empirically: window size: 2×2; step size: 2; padding: no padding.

[0049] like Figure 3As shown, in this embodiment, the encoder includes an average pooling layer, a first Transformer module (TE), a second Transformer module (TE) and a reshaping layer connected in sequence, and the Transformer module consists of two layer normalization layers (LN), a multi-head attention layer (MHA) and a feedforward layer. The LN in the encoder can reduce internal covariate shift. MHA is used to obtain global receptive field features, and the feedforward layer is used to make each MHA like a separate small model that can be trained. Unlike PLVA, which focuses on the relationship between channels, the encoder pays more attention to the global receptive field.

[0050] In this embodiment, the image classification width learning system introduces an enhancement block to better learn the input features and avoid underfitting on complex data sets. In order to ensure high training and testing speeds, the designed enhancement block is as simple as possible. The enhancement block consists of an element-by-element addition and a so-called enhancement layer, which together serve as an enhanced map. Element-by-element addition merges mapping features (feature nodes) from different groups in the flatten layer. The enhancement layer is implemented by a random deactivation layer, and for a given unit, the random deactivation layer uses a fixed probability p_t to determine whether the enhancement node and its related connections work. Each enhancement node is independent of each other, and the discarded enhancement nodes are different in each training iteration, which actually leads to different model structures. Therefore, the enhancement map changes with each iteration, and this EB structure not only guarantees speed, but also reduces underfitting.

[0051] The training method of the image classification width learning system in this embodiment includes the following steps:

[0052] 1) Create a training set: D orig =(X tr ,Y tr ), where the training sample set s is the number of samples, d is the dimension of the samples; represents the label, where c is the number of categories;

[0053] make is the test set, the test set is unlabeled, and Each sample belongs to one of c categories;

[0054] 2) Using the training samples in the training set to train the image classification width learning system;

[0055] 3) Apply the trained image classification width learning system to the test set X te , to obtain the predicted label set and its confidence score set

[0056] 4) From P te(j) Select samples whose confidence score is not less than the threshold τ from X te Form a sample set and mark it as pseudo label of

[0057] 5) Merge into X tr ,Will Merge into Y tr , to update the training set;

[0058] 6) Use the updated training set to train the image classification width learning system for one epoch;

[0059] 7) Determine whether the training meets the early stopping condition: verify that the loss does not improve for 10 consecutive times. If the early stopping condition is met, the training is interrupted and the best model encountered during the training process is saved (the model is the image classification width learning system); otherwise, repeat steps 3)-7).

[0060] During the training process, all weights in the image classification width learning system are optimized using the AdamW algorithm.

[0061] The image classification method in this embodiment includes inputting the image to be classified into the optimal image classification width learning system obtained by the above training method, and obtaining the image classification result by the optimal image classification width learning system.

[0062] The following is an experimental verification of the image classification width learning system (named: STTL-BLS) trained by the above training method:

[0063] (1) Dataset and Experimental Settings

[0064] The experiment evaluates the effectiveness and efficiency of STTL-BLS on three commonly used datasets: SVHN, CIFAR-10, and CIFAR-100. In order to facilitate comparison with other SOTA methods, the experiment reconstructs some of these datasets and divides each dataset into training, validation, and test sets. The original SVHN dataset consists of a training set, an extra set, and a test set, with the number of samples in each subset being 73257, 531131, and 26032, respectively. We use the training set to train the model and randomly select 6000 samples from the extra set to form the validation set. The number of samples in each category in the validation set is the same. CIFAR-10 contains 50,000 training samples and 10,000 test samples. We randomly select 5,000 samples from the training set to form the validation set. Similar to CIFAR-10, CIFAR-100 also consists of 50,000 training samples and 10,000 test samples. The difference is that CIFAR-100 has 100 categories, which greatly increases the difficulty of classification. There are 500 training samples and 100 test samples in each category. We randomly select 5000 images from the training set to form the validation set, and make the number of samples for each category in the validation set the same. The details of these datasets can be found in Table 1. Before training, we preprocessed the input images. For the SVHN dataset, we divided the pixel values ​​of the images by 255 to make them in the range [0,1]. For CIFAR-10 and CIFAR-100, we normalized the data using the mean and standard deviation of the channels.

[0065] Table 1: Information about the datasets used in the experiment

[0066]

[0067] The experiments were implemented in Python using pytorch and conducted using an Intel Xeon E5-2678 CPU with 128G memory and an NVIDIA GeForce RTX 3080 GPU. The key hyperparameters of the image classification width learning system are listed in Table 2.

[0068] Table 2: Hyperparameters set for different datasets

[0069]

[0070] (2) Verification of feature blocks

[0071] Experiments are conducted to evaluate the impact of FB on model performance. Accuracy (Acc), training time (Train) and testing time (Test) are selected as evaluation indicators.

[0072] First, the relationship between the number of cascaded FBs and model performance is experimentally studied. Since each FB reduces the height and width of the input image by half, and the size of the input image is 32×32, the number of feature blocks can only range from 1 to 4. For fair comparison, in this evaluation, we make all FBs contain 3 Conv-BN sequences. Other key hyperparameters in FB are shown in Table 3. The evaluation results are shown in Figure 4 As shown. Figure 4 (a) It can be seen that on all datasets, the accuracy increases with the number of cascaded FBs, but the percentage increase of the accuracy slows down with the increase of the number of FBs. For example, on CIFAR-10, the accuracy of cascading 2 FBs, 3 FBs, and 4 FBs is 4.57pp, 0.82pp, and 0.21pp higher than that of cascading 1 FB, 2 FBs, and 3 FBs, respectively. The accuracy of 3 FBs is almost the same as that of 4 FBs. Figure 4 (b) and Figure 4 (c) shows that the training and testing time of the model increases with the increase in the number of FBs, which is consistent with our expectation: increasing the number of blocks in the model tends to increase the training and testing time. When the model has 4 FBs, both the training and testing time are the longest. Similar to the increase in accuracy, the rate of increase in testing time also slows down with the increase in the number of FBs. For example, on CIFAR-10, the test time of cascading 2 FBs, 3 FBs, and 4 FBs is 0.5pp, 0.2pp, and 0.0pp higher than the test time of cascading 1 FB, 2 FBs, and 3 FBs, respectively.

[0073] Second, the experiments evaluate the performance of the model with different numbers of Conv-BN sequences in the PLVA of FBs. In the experiments, 5 Conv-BNs are gradually added for performance evaluation. The hyperparameter settings of Conv-BN follow those in Table 3. We consider the case where there are 3 FBs in the model, which is a typical value in many scenarios. Figure 5 The evaluation results are shown. Figure 5 It can be seen that the accuracy increases rapidly first and then rises slowly when there are more than 2 Conv-BNs. However, both the training time and the test time increase significantly. The appropriate number of Conv-BNs can be selected by considering the balance between the time complexity and classification accuracy of different classification tasks. We propose that STTL-BLS contains 3 FBs, each containing 5 Conv-BNs, to achieve a balance between time complexity and classification accuracy in most cases.

[0074] Table 3: Some key parameters used in FB

[0075]

[0076] Third, the performance of STTL-BLS with PLVA module and with SE block (i.e., replacing PLVA with SE) is compared experimentally to illustrate the superiority of our PLVA in terms of accuracy. Table 4 lists the comparison results, and all the indicators listed in the table are the average of 10 runs. The results show that STTL-BLS(SE) takes less training time than STTL-BLS(PLVA) and slightly shorter testing time than STTL-BLS(PLVA), but has lower accuracy than STTL-BLS(PLVA) on all datasets, especially for CIFAR-100. This suggests that STTL-BLS(PLVA) should be chosen if we pursue high accuracy. Note that the shorter training time of STTL-BLS(SE) is mainly because the introduction of SE block makes it easier for training to meet the early stopping condition. Reaching the early stopping condition too early may prevent the model from achieving its full potential and lead to underfitting.

[0077] Table 4: Performance comparison of STTL-BLS with SE block (STTL-BLS(SE)) and STTL-BLS with PLVA module (STTL-BLS(PLVA)). Train represents the training time (seconds), T est Indicates the test time (seconds).

[0078]

[0079] (3) Enhanced Block Verification

[0080] Ablation experiments are designed to verify the effectiveness of EB: the performance of STTL-BLS without EB and with EB is compared on all three datasets. The experimental results are shown in Table 5. It can be seen that after adding EB, the accuracy of STTL-BLS on all datasets has been improved. In addition, both training and testing time have decreased, which comprehensively improves the performance of the model. The element-by-element addition in EB can merge feature nodes from different FBs, while the enhancement layer in EB can further simplify the structure of EB by deleting some nodes. Without EB, feature nodes from all FBs must be directly connected to the output layer of the model, which may lead to an increase in computational complexity. Therefore, it can be seen from Table 5 that the training and testing time with EB is shorter than that without EB; all indicators listed in the table are the average of 10 runs.

[0081] It can also be seen from Table 5 that after adding EB, the accuracy is significantly improved for the CIFAR-100 dataset (2.12 percentage points), but the accuracy improvement is smaller for the SVNH (0.38 percentage points) and CIFAR-10 (0.12 percentage points) datasets. This may be because the number of categories in CIFAR-100 is much larger than that of SVNH and CIFAR-10, and EB can better help STTL-BLS learn discriminative features and avoid overfitting, especially for more complex datasets such as CIFAR-100. Figure 6 The figure provides evidence for the above conclusion. It compares the number of epochs at early stopping with and without EB in the model. From this figure, we can see that adding EB may increase the number of training epochs at early stopping. For CIFAR-100, the number of training epochs reaching early stopping is the highest when EB is added. This suggests that EB helps the model to fully realize its potential, thus avoiding underfitting as much as possible.

[0082] Table 5: Ablation experiment results for evaluating the enhancement block (EB), where Train represents the training time (seconds) and Test represents the testing time (seconds).

[0083]

[0084] Figure 7 The training and validation losses of the model without and with EB are shown to illustrate the performance improvement brought by EB. In this experiment, 200 epochs are run and the results are presented using CIFAR-100 as an example. Figure 7 It can be seen that as training progresses, the training loss of the model with EB is slightly lower than that of the model without EB. In addition, the gap between the training and validation losses of the model with EB is smaller than that of the model without EB. The smaller the gap between the training and validation loss curves, the better the model's generalization ability to new data. Figure 7 These features shown in indicate that the model with EB has better performance.

[0085] Note that our EB is different from the enhanced node cascade with random dropout (CEBLS-Dropout) described in CEBLS. The main differences are as follows: (1) When the enhancement layer in EB is implemented by a random dropout layer, only one layer is required. However, in CEBLS-Dropout, multiple random dropout layers in cascade form are required, which makes the model more complicated. (2) In EB, random dropout is implemented for each unit, while in CEBLS-Dropout, random dropout is implemented for each layer. (3) The weights in EB are optimized by the AdamW algorithm, while the weights in CEBLS-Dropout are calculated by the width-inverse ridge regression approximation. Experimental results of CEBLS show that the accuracy of CEBLS-Dropout on SVHN and CIFAR-10 is no higher than 91% and 68%, respectively, which is much lower than the STTL-BLS model with EB (97.04% and 94.10%, respectively).

[0086] Note that the so-called BLS-based model proposed by CNN-BLS also combines CNN and Adam algorithm. However, this model works differently from ours. The BLS-based model uses CNN for feature extraction, and the Adam algorithm is only used to update the weights of CNN. This means that gradient descent only occurs in CNN to update the initial weights for feature extraction. Similar to BLS and its variants, the BLS-based model still uses ridge regression learning to obtain the final output weights for pattern classification. Unlike the BLS-based model, we adopt the AdamW algorithm to optimize the entire STTL-BLS so that the model can be used with GPU in a similar way to DCNN. The CNN-based BLS and STTL-BLS are compared in Table 6, and all the metrics listed in the table are the average of 10 runs. From the results in this table, it can be seen that STTL-BLS outperforms the CNN-based BLS in terms of accuracy and test time.

[0087] Table 6: Performance comparison between the BLS-based model and STTL-BLS on different datasets. Train represents the training time (seconds), and Test represents the test time (seconds).

[0088]

[0089] (4) STTL Verification

[0090] The experiment constructed a STTL-BLS containing 3 FBs, each containing 5 Conv-BNs, to verify the effectiveness and efficiency of the STTL algorithm. All experiments follow the same settings in Tables 2 and 3. The threshold τ = 0.9. Table 7 shows the performance of STTL-BLS before and after applying STTL. All indicators listed in the table are the average of 10 runs. We can see that when applying STTL to all these datasets, the training time is slightly longer, but the accuracy of the model is improved. This shows that STTL can extract additional useful information from the unlabeled test set and promote the generalization ability of the model during training.

[0091] Table 7: Comparison of STTL-BLS performance with and without STTL strategy. Train represents the training time (seconds), and Test represents the test time (seconds).

[0092]

[0093] The hyperparameter τ in the STTL algorithm has an important impact on classification accuracy. This is because the pseudo-labels predicted by the model are not always correct. If the prediction is correct, then the corresponding sample will help learning; if the prediction is wrong, then this sample will hinder learning if it is fed into the model. Generally, the pseudo-labels of samples with higher confidence scores have a higher probability of being predicted correctly. However, too high confidence scores may lead to The number of samples selected in the set is reduced. This may result in less useful information being obtained from the test set. Therefore, the threshold τ needs to be fine-tuned. Figure 8 The classification accuracy of the model on different datasets varies with the threshold τ. Based on these experimental results, we empirically set τ to 0.9 to obtain satisfactory results on all datasets.

[0094] (5) Comparison with other models

[0095] The experiments also compare the accuracy of STTL-BLS with other state-of-the-art models listed in Table 8. All metrics listed in the table are averages of 10 runs. These models use the validation set to optimize hyperparameters. The top five models are BLS or its variants. The results in Table 8 show that the accuracy of our STTL-BLS is better than all BLS and its variants, which indicates that it can effectively improve the learning ability of BLS in image classification, especially on complex image datasets. For SVHN, STTL-BLS achieves an accuracy of 97.04%, which is 18.64pp, 17.93pp, 15.78pp, 7.34pp and 4.79pp higher than BLS, EFBLS, CNNBLS, K-means-BLS and MFBLS. For CIFAR-10, the model achieves an accuracy of 94.10%, which is 43.18pp, 41.98pp, 35.05pp, 18.05pp, and 13.07pp higher than BLS, EFBLS, CNNBLS, K-means-BLS, and MFBLS. For CIFAR-100, it achieves an accuracy of 75.64%, which is 49.43pp, 49.18pp, 46.85pp, 26.43pp, and 20.98pp higher than BLS, EFBLS, CNNBLS, K-means-BLS, and MFBLS.

[0096] In addition, the model also outperforms deep CNN networks such as Improved AlexNet, AlexNet, LeNet5, VGG16, SENet, MobileNetV2, GoogleNet, and CNN-iSSO-SGD in terms of accuracy. For ResNet50, our model can achieve comparable or better accuracy.

[0097] The training and testing times of these models are listed in Table 9. All metrics listed in the table are averages of 10 runs. For fair comparison, we used an early stopping strategy for all models without pre-training (i.e., all models were trained from the training set of our experimental dataset). Table 9 shows that STTL-BLS has the shortest testing time compared to all other models. The training time is also competitive, especially for deep CNN models.

[0098] Table 9: Training and testing time(s) of different models on SVHN, CIFAR-10, and CIFAR-100. Models below the dashed line are run on GPU.

[0099]

[0100] The purpose of this invention is to propose a model that can be used in image classification scenarios that require high accuracy and speed. As can be seen from Tables 8 and 9, some BLS-based models have shorter training time than STTL-BLS (e.g., the training time of BLS on CIFAR-100 is 175 seconds, while that of STTL-BLS is 1966 seconds), but their accuracy is much lower (e.g., the accuracy of BLS on CIFAR-100 is 26.21%, while that of STTL-BLS is 75.64%). The low accuracy makes the "fastness" of these models meaningless. Therefore, our model is a good choice for applications that require high accuracy and speed.

[0101] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. Image classification width learning system, characterized by: It includes input layer, feature extraction layer, downsampling module, convolution module, flattening layer, enhancement block and output layer; The feature extraction layer includes a number of feature blocks connected in sequence, each feature block includes a large-value ratio attention block and an encoder connected to the output of the large-value ratio attention block, the input layer inputs the image to the large-value ratio attention block of the first feature block, and the output of the large-value ratio attention block in each feature block is used as the input of the large-value ratio attention block in the next feature block; except for the last feature block, the encoder module of each feature block outputs the extracted features to a downsampling module respectively; the output of each downsampling module is connected to a convolution module, and the output of each convolution module and the output of the last feature block are respectively connected to the flattening layer; the flattening layer converts the scale of the output features of each convolution module into a whole. The flattened layer is flattened to the same size as the output feature of the last feature block, the nodes in the flattened layer are feature nodes, and the feature nodes from the same feature block in the flattened layer are classified as a feature node group; the output of the flattened layer is connected to the enhancement block, and the enhancement block includes an element-by-element addition layer and an enhancement node layer. The element-by-element addition layer adds the feature nodes of the corresponding order in each feature node group in the flattened layer and connects them to the enhancement nodes in the enhancement layer, and each enhancement node in the enhancement layer is independently connected to the output layer, and the enhancement layer randomly selects a fixed number of enhancement nodes and the connection of the output layer does not work; the feature nodes of the flattened layer are also connected to the output layer, and the weights connecting the feature nodes and the output layer are set to zero; The large-valued ratio attention block includes several Conv-BN sequences, a block-level loss layer connecting adjacent Conv-BN sequences, a GPP layer connected to the last Conv-BN sequence, an average pooling layer with the product of the output of the GPP layer and the output of the last Conv-BN sequence as input, and a random loss layer with the output of the average pooling layer as input. The Conv-BN sequence is composed of a convolutional layer and a batch normalization layer connected after the convolutional layer; the GPP layer is defined as follows: Where Z is the input of GPP, H and W are the height and width of input Z, and z i ∈Z, t is the preset threshold, [z i >t] is the indicator function.

2. The image classification width learning system according to claim 1, characterized in that: The encoder includes an average pooling layer, a first Transformer module, a second Transformer module and a reshaping layer connected in sequence, and the Transformer module consists of two layer normalization layers, a multi-head attention layer and a feed-forward layer.

3. A training method for the image classification width learning system according to claim 1 or 2, characterized in that: The following steps are involved: 1) Create a training set: D orig =(X tr ,Y tr ), where the training sample set s is the number of samples, d is the dimension of the samples; represents the label, where c is the number of categories; make is the test set, the test set is unlabeled, and Each sample belongs to one of c categories; 2) Using the training samples in the training set to train the image classification width learning system; 3) Apply the trained image classification width learning system to the test set X te , to obtain the predicted label set and its confidence score set 4) From P te(j) Select samples whose confidence score is not less than the threshold τ from X te Form a sample set and mark it as pseudo label of 5) Merge into X tr ,Will Merge into Y tr , to update the training set; 6) Use the updated training set to train the image classification width learning system for one epoch; 7) Determine whether the training meets the early stopping condition: verify that the loss does not improve for 10 consecutive times. If the early stopping condition is met, interrupt the training and save the best model encountered during the training process; otherwise, repeat steps 3)-7).

4. The training method of the image classification width learning system according to claim 3 is characterized by: During the training process, all weights in the image classification width learning system are optimized using the AdamW algorithm.

5. An image classification method, characterized in that: The image to be classified is input into the optimal image classification width learning system obtained by the training method described in claim 3, and the image classification result is obtained by the optimal image classification width learning system.

Images