An image classification method based on the fusion of deep learning and support vector machine

Abstract

The present invention provides an image classification method based on the fusion of deep learning and support vector machine, including embedding a coordinate attention module as a feature extraction network in a deep learning convolutional neural network, the convolutional neural network extracts representative image features by convolution and downsampling the image, the coordinate attention module captures position information and channel relationships with extremely low computational overhead by aggregating features in two spatial directions, and realizes image feature representation of an enhanced network; the enhanced image features output by the feature extraction network enter the support vector machine through a fully connected layer to perform image classification on the extracted image features, and the convolutional neural network uses the hinge loss function of the support vector machine to back-propagate the training network model, so that the gradient in the convolutional neural network can be back-propagated, and then learn lower-level features. This method integrates the convolutional neural network embedded with the coordinate attention mechanism with the SVM to realize image classification, thereby improving the image classification accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to an image classification method based on the fusion of deep learning and support vector machine. Background Art

[0002] Image classification is one of the key tasks of computer vision and one of the hot research directions. Its goal is to classify images into different feature categories according to the different feature information reflected in the image. How to accurately classify images and achieve the smallest classification error has become a key issue in image classification research. Improving image classification accuracy can promote the development of image classification technology. Support Vector Machine (SVM) based on statistical learning theory is a kernel-based machine learning model. The model has considerable classification effects in various fields such as pattern recognition, pattern classification, and computer vision. However, traditional SVM is difficult to process a large amount of image data and it is difficult to meet the actual classification accuracy requirements. Therefore, traditional SVM is not suitable for large and complex image classification. Deep learning (DL) has significant advantages over SVM in image feature learning and classification. Deep learning autonomously learns features from images through neural networks, extracts abstract high-dimensional features, and can closely connect the relationship between features and classifiers, breaking through the problems that SVM is difficult to solve in this regard.

[0003] Convolutional Neural Network (CNN) is a classic and widely used structure in deep learning models. In the 1990s, CNN achieved good results in image classification and recognition represented by handwritten digits, reaching the accuracy of human eye recognition. In 2012, the deep CNN trained by Krizhevsky et al. achieved the best image classification effect in the ImageNet Large Scale Visual Recognition Challenge, proving the effectiveness of CNN in image classification.

[0004] As convolutional neural networks gradually show their advantages in processing image features, more and more scholars have begun to combine traditional classification methods with deep learning methods. SVM and Softmax are two common classifiers in computer vision. With the continuous optimization and improvement of convolutional neural network algorithms, the use of these two classifiers has become more and more frequent. The classification of extracted data is one of the most important aspects of deep learning. SVM can classify pre-extracted data and use the specific scores given to each data as the basis for evaluation. Some studies have shown that linear SVM is selected to replace the Softmax classifier in most deep learning models, and SVM is used as the final classification unit of CNN, realizing the complete process from feature extraction to classification, and achieving the classification and discrimination ability of L2-SVM better than Softmax on the popular deep learning image classification data set; some studies have shown that after using CNN to extract data features, SVM is further used to extract features and realize classification, which extracts more features than CNN itself to a certain extent and obtains better accuracy; some studies have shown that the sparse autoencoder is combined with SVM to reorganize the learned original representation to obtain a new representation, thereby improving the classification accuracy of the data; some studies have shown that the SVM classifier integrated in the neural network shows a robust classification accuracy and verifies the effectiveness and objectivity of the proposed model classification; at the same time, there are studies that combining SVM with neural network in many fields such as human biology, medical imaging, remote sensing, etc. image classification and recognition has higher performance than the original neural network model in solving complex tasks, and can obtain significant results. However, after research and analysis, the inventor of this application found that the existing CNN uses the back propagation algorithm, which is easy to fall into the problem of local minimum or local optimum when extracting image features. Summary of the invention

[0005] In view of the technical problems that the existing traditional SVM is not suitable for processing large and complex image classification, and CNN is prone to fall into local optimality when extracting image features, the present invention provides an image classification method based on the fusion of deep learning and support vector machine.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] An image classification method based on the fusion of deep learning and support vector machine includes the following steps:

[0008] S1. Embed a coordinate attention module in a deep learning convolutional neural network, and use the convolutional neural network embedded with the coordinate attention module as a feature extraction network. The convolutional neural network extracts representative image features by convolution and downsampling the image. The coordinate attention module aggregates features in two spatial directions to capture position information and channel relationships with extremely low computational overhead, thereby enhancing the image feature representation of the network.

[0009] S2. The enhanced image features output by the feature extraction network enter the support vector machine through the fully connected layer, and the extracted image features are classified by the support vector machine to improve the generalization ability of image classification. The convolutional neural network uses the hinge loss function of the support vector machine to back-propagate the training network model, so that the gradient in the convolutional neural network can be back-propagated, thereby learning lower-level features.

[0010] Compared with the prior art, the image classification method based on the fusion of deep learning and support vector machine provided by the present invention fuses the convolutional neural network embedded with the coordinate attention module with the support vector machine to form a CASVM (CoordinateAttention-Support Vector Machine-Convolutional Neural Network) network model. The model uses the advantages of convolutional neural network and support vector machine to make up for their respective shortcomings, that is, the convolutional neural network (CNN) using the back propagation algorithm is easy to fall into the local minimum, while the support vector machine (SVM) can effectively avoid falling into the local optimum; however, if the SVM is used alone for classification, more complicated steps are required, and CNN can extract highly representative image features by convolution and downsampling the image, and the introduced coordinate attention module (Coordinate Attention, CA) can enhance the effective information of image features. Therefore, after using CNN to extract image features, replacing the classification layer of CNN with SVM with excellent generalization ability can further improve the accuracy of image classification. A large number of comparative experiments show that the method can effectively improve the accuracy of image classification, and compared with the conventional convolutional neural network, the method not only has a smaller overhead, but also has a small increase in algorithm complexity.

[0011] Furthermore, the coordinate attention module in step S1 includes a coordinate information embedding step and a coordinate attention generation step. The coordinate information embedding step first encodes each channel input feature map along the horizontal coordinate direction and the vertical coordinate direction using a pooling kernel of size (H, 1) and (1, W) for the given X, and then aggregates features along the two spatial directions through two transformations to generate a pair of direction-aware attention feature maps, one direction-aware attention feature map captures long-term dependencies along one spatial direction, and the other direction-aware attention feature map retains precise location information; the coordinate attention generation step first concatenates the two feature maps generated by the coordinate information embedding step, and then transforms them using a 1×1 shared convolution to generate an intermediate feature map of spatial information in the horizontal and vertical directions, and then divides the intermediate feature map into two separate tensors, and finally uses two 1×1 convolutions to transform the two separate tensors into the same number of channels as the input X, and uses Sigmoid as the activation function of the coordinate attention module.

[0012] Furthermore, the coordinate attention module embedded in step S1 is located in the middle layer or top layer of the convolutional neural network.

[0013] Furthermore, the convolutional neural network embedded with the coordinate attention module in step S1 is applicable to ResNet18, AlexNet, LeNet, MobileNet or VGG network models.

[0014] Furthermore, the convolutional neural network embedded with the coordinate attention module in step S1 is suitable for the ResNet18 network model, and the coordinate attention module is embedded between the data normalization and activation function of each residual module in the ResNet18 network model.

[0015] Furthermore, in step S2, the hinge loss function of the convolutional neural network using the support vector machine is a binary classification hinge loss function, and the binary classification hinge loss function is expressed by the following formula:

[0016] L i =max(0,1-x i w T y i )

[0017] Among them, x i Represents a given training data set, x i ∈R D , i∈(1,2…N), N represents the number of samples, D represents the dimension of each sample, xw is the predicted value of the linearly separable support vector machine, y is the correctly classified category, y i ∈{-1, +1}, w is the parameter that the support vector machine can learn through samples.

[0018] Furthermore, in step S2, the hinge loss function of the convolutional neural network using the support vector machine is a multi-classification hinge loss function, and the multi-classification hinge loss function is expressed by the following formula:

[0019]

[0020] Among them, x i Represents a given training data set, x i ∈R D , i∈(1,2…N), N represents the number of samples, D represents the dimension of each sample, xw is the predicted value of the linearly separable support vector machine, y is the correctly classified category, y i ∈{-1,+1},w j represents the jth component in the parameter vector w, represents the L2 norm regularization penalty, the hyperparameter λ is the penalty factor, k is the k sample categories in the training data set, and n is the amount of data in the training data set in the k category.

[0021] Furthermore, the value of the penalty factor λ is 0.0001.

[0022] Furthermore, the learning rate of the entire network model used in the method is set to 0.00001, and Adam is selected as the optimizer.

[0023] Furthermore, the main activation function of the entire network model adopted by the method is ReLU. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of a CNN network structure provided by an embodiment of the present invention.

[0025] Figure 2 It is a schematic diagram of the channel attention mechanism structure provided by an embodiment of the present invention.

[0026] Figure 3 It is a schematic diagram of hinge loss provided by an embodiment of the present invention.

[0027] Figure 4 It is a schematic diagram of the influence of different loss functions provided by an embodiment of the present invention on network training.

[0028] Figure 5 It is a diagram showing the influence of different models and different data sets on network training provided by an embodiment of the present invention.

[0029] Figure 6 It is a schematic diagram of the influence of different learning rates on network training provided by an embodiment of the present invention.

[0030] Figure 7It is a schematic diagram of the influence of the attention mechanism provided by an embodiment of the present invention on training.

[0031] Figure 8 It is a feature map output by the CA layer at different positions provided by an embodiment of the present invention.

[0032] Fig. 9 It is a schematic diagram of the CASVM network model structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below with reference to specific diagrams.

[0034] Please refer to Figures 1 to 3 As shown, the present invention provides an image classification method based on the fusion of deep learning and support vector machine, comprising the following steps:

[0035] S1. Embed a coordinate attention module in a deep learning convolutional neural network, and use the convolutional neural network embedded with the coordinate attention module as a feature extraction network. The convolutional neural network extracts representative image features by convolution and downsampling the image. The coordinate attention module aggregates features in two spatial directions to capture position information and channel relationships with extremely low computational overhead, thereby enhancing the image feature representation of the network.

[0036] S2. The enhanced image features output by the feature extraction network enter the support vector machine through the fully connected layer, and the extracted image features are classified by the support vector machine to improve the generalization ability of image classification. The convolutional neural network uses the hinge loss function of the support vector machine to back-propagate the training network model, so that the gradient in the convolutional neural network can be back-propagated, thereby learning lower-level features.

[0037] Compared with the prior art, the image classification method based on the fusion of deep learning and support vector machine provided by the present invention fuses the convolutional neural network embedded with the coordinate attention module with the support vector machine to form a CASVM (CoordinateAttention-Support Vector Machine-Convolutional Neural Network) network model. The model uses the advantages of convolutional neural network and support vector machine to make up for their respective shortcomings, that is, the convolutional neural network (CNN) using the back propagation algorithm is easy to fall into the local minimum, while the support vector machine (SVM) can effectively avoid falling into the local optimum; however, if the SVM is used alone for classification, more complicated steps are required, and CNN can extract highly representative image features by convolution and downsampling the image, and the introduced coordinate attention module (Coordinate Attention, CA) can enhance the effective information of image features. Therefore, after using CNN to extract image features, replacing the classification layer of CNN with SVM with excellent generalization ability can further improve the accuracy of image classification. A large number of comparative experiments show that the method can effectively improve the accuracy of image classification, and compared with the conventional convolutional neural network, the method not only has a smaller overhead, but also has a small increase in algorithm complexity.

[0038] As a specific example, CNN is a feedforward neural network that includes convolutional calculations, in which the convolutional layer is used to extract highly abstract and invariant image features. Commonly used CNNs in image classification include LeNet, AlexNet, and now mainstream MobileNet and ResNet. CNN has outstanding performance in the field of image algorithms due to its strong learning ability. A simple CNN network structure is as follows: Figure 1 As shown in the figure, a 32×32 pixel image is input, and after convolution filtering with multiple convolution kernels, 6 feature maps of the C1 layer are generated. Then these feature maps are downsampled to obtain the S2 layer feature maps, and the convolution kernel is further downsampled to obtain the C3 layer and S4 layer. After the above operations, many small-sized feature maps can be obtained, which are input into the traditional neural network (NN) to finally complete the image classification task.

[0039] As a specific embodiment, the channel attention mechanism represented by the existing SENet (Squeeze-and-Excitation Networks), such as Figure 2As shown in (a), by simply compressing the 2D feature map to establish the relationship between channels, there is a significant effect on the model performance, but SENet lacks in processing position information. Based on SENet, CA embeds the position information into the channel attention method, establishes a long-term dependency relationship that the existing CBAM (Convolutional Block Attention Module) cannot establish, and realizes the function of retaining position information. The channel attention mechanism usually converts the input into a single feature vector through 2D global pooling, such as Figure 2 As shown in (b), the process of CA is to split the channel attention into two one-dimensional feature codes in different directions, and then encode the features in the two directions separately to produce a pair of feature maps with improved position sensitivity and direction perception, which enhances the representation of the target of interest in the image in a complementary way. The attention block of SENet is divided into two steps: compression and excitation, which are used to collect global information and capture the importance of each channel respectively. The compression operation can be expressed as follows:

[0040]

[0041] The excitation operation of the capture channel can be expressed as follows:

[0042]

[0043] Among them, T 1 and T 2 Represents two learnable linear transformations to capture the importance of each channel.

[0044] Similar to the existing SENet, the coordinate attention module CA used in the present invention can be considered as a computing unit, which can also be divided into two steps: coordinate information embedding (CIE) and coordinate attention generation (CAG). CIE is a key operation in the attention module to capture accurate position information and establish long-term dependencies in spatial directions. Specifically, the coordinate information embedding step first encodes each channel input feature map in the horizontal coordinate direction and the vertical coordinate direction using a pooling kernel of size (H, 1) and (1, W) for the given X, that is, the global pooling is converted into a one-to-one feature encoding operation. The output of the Cth channel with a height and width of H and W, respectively, must satisfy the following formula:

[0045]

[0046] Then, the features are aggregated along two spatial directions through two transformations to generate a pair of direction-aware attention feature maps. This is completely different from the compression operation in the SE module. One direction-aware attention feature map captures long-term dependencies along one spatial direction, and the other direction-aware attention feature map retains precise location information, which helps to effectively locate objects of interest in image classification.

[0047] CAG uses the precise position information of the global receptive field generated by CIE to make a transformation and generate an attention map. CAG must meet the following three requirements: first, the transformation should be as simple and efficient as possible; second, the captured position information can be used to accurately locate the area of ​​interest; third, the relationship between channels can be efficiently captured. To this end, the first step of the coordinate attention generation step is to concatenate the two feature maps generated by the coordinate information embedding step, and then transform them using a 1×1 shared convolution, i.e., F 1 , generate the intermediate feature map f∈R of spatial information in the horizontal and vertical directions C / r×(H+W) ; Where r is the downsampling ratio, and f can be expressed as follows:

[0048] f=δ(F 1 ([z h ,z w ]))

[0049] Where δ is a nonlinear activation function. The second step of the coordinate attention generation step is to split the intermediate feature map f into two separate tensors f h ∈R C / r×H and f W ∈R C / r×W , and finally use two 1×1 convolution F h and F w Separately convert two separate tensors f h and f w Transformed to the same number of channels as the input X, we get the following formula:

[0050]

[0051] Using Sigmoid as the activation function of the coordinate attention module, the final output expression of CA is as follows:

[0052]

[0053] As a specific example, please refer to Figure 3 As shown in the figure, Hinge Loss is a loss function in the field of machine learning and is often used as the objective function of SVM. The standard hinge loss focuses on binary classification problems. For example, given a training data set xi ∈R D , i∈(1,2…N), y i ∈{-1, +1}. Assuming there are N samples, the dimension of each sample is D, xw is the predicted value of the linearly separable SVM, y is the correctly classified category, and w is the parameter vector that the classifier can learn through the sample. Then its hinge loss form, that is, the two-class hinge loss function, is expressed as follows:

[0054] L i =max(0,1-x i w T y i )

[0055] For the objective function of the binary classification problem, when the formula x i w T y i When it is greater than 1, the loss value is 0, that is, the loss function at this time can be judged as the correct classification result and the function interval is considered to be greater than 1.

[0056] As a specific embodiment, since SVM can filter samples far away from the hyperplane by selecting a threshold for the distance from the sample to the hyperplane, these samples are often samples that are easy to be correctly classified, and thus the loss function of multi-classification SVM can be extended. Therefore, the multi-classification hinge loss form for a single sample can be expressed as follows:

[0057]

[0058] Among them, w j Represents the jth component in the parameter vector w. Multi-classification SVM sets the prediction score of the correct classification to be at least one threshold higher than the prediction scores of other incorrect classifications. If the prediction scores of other classifications are lower than the threshold, the loss value is calculated. Due to the non-uniqueness of w, a regularization penalty is introduced to eliminate the ambiguity of w. The most commonly used regularization penalty is the L2 paradigm. Therefore, the multi-classification SVM loss function contains a regularization penalty. The loss function of the k-classification linear SVM, that is, the multi-classification hinge loss function, can be expressed as follows:

[0059]

[0060] Among them, x i Represents a given training data set, x i ∈R D , i∈(1,2…N), N represents the number of samples, D represents the dimension of each sample, xw is the predicted value of the linearly separable support vector machine, y is the correctly classified category, y i ∈{-1,+1},w j represents the jth component in the parameter vector w, represents the L2 norm regularization penalty, the hyperparameter λ is the penalty factor, k is the k sample categories in the training dataset, and n is the amount of data in the k category of the training dataset. At the same time, this formula can be regarded as the loss function of the multi-classification SVM, which is the data loss plus the regularization loss, where the hinge loss is the data loss and the L2 norm is the regularization loss. Since the penalty in L2 regularization tends to be smaller and relatively dispersed weight vectors, the L2 norm can improve the generalization performance of the classifier on images and reduce overfitting.

[0061] As a specific embodiment, the regularization penalty factor λ in the regularization loss has a relatively important influence on the similarity weight processing. Generally speaking, a smooth curve can better reflect the real situation. When the penalty factor λ is set too low, it will cause underfitting in the network training, and vice versa, it is easy to overfit. Therefore, the correct setting of this parameter helps to eliminate the similarity ambiguity of the weight. Through a large number of comparative experiments, the present invention sets the value of the penalty factor λ to 0.0001.

[0062] As a specific embodiment, in order to better reflect the beneficial effects of the present invention, the following will be compared with the prior art in combination with relevant parameters:

[0063] (1) Impact and selection of data sets

[0064] The three data sets of Fashion-Mnist, Cifar10 and Animal10 used in the present invention are image data sets with 10 categories, and CIfar100 is selected as the image data set with 100 categories. Among them, Fashion-Mnist is an image data set that replaces the MNIST handwritten digit set, with a total of 60,000 grayscale images with a resolution of 28×28, which is now widely used in the benchmark test of deep learning models; Cifar10 is a data set for universal object recognition, with the same number of 32×32 RGB color images as Fashion-Mnist. Animal10 is a total of 26,000 animal images of ten species on Kaggle. The data set has been manually reviewed, but there are also some wrong labels used to simulate the real situation. In addition, the present invention unifies the size of the data set images to 224×224. CIfar100 is an image data set with 100 categories, with a resolution of 32×32, which is used to test the classification performance of the model in multi-category problems.

[0065] The present invention uses four models, namely CNN+SVM, CNN+CA+SVM (i.e., CASVM), CNN+Softmax, and CNN+CA+Softmax, to conduct classification comparison experiments on image data sets. In addition, all experimental network models of the present invention are trained 100 times, and the training size is the number of samples N / 128.

[0066] Since the data set used in the present invention has two sizes of 32×32 and 224×224, the present invention improves two CNN structures for small-size data sets. One is an improved model CNN1 based on LeNet. First, the number of convolution kernels is increased and the original 5×5 convolution kernel is changed to 3×3, then Dropout is added, and finally CA is embedded in the last convolution layer. The other is an improved model CNN2 based on AlexNet. This model reduces the number of convolution kernels, embeds CA in the last convolution layer, and then reduces the parameters of the fully connected layer. In addition, the CNN structures used in the experiment of the present invention also include AlexNet and ResNet18.

[0067] (2) Impact and selection of loss function

[0068] The loss function is an important factor affecting the learning ability of the network. The loss functions used in the convolutional neural network of the present invention include SVM loss and the cross entropy loss (Categorical Crossentropy Loss) used by general neural networks for Softmax classifiers. The data loss functions of SVM include Hinge Loss, Squared Hinge Loss and Categorical Hinge Loss; when applying Hinge Loss and Squared Hinge Loss, one-to-many multiple SVM models are used to solve the multi-classification problem. In order to better present the experimental results, the above four loss functions are applied to SVM and Softmax for back propagation. Figure 4 The effects of four different loss functions on network training are shown in Figure 1, and the results of AlexNet training in Cifar10. The experiment shows that since SVM is not suitable for cross entropy loss, the accuracy of the training results is low and the loss value is high. However, compared with cross entropy loss, the accuracy of using SVM classification or CASVM classification is higher than Softmax, regardless of whether the attention mechanism is embedded or not and whether Softmax uses cross entropy loss. Figure 4 From the comparison results, we can see that within the same structure, the hinge loss has a lower error than the cross entropy loss, and there is a certain difference in performance between SVM and Softmax.

[0069] Since different data and different network models may also affect the final classification results, in order to expand the difference, the square hinge loss function is used and the shallow network CNN1 and the deeper network ResNet18 are used to compare the performance of the two classifiers without embedding the attention mechanism. Figure 5These are the accuracy and loss values ​​of the CNN1 and ResNet18 network models when using the Anima10, Cifar100, Cifar10, and Fashion-Mnist data.

[0070] When training CNN1, all images in the dataset are compressed to 32×32, and the network model using Softmax as the classification still uses cross entropy loss. Figure 5 From (a), we can see that the network model combining CNN1 and SVM has little improvement in accuracy on small-sized datasets. Figure 5 As can be seen from (b), ResNet18 combined with SVM has certain improvements on various data sets, especially on 100 classifications and large-size data sets. It can be seen that the loss function of the SVM classifier has certain improvements on the network classification performance.

[0071] (3) Influence and selection of learning rate

[0072] The learning rate has a great impact on the speed and stability of network training. A learning rate that is too low may slow down the network convergence, while a learning rate that is too high may cause the network to fail to converge. A higher learning rate may help increase the network training speed, but considering that a low learning rate may reduce the difficulty of training and obtain ideal results, the learning rate is set to 0.01, 0.001, 0.0001, and 0.00001 for comparative experiments. Figure 6 As shown. The network model of this figure is a CNN1 structure based on the CASVM (Squared_hinge) framework. Fashion-Mnist is used as the data set for training and testing. The influence of learning rates on network training is obtained when they are 0.01, 0.001, and 0.0001, respectively. It is easy to know through experiments that when the learning rate is 0.01, the accuracy of the test set is unstable, which eventually leads to a large error. Therefore, the present invention takes three learning rates of 0.001, 0.0001, and 0.00001 as the final model comparison parameters.

[0073] (4) Influence and selection of optimizer

[0074] The optimizer is used to adjust the parameters in the network, update and calculate the network parameters of model training and model output. Commonly used optimizers include Adam, SGD, RMSProp, etc. Adam combines the advantages of Momentum and RMSProp, and dynamically adjusts the learning rate of each parameter using the first-order moment estimation and second-order moment estimation of the gradient. Compared with SGD and RMSprop, Adam can constrain the learning rate so that each learning rate has a certain range, thereby making the parameters more stable. The following table 1 shows the impact of different optimizers on training. The table uses Animal10 as a training data set, and Adam, SGD, and RMSProp optimizers are applied respectively to compare accuracy and loss values ​​based on the CASVM (Squared_hinge) framework. Each optimizer uses four models of CNN1, AlexNet, CNN2, and ResNet18 for comparison. Obviously, the Adam optimizer is superior to the other two optimizers in terms of accuracy or loss value, so the present invention selects Adam as the main optimizer of CASVM.

[0075] Table 1 The impact of different optimizers on training

[0076]

[0077] (5) Influence and selection of attention mechanism

[0078] The role of the attention mechanism is to improve the network's extraction of image features. It is generally believed that under the same structural conditions, the network with the attention mechanism will extract image features more significantly. Figure 7 This figure shows the effect of the attention mechanism on training in network models with different loss functions. This figure uses Animal10 as the training and testing data set, and uses the AlexNet structure as the network model. The loss functions used in the network are hinge loss function and cross entropy loss function. Finally, the accuracy of the model is improved after adding CA. It is easy to see from the figure that when the loss function is consistent and CA is added, the network model using SVM classification, that is, the CASVM network model, has a higher classification accuracy than the Softmax classification. At the same time, from Figure 7 It can be seen that whether the network model uses SVM classification or Softmax classification, the accuracy will be improved after adding CA. At the same time, the introduction of CA to Softmax using cross entropy loss will have a higher classification accuracy than the SVM without CA. Therefore, adding attention usually has a higher improvement in the accuracy of image classification.

[0079] (6) Influence and selection of activation function

[0080] Activation function is an important factor affecting network learning ability and training speed. Common activation functions include Sigmoid, Tanh, and ReLU. In the CA module, the Sigmoid activation function is used to compress the values ​​propagated from the upper layer to between [0,1], that is, the Sigmoid activation function is output to a limited range, so that the CA module is optimized and stable, which is helpful for the representation of attention features. Therefore, Sigmoid is selected as the activation function in CA. ReLU is one of the very popular activation functions in recent years. Compared with Sigmoid and Tanh, the ReLU activation function can effectively alleviate the problem of gradient disappearance and can speed up the convergence speed of the overall network. Therefore, ReLU is mainly selected as the main activation function in the entire network model of the present invention.

[0081] Experimental results and beneficial effects:

[0082] In the above parameter comparison, the results of cross-combination between different parameters of four data sets of Anima10, Cifar100, Cifar10, Fashion-Mnist and four network models of CNN1, AlexNet, CNN2, and ResNet18 are compared. From the results, it is easy to know that the loss function of the SVM classifier has a certain improvement on the network classification performance under normal circumstances. The lower learning rate of 0.00001 can make the model accuracy smoother, and the selection of a better optimizer Adam can further improve the model accuracy. Therefore, the present invention finally selects the optimizer Adam and the learning rate of 0.00001 as the main parameters of the whole network model CASVM.

[0083] As a specific embodiment, in order to better analyze the feature extraction of the CA module, the image features in the CA layers at different positions are compared and analyzed, and the details are as follows: Figure 8 shown.

[0084] Figure 8 This is the feature output diagram of CASVM combined with CNN1. Figure 8 (a)-(d) are feature maps obtained after model training of a randomly selected image from the Anima10, Fashion-Mnist, Cifar10, and Cifar100 datasets. The convolutional layer and CA layer of the model both contain 64 channels, and the convolutional layers in the figure are the first convolution layer in the network model. By comparing the image features output by the CA layer at different positions, it is found that the performance of different channel features is different, and there are large differences in the role of different convolution kernels in the CNN structure. Figure 8(a) shows that the CA layer is located after the first convolutional layer (i.e. the bottom layer) of the network. By observation, we can see that the feature difference is not obvious. From this, we can conclude that when the CA layer is located after the first convolutional layer in a shallow network such as CNN1, the output features are still relatively detailed and contain a lot of basic visual feature details. And from the third and fourth rows in Figure 8(a), we can see that the feature map contains some unrelated features. These features have no key role in image classification. Therefore, the CA layer should not be located after the first convolutional layer of the network. Figure 8 As shown in (b) and 8(c), when the CA layer is located in the middle layer of the network, the CA layer enhances the representation of the edge contour information, the representation of the global information, and the representation of the texture information of the feature. Figure 8 As shown in (d), when the CA layer is close to the top layer, it can weaken invalid features and strengthen valid features, which is more conducive to improving the classification accuracy of the classifier. Therefore, the coordinate attention module embedded in step S1 can be located in the middle layer or top layer of the convolutional neural network for setting.

[0085] Table 2 Comparison of image classification accuracy results

[0086]

[0087]

[0088] The most intuitive measure to judge the accuracy of model image classification is to calculate its classification accuracy and loss value. In order to verify the advantages of CASVM in image classification, as shown in Table 2, the present invention has done eight groups of comparative experiments, and the basic framework of each group of experimental models includes the following 4 kinds: the model structure not embedded in CA and based on SVM classification, the model structure not embedded in CA and based on Softmax classification, the model structure embedded in CA and based on SVM classification, i.e. CASVM, the model structure embedded in CA and based on Softmax classification. The above 3 frameworks are selected to compare with CASVM to verify the classification accuracy of CASVM in four different CNN models of CNN1, AlexNet, CNN2 and ResNet18. In order to verify the generalization ability of CASVM, Fashion-Mnist, Cifar10, Anima10 and Cifar100 are selected as the data sets for training and testing, and the Animal10 data set images are preprocessed at the same time. The detailed results are shown in Table 2. In addition, the table uses Softmax as the network model for classification, and the loss function is cross entropy loss.

[0089] As shown in Table 2, the CNN+SVM structure without CA has the best classification accuracy in 5 of the eight groups of image classification indicators. Among them, one item was obtained by using the Cifar10 dataset, that is, the network model based on ResNet18 was selected, and the accuracy was 0.77 when the learning rate was 0.0001. Two items were obtained when the Cifar100 dataset was selected, that is, the network model based on ResNet18 was selected, and the highest classification accuracy of 0.58 was obtained when the learning rates were 0.0001 and 0.00001 respectively. Two items were obtained when the Animal10 dataset was selected, that is, the network model based on CNN2, and the highest classification accuracy of 0.74 and 0.79 were obtained when the learning rates were 0.0001 and 0.00001 respectively. The CNN+SVM structure without CA has 12 items in the experiment that have the same accuracy as other structures. Among them, 4 items were obtained by selecting the Fashion-Mnist dataset, that is, the network model was selected as the Resnet18 model, and the accuracies were 0.93 and 0.94 respectively when the learning rates were 0.001 and 0.00001; and when the learning rate was 0.0001, two optimal identical accuracies were obtained, both of which were 0.94. When the Cifar10 dataset was selected, 4 optimal identical accuracies were obtained. That is, the network model based on AlexNet was selected, and the accuracies were 0.78, 0.83, and 0.83 respectively when the learning rates were 0.001, 0.0001, and 0.00001. The network model based on ResNet18 was selected, and the accuracy was 0.74 when the learning rate was 0.001. When the Cifar100 dataset was selected, 2 optimal identical accuracies were obtained, that is, the network model based on ResNet18 was selected, and the accuracy was 0.58 when the learning rates were 0.0001 and 0.00001. When the Animal10 dataset is used, an optimal equivalent accuracy can be obtained, that is, a network model based on CNN2 is used, and an accuracy of 0.79 is obtained when the learning rate is 0.00001.

[0090] Among the eight groups of image classification indicators, the CNN+SOFTMAX structure has only one item with the best classification accuracy. Among them, the Animal10 dataset was selected, and the network model based on CNN2 was used. When the learning rate was 0.0001, the highest classification accuracy of 0.74 was obtained. There were two items in the experiment where the CNN+SOFTMAX structure achieved the same accuracy as other structures. Among them, one item was obtained using the Cifar10 dataset, that is, the network model based on AlexNet was used, and the accuracy was 0.78 when the learning rate was 0.001. One item was obtained using the Animal10 dataset, and the network model based on CNN2 was used. When the learning rate was 0.0001, the accuracy was 0.74.

[0091] The CASVM structure has the best classification accuracy in 14 of the eight groups of image classification indicators. Among them, one item was obtained by using the Cifar10 dataset, that is, the network model based on ResNet18 was used, and the accuracy was 0.79 when the learning rate was 0.00001. One item was obtained by using the Cifar100 dataset, that is, the network model based on ResNet18 was used, and the accuracy was 0.52 when the learning rate was 0.001. 12 items were obtained by using the Animal10 dataset. That is, the network model based on CNN1 was used, and the accuracy was 0.74 and 0.8 when the learning rate was 0.001 and 0.00001, respectively, and when the learning rate was 0.0001, the accuracy of both items was 0.77. The network model based on AlexNet was used, and the accuracy was 0.66 and 0.7 when the learning rate was 0.001 and 0.0001, respectively; when the learning rate was 0.00001, the accuracy of both items was 0.73. The network model based on CNN2 was selected, and the accuracy was 0.65 when the learning rate was 0.001. The network model based on ResNet18 was selected, and the accuracy was 0.62, 0.84, and 0.92 when the learning rates were 0.001, 0.0001, and 0.00001, respectively. In the experiment, the CASVM structure obtained the same accuracy as other structures in 10 items, of which 5 items were obtained by using the Fashion-Mnist dataset, and the accuracy was 0.93 when the learning rate was 0.001. Among them, when the learning rates were 0.0001 and 0.00001, respectively, two optimal identical accuracies were obtained, both of which were 0.94. Five optimal identical accuracies were obtained when the Cifar10 dataset was selected. Among them, the network model based on AlexNet was selected, and when the learning rate was 0.0001, two optimal identical accuracies were obtained, both of which were 0.83; when the learning rate was 0.00001, the accuracy was 0.83. The network model based on ResNet18 was selected. When the learning rate was 0.001, the two best identical accuracies were obtained, both of which were 0.74. When the Animal10 dataset was selected, CASVM obtained the most optimal accuracies in the experiment, and did not have the same optimal accuracies as other structures.

[0092] In Table 2, the lowest loss value of CASVM is basically lower than 0.2, and the loss value in the small-size dataset Fashion-Mnist is even lower than 0.05. Among the 12 lowest loss values ​​of the large-size dataset Animal10, CASVM has the lowest 7, while the loss value of the CNN+Softmax structure in Animal10 is in the range of 0.99-2.25. Therefore, it can be concluded that CASVM can make the loss smaller and make the CNN model based on the CASVM structure fit better.

[0093] In summary, CNN+SVM has 5 best indicators among the eight groups of image classification indicators, and 12 indicators have the same classification accuracy as other structures. CNN+SOFTMAX has 1 best indicator among the eight groups of image classification, and 2 indicators have the same classification accuracy as other structures. The CASVM of the present invention has 14 best indicators among the eight groups of image classification indicators, 10 indicators have the same classification accuracy as other structures, and 16 indicators have the lowest loss value. CASVM obtained 10 best indicators in the Animal10 large-size image dataset, and 5 indicators in the small-size dataset, which shows that CASVM can obtain higher classification accuracy when applied to large-size data. Although CASVM does not perform as well in small-size datasets as in large-size datasets, since CASVM has 10 best indicators with the same accuracy in small-size datasets, it can be seen that CASVM in small-size datasets can still improve the classification accuracy of the model. From the above results, it can be seen that CASVM will achieve better classification accuracy in shallow networks, but in specific environments, it is comparable to the classification accuracy of the SVM model without embedding CA. When based on deep models and using large-size images, CASVM has obvious advantages. When using the same CNN model, the classification accuracy and training loss value of the Softmax classification are lower than those of the SVM classification, let alone the CASVM. This once again verifies that the CASVM has better performance and robustness than the Softmax classification in general environments.

[0094] As a specific embodiment, the convolutional neural network embedded with the coordinate attention module in step S1 is suitable for the ResNet18 network model. The coordinate attention module is embedded between the data normalization and activation function of each residual module in the ResNet18 network model, and its structure is as follows: Fig. 9 As shown. That is, the CASVM in this embodiment is composed of SVM and CNN with CA introduced. Taking ResNet18 as an example, the specific steps are as follows:

[0095] (1) First, input an image of size C×H×W. C represents the number of channels of the feature map, H represents the height of the feature map, and W represents the width of the feature map. Before being sent to the residual module 1, the image is first convolved to retain the feature information as much as possible; then after the last data normalization in the residual module 1, it enters the coordinate attention module (CA).

[0096] (2) After the feature map enters the CA module, it is average pooled in the horizontal and vertical directions, and the number of channels of the feature map is reduced to 1 / r times the original. Then, the channel is compressed by connecting the spatial dimension and 1×1 convolution, and then the spatial information in two directions is divided into feature maps in two directions through batch normalization and nonlinear encoding. Then, 1×1 convolution is used to obtain the same number of channels as the input, and the weight is normalized by the sigmoid function.

[0097] (3) The design of the subsequent residual modules is the same as that of residual module 1. The coordinate attention module is embedded between the data normalization and the activation function, which effectively amplifies the difference between image features and thus better distinguishes image categories. After the residual calculation is completed, two fully connected layers are used to represent the classification categories of the divided images.

[0098] (4) Finally, the parameters of the entire network are learned through the multi-classification SVM loss function.

[0099] As a specific embodiment, the CNN in this CASVM model is not limited to ResNet18, but can also be applied to network models such as Alexnet, LeNet, MobileNet, VGG, etc. The structure of other CNNs that constitute CASVM is similar to ResNet18. After a series of convolution operations, CNN enters the CA module, and after the CA module enhances the image features, it enters the fully connected layer, and then the image classification is realized through the SVM classifier.

[0100] This CASVM model combines the advantages of CNN and SVM embedded with attention mechanism to make up for their respective shortcomings, so the image classification results of this model are better than those of a single classifier. CNN is an empirical risk minimization model. When the back propagation algorithm finds the classification hyperplane, the algorithm may easily fall into the local minimum. SVM, on the other hand, is a structural risk minimization model. It can effectively avoid falling into the local optimum by solving the quadratic programming problem. Therefore, the generalization ability of SVM is better than that of CNN. Replacing the CNN output layer with SVM can further improve the image classification accuracy.

[0101] Since convolutional neural networks can extract representative image features by downsampling and convolution operations on images, the attention mechanism is introduced to enhance effective information. Attention is set on image areas that have a greater impact on classification, thereby extracting stronger image features. In contrast, manually designed feature extraction methods are more difficult and require more skills, and the extracted features may not be universally effective for all features. Therefore, deep learning methods can avoid a lot of manual participation and can extract more significant features than traditional methods. This is the unique advantage of combining deep learning with support vector machines.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. An image classification method based on the fusion of deep learning and support vector machine, characterized in that: The following steps are involved: S1. Embed a coordinate attention module in a deep learning convolutional neural network, and use the convolutional neural network embedded with the coordinate attention module as a feature extraction network. The convolutional neural network extracts representative image features by convolution and downsampling the image. The coordinate attention module aggregates features in two spatial directions to capture position information and channel relationships with extremely low computational overhead, thereby enhancing the image feature representation of the network. S2, the enhanced image features output by the feature extraction network enter the support vector machine through the fully connected layer, and the extracted image features are classified by the support vector machine to improve the generalization ability of image classification. The convolutional neural network uses the hinge loss function of the support vector machine to back propagate the training network model, so that the gradient in the convolutional neural network can be back propagated, and then learn the lower-level features; wherein, The coordinate attention module in step S1 includes a coordinate information embedding step and a coordinate attention generation step. The coordinate information embedding step first encodes each channel input feature map along the horizontal coordinate direction and the vertical coordinate direction using a pooling kernel of size (H, 1) and (1, W) for the given X, and then aggregates features along the two spatial directions through two transformations to generate a pair of direction-aware attention feature maps, one direction-aware attention feature map captures long-term dependencies along one spatial direction, and the other direction-aware attention feature map retains precise location information; the coordinate attention generation step first cascades the two feature maps generated by the coordinate information embedding step, and then uses a 1×1 shared convolution to transform them to generate an intermediate feature map of spatial information in the horizontal and vertical directions, and then divides the intermediate feature map into two separate tensors, and finally uses two 1×1 convolutions to transform the two separate tensors into the same number of channels as the input X, and uses Sigmoid as the activation function of the coordinate attention module.

2. The image classification method based on deep learning and support vector machine fusion according to claim 1 is characterized in that: The coordinate attention module embedded in step S1 is located in the middle layer or top layer of the convolutional neural network.

3. The image classification method based on deep learning and support vector machine fusion according to claim 1 is characterized in that: The convolutional neural network embedded with the coordinate attention module in step S1 is suitable for ResNet18, AlexNet, LeNet, MobileNet or VGG network models.

4. The image classification method based on deep learning and support vector machine fusion according to claim 1, characterized in that: The convolutional neural network embedded with the coordinate attention module in step S1 is suitable for the ResNet18 network model, and the coordinate attention module is embedded between the data normalization and activation function of each residual module in the ResNet18 network model.

5. The image classification method based on the fusion of deep learning and support vector machine according to claim 1, characterized in that: In step S2, the hinge loss function of the convolutional neural network using the support vector machine is a binary classification hinge loss function, and the binary classification hinge loss function is expressed by the following formula: L i =max(0,1-x i w T y i ) Among them, x i Represents a given training data set, x i ∈R D , i∈(1,2L N), N represents the number of samples, D represents the dimension of each sample, xw is the predicted value of the linearly separable support vector machine, y is the correctly classified category, y i ∈{-1, +1}, w is the parameter vector that the support vector machine can learn through samples.

6. The image classification method based on deep learning and support vector machine fusion according to claim 1, characterized in that: In step S2, the hinge loss function of the convolutional neural network using the support vector machine is a multi-classification hinge loss function, and the multi-classification hinge loss function is expressed by the following formula: Among them, x i Represents a given training data set, x i ∈R D , i∈(1,2L N), N represents the number of samples, D represents the dimension of each sample, xw is the predicted value of the linearly separable support vector machine, y is the correctly classified category, y i ∈{-1,+1},w j represents the jth component in the parameter vector w, represents the L2 norm regularization penalty, the hyperparameter λ is the penalty factor, k is the k sample categories in the training data set, and n is the amount of data in the training data set in the k category.

7. The image classification method based on deep learning and support vector machine fusion according to claim 6 is characterized in that: The value of the penalty factor λ is 0.0001.

8. The image classification method based on deep learning and support vector machine fusion according to claim 1, characterized in that: The learning rate of the entire network model used in the method is set to 0.00001, and Adam is selected as the optimizer.

9. The image classification method based on deep learning and support vector machine fusion according to claim 1, characterized in that: The activation function of the entire network model adopted by the method is ReLU.

Images