Cross-network hierarchical connection image classification method based on convolutional neural network

Abstract

The invention provides a cross-network hierarchical connection image classification method based on a convolutional neural network, and belongs to the technical field of computer vision and machine learning, the method learns a mapping by mapping enhanced data back to original data, and the mapping can map unseen target domain data to a source domain in a training stage. The process is helpful for reducing data distribution differences among different domains, so that the generalization ability of the model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and machine learning, and in particular to the problem of image classification, and specifically to an image classification method based on a convolutional neural network with cross-network layer connections. Background Art

[0002] Image classification methods based on machine learning improve the accuracy and robustness of classification by learning features from a large amount of labeled data. Common machine learning algorithms include support vector machines (SVM) proposed by Cortes et al. in "Support-vector networks. Machine Learning", random forests (Random Forest) proposed by Breiman et al. in "Random forests", and K nearest neighbors (KNN) proposed by Altman et al. in "An introduction to kernel and nearest-neighbor nonparametric regression". These methods convert images into feature vectors through feature extraction and feature selection steps, and then use classifiers for classification. Although these methods have improved the classification performance to a certain extent, they still rely on manually designed features and find it difficult to capture complex image features.

[0003] With the development of deep learning technology, its application in image classification tasks has become increasingly widespread, and it has also shown superior performance. Deep learning automatically learns high-level features of images through multi-layer neural networks without the need for manual feature design, thereby greatly improving the accuracy and robustness of classification. The Convolutional Neural Networks (CNN) proposed by LeCun et al. in "Gradient-based learning applied to document recognition" is one of the most commonly used architectures in deep learning. It effectively extracts local and global features of images through components such as convolutional layers, pooling layers, and fully connected layers. The U-Net deep learning network proposed by Szegedy et al. in "Going Deeper with Convolutions" is a mature deep learning network that has shown obvious advantages in medical image segmentation tasks. However, U-Net has certain limitations in network depth and width, especially when processing images of different scales, its performance may decline. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides an image classification method based on convolutional neural network with cross-network layer connections. Through cross-network layer connections and attention mechanism, the model calculation amount is reduced and the overfitting problem is avoided, thereby improving the detection accuracy of image classification.

[0005] The technical solution of the present invention is:

[0006] A convolutional neural network-based image classification method with cross-network layer connections, comprising:

[0007] Full-scale cross-connection: By transferring feature information between different layers, the network can better capture and utilize multi-scale features;

[0008] Deep supervision mechanism: By introducing supervision signals at several levels, the network can converge faster during training and maintain high accuracy at all levels.

[0009] Attention mechanism: By dynamically adjusting the importance of features, the network can utilize useful features and ignore irrelevant noise, thereby improving the accuracy and robustness of classification.

[0010] Furthermore,

[0011] The full-scale span connection adds full-scale jump connections based on the UNet3+ deep learning network.

[0012] By establishing multi-scale connections between the encoder and the decoder, the network can share feature information at different levels, thereby improving the accuracy of feature extraction.

[0013] Let XEni be the feature map of the i-th layer encoder, XDej be the feature map of the j-th layer decoder, and the full-scale skip connection is expressed by the following formula:

[0014]

[0015] Among them, H() is a feature aggregation function, including convolution operation, batch normalization BN and ReLU activation function; the convolution operation extracts local features by sliding a small filter or convolution kernel on the input data; batch normalization normalizes the input data on each small batch to reduce internal covariate shift, thereby accelerating the training process; the ReLU activation function is a nonlinear activation function defined as

[0016] f(x)=max(0,x).

[0017] Furthermore,

[0018] A deep supervision mechanism is introduced into the UNet3+ deep learning network to accelerate the convergence speed of the network and improve the accuracy of position perception.

[0019] A supervised loss function is introduced in each encoder layer, the loss at each stage is calculated, and the loss function is updated through back propagation, thereby accelerating the convergence of the network.

[0020] The calculation formula is as follows:

[0021]

[0022] Where Ypredi is the predicted output of the i-th layer encoder, Ytrue is the true label, and L is the cross entropy loss function.

[0023] Furthermore,

[0024] Through the channel attention mechanism and the spatial attention mechanism, the data feature expression of the feature map is improved, and more attention is paid to the basic features, thereby achieving accurate monitoring of image classification.

[0025] The channel attention mechanism is as follows:

[0026] M avg (F) = σ(W1·W0·F c，avg )

[0027] M max (F) = σ(W1·W0·F c，max )

[0028] M c (F)=M avg (F)+M max (F)

[0029] Among them, Mavg(F) and Mmax(F) are the average pooling features and the maximum pooling features respectively, σ is the sigmoid activation function, and W1 and W0 are the weights of the multi-layer perceptron (MLP);

[0030] The spatial attention mechanism is as follows:

[0031] M s (F) = σ(f 7×7 ([F s，avg ; F s，max ]))

[0032] Among them, Fs,avg and Fs,max are spatial pooling features, f 7×7 It is a 7×7 convolutional layer.

[0033] The beneficial effects of the present invention are

[0034] This method can share feature information at different levels, so as to better capture and utilize multi-scale features. This not only improves the accuracy of feature extraction, but also enhances the network's adaptability to images of different scales. The full-scale jump connection enables the network to share feature information at different levels by establishing multi-scale connections between the encoder and decoder, thereby improving the accuracy of feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the network structure framework of the present invention;

[0036] Figure 2 It is a schematic diagram of the overall training process. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0038] To address the gap between the source domain and the target domain, this method uses full-scale cross-linking, deep supervision mechanism, and attention mechanism. By mapping the enhanced data back to the original data, a mapping is learned that can map the unseen target domain data to the source domain during the training phase. This process helps to reduce the data distribution differences between different domains, thereby improving the generalization ability of the model.

[0039] Full-scale skip connections: In order to ensure that the network captures features of different scales, this method adds full-scale skip connections based on the UNet3+ deep learning network. Specifically, by establishing multi-scale connections between the encoder and the decoder, the network can share feature information at different levels, thereby improving the accuracy of feature extraction. The main purpose of full-scale skip connections is to solve the shortcomings of traditional convolutional neural networks in processing multi-scale features, especially in image classification tasks, where features of different scales have an important impact on the final classification results.

[0040] Assuming XEni is the feature map of the i-th layer encoder and XDej is the feature map of the j-th layer decoder, the full-scale skip connection can be expressed by the following formula:

[0041]

[0042] Among them, H() is the feature aggregation function, including convolution operation, batch normalization (BN) and ReLU activation function. The convolution operation is the core component of the convolutional neural network (CNN), which extracts local features by sliding a small filter (or convolution kernel) on the input data. Batch normalization is a commonly used technique in neural networks. It normalizes the input data on each mini-batch to reduce internal covariate shift, thereby accelerating the training process. The ReLU (Rectified Linear Unit) activation function is a nonlinear activation function defined as f(x) = max(0,x). The full-scale skip connection enables the network to better capture and utilize multi-scale features by transferring feature information between different levels. This is particularly important for image classification tasks, because features of different scales can provide rich contextual information, helping the network to more accurately identify and classify objects in the image. In addition, the full-scale skip connection can also reduce the loss of information during the transmission process and improve the robustness and generalization ability of the network.

[0043] Deep supervision mechanism: In order to accelerate the convergence speed of the network and improve the accuracy of position perception, this method introduces a deep supervision mechanism in the UNet3+ deep learning network. Specifically, a supervision loss function is introduced in each encoder layer, the loss of each stage is calculated, and the loss function is updated through back propagation, thereby accelerating the convergence speed of the network. The main purpose of the deep supervision mechanism is to enable the network to converge faster during training and maintain high accuracy at all levels by introducing supervision signals at multiple levels.

[0044] Specifically, the formula is as follows:

[0045]

[0046] Where Ypredi is the predicted output of the i-th layer encoder, Ytrue is the true label, and L is the cross entropy loss function. The deep supervision mechanism enables the network to converge faster during training by introducing supervision signals at multiple levels. This not only improves the training efficiency of the network, but also maintains high accuracy at all levels, thereby improving the final classification performance. In addition, the deep supervision mechanism can also help the network better learn and utilize multi-scale features, improving the robustness and generalization ability of the model.

[0047] Attention mechanism: In order to further optimize the feature extraction process and avoid overfitting problems, this method introduces an attention mechanism. Specifically, through the channel attention mechanism and the spatial attention mechanism, the data feature expression of the feature map is improved, and more attention is paid to the basic features, so as to achieve accurate monitoring of image classification. The main purpose of the attention mechanism is to dynamically adjust the importance of features so that the network can more efficiently utilize useful features and ignore irrelevant noise, thereby improving the accuracy and robustness of classification. Specifically, the channel attention mechanism is as follows:

[0048]

[0049] In the cross-domain semantic segmentation task, the generalization ability of the model is crucial. In order to improve the performance of the model on unseen target domains, Mavg(F) and Mmax(F) are the average pooling features and the maximum pooling features, respectively, σ is the sigmoid activation function, and W1 and W0 are the weights of the multi-layer perceptron (MLP).

[0050] The spatial attention mechanism is as follows:

[0051] M s (F) = σ(f 7×7 ([F s，avg ; F s，max ])) (4.4)

[0052] Among them, Fs,avg and Fs,max are spatial pooling features, f 7×7 It is a 7×7 convolutional layer.

[0053] The attention mechanism dynamically adjusts the importance of features, allowing the network to more efficiently utilize useful features and ignore irrelevant noise. The channel attention mechanism calculates the weight of each channel, allowing the network to pay more attention to important channel features, thereby improving the accuracy of feature expression. The spatial attention mechanism calculates the weight of each spatial position, allowing the network to pay more attention to important spatial areas, thereby improving the accuracy of feature expression. The combination of these two attention mechanisms not only improves the efficiency of feature extraction, but also effectively avoids overfitting problems and improves the robustness and generalization ability of the model.

[0054] In contrast, the UNet3+ deep learning network is more suitable for image classification applications. The UNet3+ deep learning network significantly improves the accuracy of boundary segmentation and multi-scale feature extraction through deep supervision and full-scale jump connections.

[0055] Through the above technical solutions, this method not only improves the detection accuracy of image classification, but also effectively reduces the amount of model calculation and avoids the overfitting problem. Experimental results show that this method has excellent performance in image classification tasks of different scales.

[0056] The present invention combines full-scale skip connections, deep supervision mechanism and attention mechanism, which not only improves the detection accuracy of image classification, but also effectively reduces the amount of model calculation, avoids overfitting problems, and enhances the robustness and generalization ability of the model.

[0057] The above description is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A convolutional neural network-based image classification method with cross-network layer connections, characterized in that: include: Full-scale cross-connection: By transferring feature information between different layers, the network can better capture and utilize multi-scale features; Deep supervision mechanism: By introducing supervision signals at several levels, the network can converge faster during training and maintain accuracy at all levels; Attention mechanism: By dynamically adjusting the importance of features, the network can utilize useful features and ignore irrelevant noise, thereby improving the accuracy and robustness of classification.

2. The method according to claim 1, characterized in that The full-scale span connection adds full-scale jump connections based on the UNet3+ deep learning network.

3. The method according to claim 2, characterized in that By establishing multi-scale connections between the encoder and the decoder, the network can share feature information at different levels, thereby improving the accuracy of feature extraction.

4. The method according to claim 3, characterized in that Let XEni be the feature map of the i-th layer encoder, XDej be the feature map of the j-th layer decoder, and the full-scale skip connection is expressed by the following formula: Among them, H() is a feature aggregation function, including convolution operation, batch normalization BN and ReLU activation function; the convolution operation extracts local features by sliding a small filter or convolution kernel on the input data; batch normalization normalizes the input data on each small batch to reduce internal covariate shift, thereby accelerating the training process; the ReLU activation function is a nonlinear activation function defined as f(x)=max(0,x).

5. The method according to claim 1, characterized in that: A deep supervision mechanism is introduced into the UNet3+ deep learning network to accelerate the convergence speed of the network and improve the accuracy of position perception.

6. The method according to claim 5, characterized in that A supervised loss function is introduced in each encoder layer, the loss at each stage is calculated, and the loss function is updated through back propagation, thereby accelerating the convergence of the network.

7. The method according to claim 6, characterized in that The calculation stage loss formula is as follows: Where Ypredi is the predicted output of the i-th layer encoder, Ytrue is the true label, and L is the cross entropy loss function.

8. The method according to claim 1, characterized in that Through the channel attention mechanism and the spatial attention mechanism, the data feature expression of the feature map is improved, and more attention is paid to the basic features, thereby achieving accurate monitoring of image classification.

9. The method according to claim 8, characterized in that The channel attention mechanism is as follows: M avg (F)=σ(W1·W0·F c，avg ) M max (F)=σ(W1·W0·F c，max ) M c (F)=M avg (F)+M max (F) Among them, Mavg(F) and Mmax(F) are the average pooling features and the maximum pooling features respectively, σ is the sigmoid activation function, and W1 and W0 are the weights of the multi-layer perceptron (MLP); The spatial attention mechanism is as follows: M s (F)=σ(f 7×7 ([F s，avg ;F s,max ])) Among them, Fs,avg and Fs,max are spatial pooling features, f 7×7 It is a 7×7 convolutional layer.

Images