Multi-branch residual convolutional neural network model and its image classification method

Abstract

This invention belongs to the field of image classification technology, and relates to a multi-branch residual convolutional neural network model and its image classification method. The multi-branch residual convolutional neural network model includes a first classification unit, a second classification unit, a third classification unit, and a fourth classification unit connected in sequence; the second classification unit is composed of two stacked multi-branch residual structures; the third classification unit is composed of three stacked multi-branch residual structures. The model provided by this invention has the advantages of high accuracy, small model size, and short training time, and is used for image classification.

Description

Technical Field

[0001] This invention belongs to the field of image classification technology, and relates to a multi-branch residual convolutional neural network model and its image classification method. Background Technology

[0002] Image classification, as a computer vision technology, plays a pioneering role in the development of artificial intelligence. It is also ubiquitous in daily life, used in applications such as facial recognition, vehicle recognition, agricultural pest and disease identification, and machine vision. Among these, flower recognition has significant application value in areas such as flower cultivation and identifying invasive plants. Flower recognition technology can clearly locate weeds or worthless plants during flower cultivation, protecting the economic value of the planting area. Similarly, in real life, invasive plants can seriously impact local ecosystems, damaging biodiversity and human health. Examples include invasive plants such as Mikania micrantha, Forsythia suspensa, and Wedelia candida. Flower / plant recognition technology can be used to distinguish these plants and then take measures to control them, maintaining local ecological stability.

[0003] Image classification techniques can be categorized into multi-class image classification, fine-grained image classification, and multi-label image classification based on label categories. Fine-grained image classification involves further subdividing images after identifying broad categories; however, due to the more similar appearance and features of images, coupled with interference factors such as pose, viewing angle, lighting, occlusion, and background during the acquisition process, the data exhibits significant inter-class differences but small intra-class differences, making classification more difficult.

[0004] Yann LeCun's research team proposed improving Convolutional Neural Networks (CNNs) for image classification. The convolutional layers in the CNN play a crucial role in this improved classification because the results of each CNN layer can represent high-order features in the image. Since a convolutional layer consists of multiple one-dimensional or multi-dimensional filters, the filters can learn their parameters based on the sample labels, while the parameters of the operators are generally set by researchers based on experience. With the development of CNNs, the theory of the convolution process has been continuously enriched, expanding the application scope of CNNs. In addition to traditional convolutional kernels, various types of convolutional kernels, such as transposed convolution, dilated convolution, and grouped convolution, have been proposed.

[0005] Although existing CNN technology can be used for image classification, it has the following problems: (1) CNN models with high accuracy are large in size and difficult to move; (2) Networks with smaller model size have the disadvantage of insufficient accuracy; (3) Existing models take too long to train. Summary of the Invention

[0006] To address the technical problems of large model size, low accuracy, and long training time in existing CNN image classification, this invention provides a multi-branch residual convolutional neural network model, which features high accuracy, small model size, and short training time, and can be used for image classification.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] An image classification method based on a multi-branch residual convolutional neural network model includes the following steps:

[0009] A multi-branch residual convolutional neural network model includes a first classification unit, a second classification unit, a third classification unit, and a fourth classification unit connected in sequence; the second classification unit is composed of two stacked multi-branch residual structures; the third classification unit is composed of three stacked multi-branch residual structures.

[0010] Furthermore, the multi-branch residual structure includes a 1*1 first convolutional layer, a depth-separable convolutional pooling mechanism of the multi-branch variant, and a 1*1 second convolutional layer connected in sequence; the 1*1 first convolutional layer is connected to the previous classification unit; and the 1*1 second convolutional layer is connected to the next classification unit.

[0011] Furthermore, the depth-separable convolutional pooling mechanism of the multi-branch variant includes a first branch, a second branch, and a third branch in parallel; the 1*1 first convolutional layer is simultaneously connected to the 1*1 second convolutional layer via the first branch, the second branch, and the third branch.

[0012] Furthermore, the first branch includes a 3*3 depth-separable convolutional layer and a 3*3 first pooling layer connected in sequence; the second branch includes a 5*5 depth-separable convolutional layer and a 3*3 second pooling layer connected in sequence; and the third branch is a 3*3 third pooling layer.

[0013] Furthermore, the first classification unit sequentially includes a 7*7 convolutional layer and a 3*3 pooling layer; the 3*3 pooling layer is connected to the 1*1 first convolutional layer of the second unit.

[0014] Furthermore, the fourth classification unit includes a pooling layer and a fully connected layer in sequence; the pooling layer is connected to the 1*1 second convolutional layer of the third classification unit.

[0015] Furthermore, the pooling layer consists of a parallel adaptive max pooling layer and an adaptive average pooling layer; the adaptive max pooling layer is connected to the 1*1 second convolutional layer and the fully connected layer of the third classification unit, respectively; the adaptive average pooling layer is connected to the 1*1 second convolutional layer and the fully connected layer of the third classification unit, respectively.

[0016] An image classification method based on the aforementioned multi-branch residual convolutional neural network model includes the following steps:

[0017] 1) Establish the aforementioned multi-branch residual convolutional neural network model;

[0018] 2) Acquire image data and create a dataset;

[0019] 3) Randomly process the image data in the dataset from step 2) to obtain the original matrix A;

[0020] 4) Input the original matrix A from step 3) into the multi-branch residual convolutional neural network model from step 1) for training, extract image features, and perform image recognition to complete image classification.

[0021] Furthermore, the training process in step 4) is as follows:

[0022] 4.1) Input the original matrix A into the first classification unit, and obtain the denoised feature matrix A0 after sequential 7*7 convolution and 3*3 pooling;

[0023] 4.2) Input the denoised feature matrix A0 into the second classification unit, and after convolution and pooling through two stacked multi-branch residual structures, extract the matrix information to obtain the high-dimensional residual feature matrix A5;

[0024] 4.3) Input the high-dimensional residual feature matrix A5 into the third classification unit. After convolution and pooling through three stacked multi-branch residual structures, extract the matrix information to obtain the high-dimensional feature matrix A6 with invariance.

[0025] 4.4) The high-dimensional, invariant feature matrix A6 obtained above is simultaneously input into the max pooling layer and the average pooling layer. The information extracted by pooling is merged and then input into the fully connected layer. The extracted distribution features are combined to form a one-dimensional feature vector. The feature vector is then weighted and summed before being output to complete the classification.

[0026] Furthermore, the convolutions all include batch normalization and Mish activation function processing.

[0027] The beneficial effects of this invention are:

[0028] 1. The multi-branch residual convolutional neural network model used in this invention combines a multi-branch structure, a variant of depthwise separable convolution, and a ResNet network. While significantly reducing the number of model parameters and computational cost, it achieves high image recognition and classification accuracy, has a small model size, and short training time, making it widely applicable for image recognition and classification.

[0029] 2. This invention utilizes a multi-branch variant of a depthwise separable convolutional pooling mechanism. It employs multi-branch parallel processing, involving pooling after convolution with kernels of different scales and pooling downsampling of the original feature matrix. The feature matrices from each branch are then merged, followed by channel-wise and point-wise convolution. Using multiple branches for different operations increases feature diversity and neural network width. In one branch of the feature extraction stage, a 5x5 convolutional layer is used to expand the receptive field and improve classification accuracy.

[0030] 3. In the fourth classification unit, this invention first uses the results of parallel max pooling and average pooling layers to preserve the texture and contour of the image while preventing the loss of high-dimensional information and extracting invariant feature information. Then, the processed results are merged into a fully connected layer to combine the extracted distributed features and achieve classification, resulting in higher accuracy of the classification results. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the Inception-LightlyRNet lightweight convolutional neural network model;

[0032] Figure 2 This is a schematic diagram of a multi-branch residual module;

[0033] Figure 3A This is a comparison chart of the parameter quantities of each CNN in Example 1;

[0034] Figure 3B This is a comparison chart of the model volumes of various CNNs in Example 1;

[0035] Figure 3C This is a comparison chart of the training time for each CNN in Example 1;

[0036] Figure 4 A diagram illustrating the partitioning of the dataset. Detailed Implementation

[0037] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0038] The experimental environment for this invention was as follows: CentOS operating system, Intel(R) Xeon(R) E5-2640 processor, GeForce RTX 3090 graphics card, Python version 3.9.1, PyTorch version 1.10.0, and CUDA version 11.3.

[0039] The technical idea of ​​this invention is to establish a multi-branch residual convolutional neural network model for image recognition and classification. The multi-branch residual convolutional neural network model is composed of a multi-branch structure, a variant of depthwise separable convolution, and a nested and adapted residual module of the ResNet network, forming the multi-branch residual convolutional neural network model Inception-LightlyRNet.

[0040] The multi-branch residual module H(x) of this invention is defined as follows:

[0041] H(x)=R{F[P(F1(x))+P(F2(x))+P(x)]+x}(1)

[0042] Where R represents the activation function, F represents convolution, P represents max pooling, and x is the input feature matrix; F1(x) and F2(x) are 3*3 and 5*5 depth separable convolutional layers, respectively.

[0043] Existing depthwise separable convolutions separate feature extraction and feature aggregation into two steps, as in standard convolutions. Depthwise separable convolutions utilize channel-wise convolution, assigning a kernel to each channel for feature extraction, and then use pointwise convolution to fuse the information extracted from each channel. The calculation formula for depthwise separable convolution is as follows:

[0044]

[0045]

[0046] SepConv(W p W d ,y) (i,j) =Point wiseConv (i,j) (W p ,DepthwiseConv (i,j) (W d ,y))

[0047] Where: W is the convolution kernel, y is the input feature map, i and j are the input feature map resolutions i×j, k and l are the output feature map resolutions k×l, and m is the number of channels. (m) This represents a convolution with a kernel size of 1*1 and m channels; y (i，j，m) Let i*j be the input feature map with size i*j and m channels. This represents a convolution kernel of size W and y i*j The input feature map is convolved to obtain a feature map of size k*l; This represents the sum of all channels.

[0048] This invention employs a variant of depthwise separable convolution. First, it utilizes channel-wise convolution, followed by pooling after convolution with kernels of different scales, and then multi-branch parallel processing of the pooled downsampled original feature matrix. After merging the feature matrices from each branch, channel-wise convolution is performed, followed by pointwise convolution, aggregating the channel information of diverse features. The calculation formula for the variant of depthwise separable convolution is as follows:

[0049]

[0050]

[0051] SepConv(W p W d ,y) (i,j) =Point wiseConv (i,j) (W p ,DepthwiseConv (i,j) (W d ,y))

[0052] Where: W is the convolution kernel, P is the pooling kernel, y is the input feature map, i and j are the input feature map resolutions, k and l are the output feature map resolutions, and m is the number of channels. i For a convolution kernel of size i, W i =W3 and W5; W (m) For a convolution with a kernel size of 1*1 and m channels, y (i，j，m) The input feature map has size i*j and number of channels m; This represents a convolution kernel of size i and y i*j After convolution of the input feature map, pooling is performed to obtain a feature map of size k*l, W. i =W3 and W5; Wp represents point-wise convolution, and Wd represents depthwise separation convolution.

[0053] Furthermore, depthwise separable convolutions of the variant must satisfy the following condition:

[0054] (1) Ensure that each channel is assigned a convolution kernel during channel-wise convolution, and there are no redundant channels or convolution kernels;

[0055] (2) Ensure that the feature matrices obtained from processing each branch in the multi-branch structure remain consistent in dimension;

[0056] (3) Ensure that the feature matrices obtained from each branch are merged before performing pointwise convolution.

[0057] Activation layers are also a crucial component in neural convolutional networks. Activation functions can map feature matrices to new feature spaces, which is more conducive to training. More importantly, they introduce non-linear expressive power, enabling the network to understand and learn complex non-linear functions. Therefore, choosing an appropriate activation function can accelerate model convergence, improve the model's recognition accuracy and robustness, alleviate the gradient vanishing problem, enhance the network's non-linear expressive power, and give the network stronger learning and expressive capabilities.

[0058] See Figure 1 and Figure 2 The multi-branch residual convolutional neural network model provided by the present invention includes a first classification unit, a second classification unit, a third classification unit, and a fourth classification unit connected in sequence; the second classification unit is composed of two stacked multi-branch residual structures; and the third classification unit is composed of three stacked multi-branch residual structures.

[0059] The multi-branch residual structure includes a 1*1 first convolutional layer, a depth-separable convolutional pooling mechanism of the multi-branch variant, and a 1*1 second convolutional layer connected in sequence; the 1*1 first convolutional layer is connected to the previous classification unit; and the 1*1 second convolutional layer is connected to the next classification unit.

[0060] The depth-separable convolutional pooling mechanism of the multi-branch variant includes a first branch, a second branch, and a third branch in parallel; the 1*1 first convolutional layer is simultaneously connected to the 1*1 second convolutional layer via the first branch, the second branch, and the third branch.

[0061] The first branch consists of a 3x3 depth-separable convolutional layer and a 3x3 first pooling layer connected in sequence; the second branch consists of a 5x5 depth-separable convolutional layer and a 3x3 second pooling layer connected in sequence; and the third branch consists of a 3x3 third pooling layer.

[0062] The 1*1 first convolutional layer is connected to the 1*1 second convolutional layer via a 3*3 depth-separable convolutional layer and a 3*3 pooling layer in the first branch; the 1*1 first convolutional layer is connected to the 1*1 second convolutional layer via a 5*5 depth-separable convolutional layer and a 3*3 pooling layer in the second branch; the 1*1 first convolutional layer is connected to the 1*1 second convolutional layer via a 3*3 pooling layer in the third branch.

[0063] The first classification unit consists of a 7*7 convolutional layer and a 3*3 pooling layer; the 3*3 pooling layer is connected to the 1*1 first convolutional layer of the second unit.

[0064] The fourth classification unit consists of a pooling layer and a fully connected layer; the pooling layer is connected to the 1*1 second convolutional layer of the third classification unit.

[0065] The pooling layers consist of parallel adaptive max pooling and adaptive average pooling layers; the adaptive max pooling layers are connected to the 1*1 second convolutional layer and the fully connected layer of the third classification unit, respectively; the adaptive average pooling layers are connected to the 1*1 second convolutional layer and the fully connected layer of the third classification unit, respectively.

[0066] This invention designs a lightweight CNN model with a multi-branch structure and depthwise separable convolution as its core, namely the multi-branch residual convolutional neural network model Inception-LightlyRNet. In the feature extraction stage, to increase feature diversity, multiple branches are used to perform different operations to increase the network width; to expand the receptive field, one branch uses a 5*5 convolutional layer; to reduce the number of model parameters, depthwise separable convolution is introduced into the model design; in the classification stage, the results processed by adaptive max pooling and adaptive average pooling layers are first merged, preserving the image texture and contour while preventing the loss of high-dimensional information, extracting invariant feature information, and then a fully connected layer is designed to combine distributed features and achieve classification.

[0067] The image classification method based on a multi-branch residual convolutional neural network model provided by this invention includes the following steps:

[0068] 1) Establish a multi-branch residual convolutional neural network model;

[0069] 2) Acquire image data and create a dataset;

[0070] 3) Randomly process the image data in the dataset from step 2) to obtain the original matrix A; random processing refers to cropping, rotating, reversing, and standardizing the acquired data according to random sizes.

[0071] 4) Input the original matrix A from step 3) into the multi-branch residual convolutional neural network model from step 1) for training, extract image features, and perform image recognition to complete image classification.

[0072] The training process in step 4) of this invention is as follows:

[0073] 4.1) Input the original matrix A into the first classification unit, and obtain the denoised feature matrix A0 after sequential 7*7 convolution and 3*3 pooling;

[0074] 4.2) Input the denoised feature matrix A0 into the second classification unit, and after convolution and pooling through two stacked multi-branch residual structures, extract the matrix information to obtain the high-dimensional residual feature matrix A5;

[0075] 4.3) Input the high-dimensional residual feature matrix A5 into the third classification unit. After convolution and pooling through three stacked multi-branch residual structures, extract the matrix information to obtain the high-dimensional feature matrix A6 with invariance.

[0076] 4.4) The high-dimensional, invariant feature matrix A6 obtained above is simultaneously input into the max pooling layer and the average pooling layer. The information extracted by pooling is merged and then input into the fully connected layer. The extracted distribution features are combined to form a one-dimensional feature vector. The feature vector is then weighted and summed before being output to complete the classification.

[0077] The specific training process in step 4.2) of this invention is as follows:

[0078] 4.2.1) The denoised feature matrix A0 is subjected to a 1*1 first convolution to obtain the dimension-reduced matrix A1;

[0079] 4.2.2) The dimensionality reduction matrix A1 enters the first branch and is sequentially processed by 3*3 depthwise separable convolution and 3*3 pooling to extract the main feature matrix A21 of the image;

[0080] 4.2.3) The dimension reduction matrix A1 enters the second branch, and the feature matrix information of the image is extracted by a 5*5 depthwise separable convolution, and then the wide receptive field matrix A22 is extracted by 3*3 pooling.

[0081] 4.2.4) The dimension reduction matrix A1 enters the third branch, and is downsampled by 3*3 pooling to retain the image texture matrix A23;

[0082] 4.2.5) Combine the main feature matrix A21 extracted in step 4.2.2), the wide receptive field matrix A22 extracted in step 4.2.3), and the image texture matrix A23 extracted in step 4.2.4), and perform a 1*1 second convolution pointwise convolution to increase the data dimension and obtain the multi-branch information matrix A3;

[0083] 4.2.6) Merge the multi-branch information matrix A3 from step 4.2.5) with the feature matrix A0 after dimensional transformation to obtain the residual feature matrix A4;

[0084] 4.2.7) Repeat the training of residual feature matrix A4 once according to steps 4.2.1) to 4.2.6) to obtain high-dimensional residual feature matrix A5.

[0085] In this invention, the high-dimensional residual feature matrix A5 enters the third classification unit and is trained three times in a loop according to the process of steps 4.2.1) to 4.2.6), resulting in a high-dimensional feature matrix A6 with invariance.

[0086] During training, this invention performs batch normalization and Mish activation function processing in addition to the convolutional processing itself. For example, a 1x1 convolution involves first performing a 1x1 convolution on the feature matrix, then batch normalization, and finally Mish activation function processing. A 3x3 depthwise separable convolution includes 3x3 convolution, batch normalization, and Mish activation function processing. Batch normalization is essentially a batch standardization operation.

[0087] In traditional and classic networks, ReLU is commonly used as the activation function. Although ReLU has advantages such as fast convergence speed, no gradient vanishing / saturation problem when x>0, and low computational complexity, it will cause the "dead neuron" phenomenon when x<0: when x<0, the gradient is 0, and the corresponding parameters will never be learned and updated.

[0088] In the Inception-LightlyRNet multi-branch residual convolutional neural network model of this invention, Mish is chosen as the activation function. While the Mish activation function shares the same unbounded upper and bounded lower characteristics as the ReLU activation function, avoiding gradient saturation caused by capping and thus accelerating the training process, its non-monotonicity helps maintain small negative values ​​to stabilize the network gradient flow. As a smooth function, it possesses good generalization ability and result optimization capabilities, significantly improving network performance. The formula for the Mish activation function is as follows:

[0089] Mish(x)=x×Tanh(Softplus(x))

[0090] The convolutional layer designed in this invention: In CNNs, the convolutional kernel is essentially an updatable weight matrix. The convolution operation mainly involves convolving the kernel with learnable parameters with the feature map of the previous layer to obtain the output feature map. The convolutional layer is primarily used to extract features from the input signal; different convolutional kernels extract different features. The formula for the convolution operation of the output feature map is as follows:

[0091]

[0092] Where: i represents the number of input channels, j represents the number of output channels; F i (l) b represents the convolutional input feature map of the i-th input channel of the l-th layer of the CNN. j It is the bias term of the j-th output channel, k ij Let i represent the number of input channels and j represent the number of output channels of the convolution kernel. The asterisk (*) in the formula represents the convolution operation, and f is the activation function of the CNN. i (l) This represents the i-th feature map of the (l+1)-th layer of the CNN; This represents the convolutional output feature map of the j-th output channel of the (l+1)-th layer.

[0093] This invention employs batch normalization and pooling layers: During network training, the distribution of hidden layer parameters in the model frequently changes, leading to significant differences in parameter distributions across different network layers, known as covariate drift. To address this issue, batch normalization (BN) technology has been increasingly applied in the design and optimization of deep neural network structures.

[0094] First, during the training of deep neural networks, it can resolve covariate drift, thus making the training of deep neural networks more stable. Second, the Batch Normalization (BN) method can accelerate the convergence speed of the network. Finally, BN can also act as a regularization mechanism. Specifically, the BN operation involves transforming the activation value of each neuron in the hidden layer as follows:

[0095]

[0096] (k)

[0097] x is the k-dimensional input signal of the hidden layer; E is the average value.

[0098] The pooling layer provided in this invention, also known as a downsampling layer, is generally placed after a convolutional layer and is a parameterless layer. The pooling layer is primarily used to reduce the resolution of the feature maps obtained by the convolutional layers, thus obtaining spatially invariant features, thereby reducing the amount of data processing and accelerating the training speed of the neural network. In the Inception-LightlyRNet network design, both max pooling and adaptive pooling methods are used for different network structure layers.

[0099] The fourth classification unit provided by this invention includes parallel adaptive max pooling layers and adaptive average pooling layers, connected by a fully connected layer. Max pooling refers to preserving the maximum value within the receptive field after pooling, while average pooling refers to preserving the average value of all data within the receptive field after pooling; these are well known in the art.

[0100] This network structure effectively reduces the number of model parameters and computational cost while significantly improving detection accuracy and speed. Assuming the input data is M×M×N, the convolution kernel is K×K×P, and the stride is 1, the standard convolution parameter count W... SC for:

[0101] W SC =K×K×N×P

[0102] And the corresponding computational cost O SC for:

[0103] OSC =M×M×N×K×K×P

[0104] The number of parameters W in DSC DSC for:

[0105] W DSC =K×K×N+N×P

[0106] And the corresponding computational cost O DSC for:

[0107] O DSC =M×M×N×K×K+M×M×N×P

[0108] Therefore, for the two structures with a parameter ratio F W And the computational complexity ratio F O They are respectively:

[0109]

[0110]

[0111] In the Inception-LightlyRNet network, when a 5x5 convolutional kernel is used (K=5), the number of parameters in DSC can be reduced to approximately [a smaller number] compared to the traditional convolution process. However, since the Inception-LightlyRNet model of this invention uses a multi-branch variant of depthwise separable convolution, the reduction in the number of parameters of the model is limited.

[0112] The fourth classification unit of the Inception-LightlyRNet network model utilizes adaptive max pooling and adaptive average pooling to preserve image texture and contours while preventing the loss of high-dimensional information, extracting invariant feature information. The fully connected layer is equivalent to the hidden layer of a multilayer perceptron model; every neuron in the next layer is connected to every neuron in the previous layer. Neurons within the same layer are independent and have no connections. In the fourth classification unit of the Inception-LightlyRNet model, the input is a one-dimensional feature vector formed by combining and expanding the distributed features learned by the previous CNN, and the output is a weighted sum of these feature vectors. The definition of a fully connected layer is as follows:

[0113] a i =W i1 *x1+W i2 *x2+…+W in *x n +b i

[0114] Where, x iFor the input of the i-th channel, i = 1, 2, ..., n; a i For the output of the i-th channel, W ij Let b be the weight parameter for the i-th channel, i = 1, 2, ..., n; i It is a bias parameter.

[0115] In summary, the multi-branch residual convolutional neural network model provided by this invention achieves better results in terms of accuracy, model size, and training time when classifying images. It can improve the accuracy of image recognition and significantly reduce training time while greatly reducing the number of model parameters and computational cost.

[0116] The images with different degrees of multi-branch residual convolutional neural network model provided in this invention are classified below.

[0117] Example 1: Lightweight Model Analysis

[0118] To evaluate the number of parameters and model size, the convolutional layers, bias layers, and fully connected layers involved in the Inception-LightlyRNet model are calculated layer by layer, and then summed to evaluate the training time of the model.

[0119] We also compared Inception-LightlyRNet with four classic CNNs.

[0120] A comparison of the training time of four classic CNN models: Resnet50, MobleNetV2, ShuffleNetV2, and EfficientNetV2.

[0121] During the evaluation, a 3-channel 224*224 tensor was uniformly used as the input, and the summary resource package was imported for analysis. The parameter count and model size of the four convolutional neural network models were analyzed. The results are shown in Figure 3.

[0122] Depend on Figure 3AThe results shown indicate that the Inception-LightlyRNet model designed in this invention has 7.50M parameters, while the classic ResNet50 model has 25.55M parameters, and the EfficientNetV2 model has 21.46M parameters. Meanwhile, the lightweight models MobleNetV2 and ShuffleNetV2 have 3.51M and 2.28M parameters, respectively. Compared to the ResNet50 model, Inception-LightlyRNet reduces the number of parameters by 70.64%, and compared to the EfficientNetV2 model, it reduces the number of parameters by 65.05%. However, its number of parameters is still slightly larger than that of the mainstream lightweight deep neural network models MobleNetV2 and ShuffleNetV2.

[0123] exist Figure 3B In the model comparison, with a 3-channel 224*224 tensor as input, the Inception-LightlyRNet model size is 20.98M, while the models of ResNet50, EfficientNetV2, MobleNetV2, and ShuffleNetV2 are 90.02M, 77.85M, 8.74M, and 4.97M, respectively. Compared to ResNet50 and EfficientNetV2, the Inception-LightlyRNet model size is reduced by 76.7% and 73.06%, respectively. The Inception-LightlyRNet model shows a significant difference in size compared to MobleNetV2.

[0124] exist Figure 3C In the training test, with a 3-channel 224*224 tensor as input and 200 iterations, Inception-LightlyRNet took 1536 seconds to train. In contrast, ResNet50, EfficientNetV2, MobleNetV2, and ShuffleNetV2 took 2502 seconds, 4881 seconds, 2089 seconds, and 2284 seconds, respectively. Compared to ShuffleNetV2 and EfficientNetV2, Inception-LightlyRNet reduced training time by 32.75% and 65.73%, respectively. Therefore, Inception-LightlyRNet significantly reduces training time compared to ResNet50, EfficientNetV2, and popular lightweight models.

[0125] Example 2

[0126] This embodiment takes flower image classification as an example. The flower data obtained comes from the Heywhale community flower dataset, which can be downloaded from the Heywhale website at http: / / download.tensorflow.org / example_images / flower_photos.tgz.

[0127] The dataset contains 3,670 flower images categorized into 5 classes, including 633 daisies, 898 dandelions, 641 roses, 699 sunflowers, and 799 tulips.

[0128] See Figure 4 In the classification process, a dataset segmentation procedure was used to randomly divide the flower images into training, validation, and test sets in an 8:1:1 ratio. The training set contained 2939 images of all categories, while the validation and test sets each contained 367 images of all categories. The acquired data was preprocessed by randomly cropping, rotating, flipping, and standardizing the data. Then, the Inception-LightlyRNet model provided in this invention was used, and the resulting image information matrix was input into the network model to complete the classification. The number of model parameters, processing time, and accuracy were recorded, and the results are shown in Table 1.

[0129] Meanwhile, four classic CNN models—Resnet50, MobleNetV2, ShuffleNetV2, and EfficientNetV2—were used to identify flower images. The number of model parameters, processing time, and accuracy were recorded and compared. The results are shown in Table 1.

[0130] Table 1. Recognition performance of five types of flower images

[0131] network accuracy Model volume (Mb) Time (s) Inception-LightlyRNet 0.916 20.98 1536 Resnet50 0.851 90.02 2502 Mobilenetv2 0.816 8.74 2089 Shufflenet v2 0.811 4.97 2284 Efficientnetv2 0.866 77.85 4881

[0132] Table 1 shows that the Inception-LightlyRNet network performs well even with data interference, achieving the highest recognition accuracy of 91.6%. Its training time is reduced by 26%-32% compared to ResNet50, ShuffleNetV2, and MobileNetV2, and by approximately 69% compared to EfficientNetV2. The mainstream lightweight networks ShuffleNetV2 and MobileNetV2 have accuracies of 81.1% and 81.6% respectively, significantly lower than Inception-LightlyRNet; their training times are also considerably longer.

[0133] Example 3

[0134] Unlike Example 2, this example uses 17 flower classification images as input to the CNN model. The flower data comes from a data resource distribution center (data mart) and can be downloaded from the URL https: / / www.robots.ox.ac.uk / ~vgg / data / flowers / 102 / index.html provided by the data mart.

[0135] The data consists of 2391 images of 17 types of flowers selected from the dataset, including 100 images of California poppy, 63 images of dahlia, 120 images of gerbera, 180 images of nasturtium, 168 images of petunia, 102 images of anthurium, 165 images of cockscomb, 189 images of water lily, 120 images of gladiolus, 189 images of sugar mustard, 80 images of bird of paradise, 111 images of geranium, 246 images of passionflower, 72 images of gazania, 198 images of poinsettia, 165 images of rose, and 123 images of hibiscus.

[0136] In the classification process, a dataset segmentation procedure was used to randomly divide the flower images into training, validation, and test sets in an 8:1:1 ratio. On the same test set, five classic convolutional neural networks—ResNet50, EfficientNetV2, Inception-LightlyRNet, MobleNetV2, and ShuffleNetV2—were used for feature extraction and classification. The classification results are shown in Table 2.

[0137] Table 2 shows the recognition performance of the 17-category flower images in the dataset.

[0138] network accuracy Model volume (Mb) Time (s) Inception-LightlyRNet 0.987 21.08 1183 Resnet50 0.906 90.11 1723 Mobilenetv2 0.940 8.80 1465 Shufflenet v2 0.906 5.01 1552 Efficientnetv2 0.948 77.91 3229

[0139] As can be seen from the classification results in Table 2, Inception-LightlyRNet and EfficientNetV2 have relatively high accuracies of 98.7% and 94.8% respectively, while MobleNetV2 has a slightly lower accuracy of only 94.0%. ResNet50 and ShuffleNetV2 have lower accuracies of 90.6%. Furthermore, combined with the results regarding the parameter count and model size of the five CNN models in the previous section, Inception-LightlyRNet significantly improves accuracy and greatly reduces training time compared to mainstream lightweight models, indicating that the neural network model design provided in this invention is reasonable and effective.

[0140] Example 4

[0141] In this embodiment, the dataset consists of insect classification images as input to the convolutional neural network model. The obtained insect image data comes from the Heywhale community image dataset, which can be downloaded from the Heywhale website at https: / / www.heywhale.com / mw / dataset / 5f50c54f368dfa003105bb5d / file.

[0142] The dataset contains a total of 5,969 images in three categories: bees, wasps, and other insects. Among them, there are 714 images of bees, 2,816 images of wasps, and 2,439 images of other insects.

[0143] In this embodiment, a dataset segmentation program is used to randomly divide insect images into training, validation, and test sets in an 8:1:1 ratio during classification.

[0144] On the same test set, five classic convolutional neural networks—ResNet50, EfficientNetV2, Inception-LightlyRNet, MobleNetV2, and ShuffleNetV2—were used for feature extraction and classification. The classification results are shown in Table 3.

[0145] Table 3. Recognition results of insect classification images

[0146] network accuracy Model volume (Mb) Time (s) Inception-LightlyRNet 0.980 20.97 1546 Resnet50 0.970 90.00 1873 Mobilenet v2 0.966 8.73 1646 Shufflenet v2 0.956 4.96 1777 Efficientnet v2 0.975 77.84 3904

[0147] As shown in Table 3, using the same test set, Inception-LightlyRNet performs best in terms of accuracy and training time, at 98.00% and 1546s, respectively. In terms of accuracy, Inception-LightlyRNet is comparable to EfficientNetV2. Combined with the previous section's conclusion that EfficientNetV2 has 2.86 times more parameters and 3.71 times more model size than Inception-LightlyRNet, it can be seen that Inception-LightlyRNet, despite its smaller size and shorter training time, exhibits strong classification performance.

[0148] Example 5

[0149] In this embodiment, the dataset uses vehicle classification images as input to the convolutional neural network model. The acquired vehicle image data comes from the Heywhale community image dataset, which can be downloaded from the Heywhale website at https: / / www.heywhale.com / mw / dataset / 5f6b00ff71c700003079dbc2 / file.

[0150] The dataset contains 4028 vehicle images across five categories: chevrolet_impala_2008, chevrolet_silverado_2004, ford_f150_2006, honda_accord_1997, and honda_civic_1998. Specifically, there are 875 images in chevrolet_impala_2008, 815 in chevrolet_silverado_2004, 879 in ford_f150_2006, 647 in honda_accord_1997, and 812 in honda_civic_1998.

[0151] In this embodiment, a dataset segmentation program is used to randomly divide vehicle images into training, validation, and test sets in an 8:1:1 ratio during classification.

[0152] On the same test set, five classic convolutional neural networks—ResNet50, EfficientNetV2, Inception-LightlyRNet, MobleNetV2, and ShuffleNetV2—were used for feature extraction and classification. The classification results are shown in Table 4.

[0153] Table 4 Recognition Results of Vehicle Classification Images

[0154]

[0155]

[0156] Table 4 shows that when vehicle images are used as input to the convolutional neural network model, and the same test set is used, Inception-LightlyRNet performs well in terms of accuracy and training time. Inception-LightlyRNet and EfficientNetV2 have similar accuracy figures, but Inception-LightlyRNet has a smaller model size and significantly shorter training time. It is evident that the traditional ResNet50 network model is large and has a long training time; the two mainstream lightweight networks have significant drawbacks: ShuffleNetV2 and MobileNetV2 suffer from low recognition accuracy.

[0157] In summary, the above four experimental comparisons demonstrate that, regardless of the dataset, the Inception-LightlyRNet network model designed and constructed in this invention has significant advantages in terms of recognition accuracy and training time. It can improve the image recognition accuracy and accelerate training time while minimizing the number of model parameters and computational load.

[0158] Example 6: Comparison of recognition results with different learning rates (lr)

[0159] 1) For image classification, we used the Inception-LightlyRNet method to conduct experiments on different learning rate parameters. The results are shown in Table 5.

[0160] First, different learning rate parameters were used during training. Then, the Inception-LightlyRNet model was trained for 200 iterations and exported. Finally, the trained Inception-LightlyRNet models with different learning rates were applied to the test set for performance testing. The classification results on the test set are shown in Table 5.

[0161] Table 5. Impact of Learning Rate on Performance

[0162] Learning rate (lr) accuracy time 0.0001 0.916 1536 0.0005 0.920 1567 0.001 0.929 1587

[0163] As shown in Table 5, it is evident that for the Inception-LightlyRNet model with its multi-branch structure, using a larger learning rate can significantly improve recognition accuracy with a slight increase in training time. Using 224*224 three-channel image data as input, the highest recognition accuracy of Inception-LightlyRNet is 92.9% when lr = 0.001. Even when lr = 0.0005, the network's recognition accuracy is still higher than the 91.6% achieved with the original learning rate of 0.0001. This indicates that when the feature matrix has diversity and the network width increases, using a larger learning rate can significantly improve network performance without excessively affecting the network's good performance in terms of training time.

[0164] The Inception-LightlyRNet provided by this invention significantly reduces the number of parameters and computational cost. Using a multi-branch structure with lr=0.001, residual learning, and depthwise separable convolution techniques, Inception-LightlyRNet achieves an accuracy of 92.9% with a training time of 1587 seconds. However, increasing the learning rate may lead to getting stuck in local optima. The image classification model provided by this invention demonstrates that depth and width have a significant impact on network performance. Furthermore, it not only designs a lightweight and efficient model but also rationally designs the network's depth and width. The Inception-LightlyRNet model constructed and designed in this invention has significant practical application value.

[0165] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention; however, the technical scope protected by the present invention is not limited thereto. For those skilled in the art, any simple deductions or substitutions made based on the technical solutions of the present invention without departing from the inventive concept should fall within the protection scope of the present invention.

Claims

1. An image classification method based on a multi-branch residual convolutional neural network model, characterized in that, The method comprises the following steps: 1) establishing a multi-branch residual convolutional neural network model; The multi-branch residual convolutional neural network model comprises a first classification unit, a second classification unit, a third classification unit and a fourth classification unit connected in sequence; the second classification unit is composed of two multi-branch residual structure stacks; the third classification unit is composed of three multi-branch residual structure stacks; The multi-branch residual structure comprises a 1*1 first convolutional layer, a multi-branch variant deep separable convolutional pooling mechanism and a 1*1 second convolutional layer connected in sequence; the 1*1 first convolutional layer is connected with the previous classification unit; the 1*1 second convolutional layer is connected with the next classification unit; The multi-branch variant deep separable convolutional pooling mechanism comprises a first branch, a second branch and a third branch arranged side by side; the 1*1 first convolutional layer is connected with the 1*1 second convolutional layer through the first branch, the second branch and the third branch; The first branch comprises a 3*3 deep separable convolutional layer and a 3*3 first pooling layer connected in sequence; the second branch comprises a 5*5 deep separable convolutional layer and a 3*3 second pooling layer connected in sequence; and the third branch is a 3*3 third pooling layer; The first classification unit comprises a 7*7 convolutional layer and a 3*3 pooling layer connected in sequence; the 3*3 pooling layer is connected with the 1*1 first convolutional layer of the second unit; The fourth classification unit comprises a pooling layer and a fully connected layer connected in sequence; the pooling layer is connected with the 1*1 second convolutional layer of the third classification unit; The pooling layer is an adaptive maximum pooling layer and an adaptive average pooling layer arranged side by side; the adaptive maximum pooling layer is connected with the 1*1 second convolutional layer and the fully connected layer of the third classification unit respectively; and the adaptive average pooling layer is connected with the 1*1 second convolutional layer and the fully connected layer of the third classification unit respectively; 2) obtaining image data and establishing a data set; 3) performing random processing on the image data in the data set of step 2) to obtain an original matrix A; 4) inputting the original matrix A of step 3) into the multi-branch residual convolutional neural network model of step 1) for training, extracting image features, and identifying the image to complete image classification.

2. The method of claim 1, wherein the multi-branch residual convolutional neural network model-based image classification method is characterized by, The training process of step 4) is as follows: 4.1) inputting the original matrix A into the first classification unit, and obtaining a denoising feature matrix A0 through 7*7 convolution and 3*3 pooling in sequence; 4.2) inputting the denoising feature matrix A0 into the second classification unit, extracting matrix information through convolution and pooling of two stacked multi-branch residual structures, and obtaining a high-dimensional residual feature matrix A5; 4.3) inputting the high-dimensional residual feature matrix A5 into the third classification unit, extracting matrix information through convolution and pooling of three stacked multi-branch residual structures, and obtaining a high-dimensional feature matrix A6 with invariance; 4.4) inputting the high-dimensional feature matrix A6 with invariance obtained above into the maximum pooling layer and the average pooling layer at the same time, inputting the pooled information into the fully connected layer after merging, combining the extracted distribution features, forming a one-dimensional feature vector, and outputting after weighted summation of the feature vector, to complete the classification.

3. The method of claim 2, wherein the multi-branch residual convolutional neural network model is a ResNet model. The convolution further comprises batch normalization processing and Mish activation function processing.

Images