Image classification method based on lightweight residual attention enhancement network LRAE-Unet model

Abstract

This paper proposes an image classification method based on a lightweight residual attention enhancement network (LRAE-Unet) model, comprising: obtaining a training image dataset; constructing a lightweight residual attention enhancement network (LRAE-Unet) model, which includes a shallow feature extraction pathway and a deep feature extraction pathway; training the model using the LRAE-Unet model and the training image dataset to generate a classification model; and obtaining an image to be classified, inputting the image to be classified into the classification model, and obtaining an image classification result. The method proposed in this paper can fully automatically classify images, achieving pixel-level classification without being restricted by the image category. Furthermore, the network model has few parameters, strong adaptability, and excellent model performance, effectively improving the accuracy and convenience of image classification.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing and computer vision. Background Art

[0002] Traditional image classification algorithms typically begin by manually designing relatively specific features, then extracting them, and finally classifying them using a classifier. Alternatively, a more powerful classifier can be manually designed to improve classification performance, thereby achieving image classification. However, compared to deep learning, many existing methods can only extract shallow features from the original image. These features are not robust enough for classification tasks, whereas deep learning can extract more complex and abstract features layer by layer from the raw data. These features are richer and more robust, thereby improving classification accuracy.

[0003] The basic idea of ​​deep learning mainly includes using multiple nonlinear operation layers to build a deep learning network. The bottom layer of the network can learn detailed appearance information from the original data. The output of the bottom layer is directly used as the input of the high-level layer, thereby adaptively learning high-order, more abstract and more complex feature representations from the input data layer by layer. Finally, these feature representations are used to solve classification problems. A large number of experiments have shown that deep learning networks have very superior performance in image classification.

[0004] Among related technologies, deep learning initially focused on simple image classification tasks, such as MNIST handwriting classification. The 2012 ImageNet competition saw the first application of deep learning to this complex image classification task. The ImageNet dataset consists of 1 million images across 1,000 categories, with a resolution of approximately 300x300 pixels. This competition improved image classification accuracy from 74.3% to 84.7%. In the 2014 ImageNet competition, the championship went to the deep learning model GoogleNet, with a classification accuracy of 93.4%. Finally, deep residual learning models have raised ImageNet classification accuracy to over 95%, nearly approaching human performance.

[0005] However, the models mentioned above are all developing in the direction of deeper and more complex networks, and there is no proof that relatively simplified models cannot achieve the above effects. Therefore, there is still a lot of room for exploration in the design of deep learning network models for image classification. This patent proposes an image classification method and system based on a lightweight residual attention enhancement network LRAE-Unet model. This method can achieve pixel-level classification purposes without being restricted by the category of the image to be classified. In addition, the network model has few parameters, strong adaptability, and good model performance, thereby effectively improving the accuracy and convenience of image classification and obtaining good classification results. Summary of the Invention

[0006] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0007] To this end, the purpose of the present invention is to propose an image classification method based on a lightweight residual attention enhancement network (LRAE-Unet) model to improve the accuracy and convenience of image classification.

[0008] To achieve the above objectives, the first embodiment of the present invention proposes an image classification method based on a lightweight residual attention enhancement network (LRAE-Unet) model, comprising:

[0009] Get a training image dataset;

[0010] Constructing a lightweight residual attention enhancement network (LRAE-Unet) model, wherein the lightweight residual attention enhancement network (LRAE-Unet) model includes a shallow feature extraction pathway and a deep feature extraction pathway;

[0011] Performing model training using the lightweight residual attention enhancement network (LRAE-Unet) model and the training image dataset to generate a classification model;

[0012] Obtain an image to be classified, input the image to be classified into the classification model, and obtain an image classification result.

[0013] In addition, the image classification method based on the lightweight residual attention enhancement network LRAE-Unet model according to the above embodiment of the present invention may also have the following additional technical features:

[0014] Furthermore, in one embodiment of the present invention, the shallow feature extraction path includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, and a first downsampling module, a second downsampling module, a third downsampling module, and a fourth downsampling module, wherein the first convolution block, the second convolution block, the third convolution block, and the fourth convolution block each contain three consecutive convolution layers; the FMs scale output by each convolution module of the shallow feature extraction path is calculated as:

[0015] C=M*2,

[0016] Among them, C×C is the FMs scale of the convolutional layer output, and M×M is the FMs scale of the convolutional layer input.

[0017] Furthermore, in one embodiment of the present invention, the deep feature extraction path includes a first convolution block, a second convolution block, a third convolution block, and a fourth convolution block, and the first convolution block, the second convolution block, the third convolution block, and the fourth convolution block each include seven consecutive convolution layers; the FMs scale of each convolution block output of the deep feature extraction path is calculated as:

[0018] C=M / 2,

[0019] Among them, C×C is the FMs scale of the convolutional layer output, and M×M is the FMs scale of the convolutional layer input.

[0020] Furthermore, in one embodiment of the present invention, constructing a lightweight residual attention enhancement network LRAE-Unet model further includes:

[0021] In the shallow feature extraction path, batch normalization is applied to the weights of each convolutional layer of the shallow feature extraction path, wherein,

[0022]

[0023] Among them, y is a batch of FMs with a mean of 0 and a variance of 1, x represents the input, mean[x] represents the average value of the input, Var[x] represents the variance of the input, eps represents the stability coefficient to prevent the denominator from being 0, γ and β are adjustments to the pixels based on BN, the initial value of γ is 1, which is used to adjust the variance, and the initial value of β is 0, which is used to adjust the mean.

[0024] Furthermore, in one embodiment of the present invention, the performing model training using the lightweight residual attention enhancement network LRAE-Unet model and the training image dataset includes:

[0025] The lightweight residual attention enhancement network LRAE-Unet model is trained using the stochastic gradient descent method, and the model weights are adjusted using the learning rate function, and the Pytorch deep learning framework is used, wherein,

[0026] The parameter setting form of the stochastic gradient descent method is as follows:

[0027] sgd=SGD(lr=lr_base, momentum=0.9, nesterov=True),

[0028] The learning rate function is as follows:

[0029]

[0030] Where SGD(·) is the stochastic gradient descent method in the Pytorch deep learning framework, momentum is the momentum parameter, nesterov = True indicates the use of nesterov momentum, lr is the learning rate for each round of training, lr_base indicates the learning rate at the beginning of training, epoch is the number of training rounds, epoches is the total number of training rounds, and lr_power is the exponential parameter.

[0031] Furthermore, in one embodiment of the present invention, the model training using the lightweight residual attention enhancement network LRAE-Unet model and the training image dataset further includes:

[0032] Use EarlyStopping to control the training process. When the accuracy of the validation set no longer improves or the number of training rounds reaches the maximum, the model training is terminated.

[0033] To achieve the above objectives, the second embodiment of the present invention proposes an image classification device based on a lightweight residual attention enhancement network LRAE-Unet model, comprising the following modules:

[0034] An acquisition module is used to obtain a training image dataset;

[0035] A construction module for constructing a lightweight residual attention enhancement network LRAE-Unet model, wherein the lightweight residual attention enhancement network LRAE-Unet model includes a shallow feature extraction path and a deep feature extraction path;

[0036] A training module, configured to perform model training using the lightweight residual attention enhancement network LRAE-Unet model and the training image dataset to generate a classification model;

[0037] The classification module is used to obtain the image to be classified, input the image to be classified into the classification model, and obtain the image classification result.

[0038] To achieve the above-mentioned purpose, the third aspect of the present invention proposes a computer device, characterized in that it includes a memory, a processor, and a computer program stored on the memory and runnable on the processor. When the processor executes the computer program, it implements the above-mentioned image classification method based on the lightweight residual attention enhancement network LRAE-Unet model.

[0039] To achieve the above-mentioned purpose, the fourth aspect of the present invention proposes a computer-readable storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, it implements the above-mentioned image classification method based on the lightweight residual attention enhancement network LRAE-Unet model.

[0040] The image classification method based on the lightweight residual attention enhancement network LRAE-Unet model proposed in the embodiment of the present invention obtains the image classification model of the lightweight residual attention enhancement network LRAE-Unet through convolutional neural network theory and deep learning network model training method, thereby realizing end-to-end classification, and can be used to classify images fully automatically to achieve pixel-level classification purposes, without being restricted by the category of the image to be classified, with strong applicability, good model performance, and robustness, making classification not only more convenient and effective, but also simple to operate and with better performance, thereby effectively improving the accuracy and convenience of image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0042] Figure 1 Flowchart of an image classification method using a lightweight residual attention enhancement network LRAE-Unet according to an embodiment of the present invention;

[0043] Figure 2 Flowchart of an image classification method using a lightweight residual attention enhancement network (LRAE-Unet) according to an embodiment of the present invention;

[0044] Figure 3 Schematic diagram of green plant pictures under different backgrounds according to one embodiment of the present invention;

[0045] Figure 4 Schematic diagram of pictures of different types of green plants according to one embodiment of the present invention;

[0046] Figure 5 Schematic diagram of a lightweight residual attention enhancement network LRAE-Unet model framework according to one embodiment of the present invention;

[0047] Figure 6 2 is a schematic diagram of the structure of a shallow feature extraction pathway according to one embodiment of the present invention;

[0048] Figure 7 A schematic diagram of the structure of a deep feature extraction pathway according to an embodiment of the present invention;

[0049] Figure 8 A schematic diagram of the fusion of shallow information and deep information according to an embodiment of the present invention;

[0050] Figure 9 Schematic diagram of the connection between the convolution block and the pooling layer according to one embodiment of the present invention;

[0051] Figure 10 Schematic diagram of the structure of an image classification system based on a lightweight residual attention enhancement network LRAE-Unet according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0053] The following describes an image classification method based on a lightweight residual attention enhancement network LRAE-Unet model according to an embodiment of the present invention with reference to the accompanying drawings.

[0054] Figure 1 A flowchart of an image classification method based on a lightweight residual attention enhancement network LRAE-Unet model provided in an embodiment of the present invention.

[0055] like Figure 1 As shown in FIG, the image classification method based on the lightweight residual attention enhancement network LRAE-Unet model includes the following steps:

[0056] S101: Get a training image dataset;

[0057] It is understandable that if Figure 2 As shown in FIG, step A1: input the green plant image data to be classified. For example, the images to be classified include different categories, and each category of images is taken in different natural scenes. The images of the same type of plant taken in different natural scenes are as follows: Figure 3 As shown, different types of plant leaves are Figure 4 shown.

[0058] S102: constructing a lightweight residual attention enhancement network LRAE-Unet model, wherein the lightweight residual attention enhancement network LRAE-Unet model includes a shallow feature extraction path and a deep feature extraction path;

[0059] Furthermore, in one embodiment of the present invention, the shallow feature extraction path includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, and a first downsampling module, a second downsampling module, a third downsampling module, and a fourth downsampling module, wherein the first convolution block, the second convolution block, the third convolution block, and the fourth convolution block each contain three consecutive convolution layers; the FMs scale output by each convolution module of the shallow feature extraction path is calculated as:

[0060] C=M*2,

[0061] Among them, C×C is the FMs scale of the convolutional layer output, and M×M is the FMs scale of the convolutional layer input.

[0062] Specifically, the first, second, third, and fourth convolutional blocks each contain three consecutive convolutional layers, with convolution kernel sizes of 1*1, 3*3, and 1*1, respectively. The convolution kernel sizes of the shallow feature extraction pathway are 1*1 and 3*3, and the size of the feature maps (FMs) output by the convolutional layer is equal to the size of the FMs input to the convolutional layer. When the convolution kernel size is 1*1, the size of the output FMs is doubled. The number of FMs output by different convolutional blocks is 128, 256, 512, and 1024, respectively. The FMs output by the convolutional layer are residually connected between layers, and each convolution layer is looped a specified number of times. The first and second downsampling modules are maximum pooling modules with a convolution kernel size of 2*2 and a stride size of 2. The third and fourth downsampling modules are average pooling modules with a convolution kernel size of 3*3, a stride size of 2, and a divisor size of 2.

[0063] Furthermore, in one embodiment of the present invention, the deep feature extraction path includes a first convolution block, a second convolution block, a third convolution block, and a fourth convolution block, and the first convolution block, the second convolution block, the third convolution block, and the fourth convolution block each include seven consecutive convolution layers; the FMs scale of each convolution block output of the deep feature extraction path is calculated as:

[0064] C=M / 2,

[0065] Among them, C×C is the FMs scale of the convolutional layer output, and M×M is the FMs scale of the convolutional layer input.

[0066] Specifically, the first convolution block, the second convolution block, the third convolution block and the fourth convolution block each contain seven consecutive convolution layers, the convolution kernel sizes are 3*3, 1*1, 3*3, 1*1, 3*3, 1*1, and 1*1, respectively, and the number of FMs output by the convolution layer of the deep feature extraction path is 512, 256, 128 and 64, respectively.

[0067] Furthermore, in one embodiment of the present invention, constructing a lightweight residual attention enhancement network LRAE-Unet model further includes:

[0068] In the shallow feature extraction path, batch normalization is applied to the weights of each convolutional layer of the shallow feature extraction path, wherein,

[0069]

[0070] Among them, y is a batch of FMs with a mean of 0 and a variance of 1, x represents the input, mean[x] represents the average value of the input, Var[x] represents the variance of the input, eps represents the stability coefficient to prevent the denominator from being 0, γ and β are adjustments to the pixels based on BN, the initial value of γ is 1, which is used to adjust the variance, and the initial value of β is 0, which is used to adjust the mean.

[0071] In the deep feature extraction pathway, BN processing is also used.

[0072] For example, the lightweight residual attention enhancement network LRAE-Unet model is as follows Figure 5 As shown. In the figure, (1) represents a green plant image, (2) represents a single convolution layer, (3), (4), (5) and (6) represent convolution blocks, all of which contain three convolutions. The convolution blocks (3) and (4) are paired with a maximum pooling layer, and the convolution blocks (5) and (6) are paired with an enhanced visual average pooling layer. (8), (10), (12) and (14) represent convolution blocks, (9), (11), (13) and (15) represent skip connections, i.e., the fusion of shallow information and deep information, and (16) is a classification layer. The lightweight residual attention enhancement network LRAE-Unet model includes two forms: a path for extracting shallow features and a path for extracting deep features. The shallow feature path is as follows: Figure 6 As shown, the lightweight residual attention enhancement network LRAE-Unet model extracts deep features as follows Figure 7 shown. Figure 8 This is a schematic diagram of the fusion of shallow information and deep information according to an embodiment of the present invention.

[0073] It is understandable that if Figure 2 As shown, step A2: Design a lightweight residual attention enhancement network LRAE-Unet. The lightweight residual attention enhancement network LRAE-Unet model consists of two pathways, including a shallow pathway and a deep pathway. The model is as follows Figure 5The input size of the model is M×M×3, where M×M represents the image block scale and “3” represents the three channels of the image block, namely R (red), G (green), and B (blue). (2), (3), (4), (5) and (6) constitute the shallow information extraction path, where (3), (4), (5) and (6) represent convolution blocks, each including three convolution layers. The residual blocks of (3) and (4) are paired with the maximum pooling layer, and the residual blocks of (5) and (6) are paired with the enhanced field of view average pooling layer. (7), (8), (10), (12) and (14) constitute the deep information extraction path, where (8), (10), (12) and (14) represent convolution blocks, each including seven convolution layers. (9), (11), (13) and (15) represent skip connections, i.e., the fusion of shallow information and deep information. (16) is the classification layer, which calculates the probability that the input image block belongs to 80 categories, and takes the category with the highest probability as the category of the image block.

[0074] It is understandable that, Figure 2 As shown, the shallow information extraction path in the embodiment of the present invention is: in (2), the input of the convolution layer is 160*160*3, and the output is 160*160*64; in (3), the input and output of the first and second convolution layers are 160*160*64, and the input of the third convolution layer is 160*160*64, and the output is 160*160*128; in (4), the input and output of the first and second convolution layers are 80*80*128, and the input of the third convolution layer is 80*80*128, and the output is 80*80*256; in (5) , the input and output of the first and second convolutional layers are 40*40*256, the input of the third convolutional layer is 40*40*256, and the output is 40*40*512; in (6), the input and output of the first and second convolutional layers are 20*20*512, the input of the third convolutional layer is 20*20*512, and the output is 20*20*1024; in (7), the output is 10*10*1024; the pooling windows of (3) and (4) are both 2*2, and the sliding step size is 2, and the pooling windows of (5) and (6) are both 3*3, and the sliding step size is 2. The shallow path has four convolution blocks, each of which contains three consecutive convolution layers with kernel sizes of 1*1, 3*3, and 1*1. The FMs output by the convolution layers with kernel sizes of 1*1 and 3*3 are equal to the FMs input to the convolution layers. When the kernel size is 1*1, the FMs output by the convolution layer are doubled. The FMs output by the convolution layers are connected to each other, and their form is as follows: Figure 9As shown in Figure 1. Conv1, Conv2, and Conv3 are convolutional layers, and FM1s, FM2s, and FM3s are FMs output by the convolutional layers. In the shallow path, the convolution block is followed by a pooling layer with a pooling window size of 2*2 and a step size of 2*2. Figure 10 As shown in Figure 2, (1), (2) and (3) are convolutional layers, (4) represents FMs connection, and (5) is the pooling layer.

[0075] Furthermore, if Figure 2 As shown, the deep information extraction pathway: (7), (8), (10), (12) and (14) constitute the deep information extraction pathway; in (7), (8), (10), (12) and (14), the input forms of the convolutional layers are 10*10*1024, 20*20*512, 40*40*256, 80*80*128, 160*160*64, respectively, and the output forms are 20*20*512, 40*40*256, 80*80*128, 160*160*64, respectively. The activation function of all convolutional layers is the LReLU function (Leaky Rectified Linear Unit), and the activation function of the classification layer is the softmax function. It should be noted that the LReLU function and the softmax function do not have corresponding Chinese names. The softmax function is the most commonly used multi-classification function, and its output is the probability of belonging to various classes. In the local pathway, the regularization terms applied to the weights of the convolutional layers are all In the deep path, the Batch Normalization method is used after the convolutional layers (7), (8), (10), (12) and (14).

[0076] The LReLU function form is as follows:

[0077]

[0078] Among them, y i Represents the input of the function, a i Indicates the slope.

[0079] The deep feature extraction path includes a first convolution block, a second convolution block, a third convolution block, and a fourth convolution block, each of which includes seven consecutive convolution layers, and the convolution kernel sizes are 3*3, 1*1, 3*3, 1*1, 3*3, 1*1, and 1*1, respectively. The number of FMs output by the convolution layer of the deep feature extraction path is 512, 256, 128, and 64, respectively, and the scale of the FMs output by each convolution block of the deep feature extraction path is calculated as:

[0080] C=M / 2,

[0081] Among them, C×C is the FMs scale of the convolutional layer output, and M×M is the FMs scale of the convolutional layer input.

[0082] It should be noted that the lightweight residual attention enhancement network (LRAE-Unet) classification model was trained on a single NVIDIA GeForce (3060Ti) GPU (Graphics Processing Unit). The model achieved remarkable results, eliminating the tedious steps of manually extracting image features and then classifying them, as in traditional image processing methods. Instead, it directly extracts features and classifies image patches. Furthermore, aside from selecting an appropriate crop size for model pre-training, no further image pre-processing is required.

[0083] S103: Performing model training using the lightweight residual attention enhancement network LRAE-Unet model and the training image dataset to generate a classification model;

[0084] Furthermore, in one embodiment of the present invention, the lightweight residual attention enhancement network LRAE-Unet model is trained using stochastic gradient descent, and the model weights are adjusted using a learning rate function, and the Pytorch deep learning framework is used, wherein,

[0085] The parameter setting form of the stochastic gradient descent method is as follows:

[0086] sgd=SGD(lr=lr_base, momentum=0.9, nesterov=True),

[0087] The learning rate function is as follows:

[0088]

[0089] Where SGD(·) is the stochastic gradient descent method in the Pytorch deep learning framework, momentum is the momentum parameter, nesterov = True indicates the use of nesterov momentum, lr is the learning rate for each round of training, lr_base indicates the learning rate at the beginning of training, epoch is the number of training rounds, epoches is the total number of training rounds, and lr_power is the exponential parameter.

[0090] Furthermore, in one embodiment of the present invention, the model training using the lightweight residual attention enhancement network LRAE-Unet model and the training image dataset further includes:

[0091] Use EarlyStopping to control the training process. When the accuracy of the validation set no longer improves or the number of training rounds reaches the maximum, the model training is terminated.

[0092] It is understandable that, Figure 2 As shown, step A3: model training, generating a lightweight residual attention enhancement network LRAE-Unet classification model. Among them, step A3 model training can include four steps in A31, A32, A33 and A34.

[0093] Step A31: First, the model parameters are initialized. Second, the weights in the convolutional layer and the fully connected layer are initialized using the Xavier normal distribution method, and the bias vector is initialized to zero.

[0094] Among them, the Xavier normal distribution initialization method is an effective neural network initialization method proposed by Glorot et al. in 2010.

[0095] Step A32: Adjust the model weights. When training the model, SGD (Stochastic Gradient Descent) is used to optimize the model using the Pytorch deep learning framework. The SGD parameter settings are shown in Formula 4.

[0096] sgd=SGD(lr=lr_base, momentum=0.9, nesterov=True),

[0097] Using a variable learning rate, the learning rate function is shown as follows:

[0098]

[0099] Here, SGD(·) is the SGD function in the PyTorch deep learning framework, momentum is the momentum parameter, nesterov = True indicates the use of Nesterov momentum (Newton momentum), lr is the learning rate for each training round, lr_base is the learning rate at the beginning of training, epoch is the number of training rounds, epoches is the set total number of training rounds, and lr_power is the set exponential parameter. Using deep learning methods for image classification can greatly simplify the classification process, reduce human intervention, and achieve high classification accuracy, making it a very promising approach.

[0100] Step A33: Set the termination condition for training.

[0101] The EarlyStopping method is used during model training. When the accuracy of the validation set no longer improves or when the number of training times reaches the set maximum value, the training process automatically terminates.

[0102] Optionally, EarlyStopping means that during model training, once the set termination conditions are met, training is automatically terminated regardless of whether the maximum number of training rounds has been reached.

[0103] Step A33 utilizes Early Stopping to control the training process. Training is terminated when the accuracy of the validation set stops improving or the maximum number of training rounds is reached. Early Stopping means that during model training, once the specified termination conditions are met, training automatically terminates, regardless of whether the maximum number of training rounds has been reached. The validation set during training accounts for 20% of the total training set.

[0104] It should be noted that the embodiment of the present invention can use a time-varying learning rate to train the model. The time-varying learning rate can not only speed up the convergence speed, but also effectively avoid overshoot or violent oscillation that may occur during the model training process. The grid search method is used to determine the relevant parameters of the model, such as the number of convolution kernels, activation function, Batch Normalization parameter values, etc., which effectively reduces the difficulty of selecting parameters when optimizing the model. The Early Stopping method is used to terminate the training in time when the model performance no longer improves during the training process.

[0105] Step A34: Save the trained model.

[0106] After training is terminated, save the trained model: including model weights (file type is .hdf), model structure information (type is .json), and model structure diagram (file type is .png).

[0107] S104: Acquire an image to be classified, input the image to be classified into the classification model, and obtain an image classification result.

[0108] It is understandable that, Figure 2 As shown, step A4: output the green plant species classification results.

[0109] The process of determining the category of each image to be classified is shown in Table 1. Table 1 is a process table for determining the category of a green plant image to be classified.

[0110] Table 1

[0111]

[0112] Specifically, the image classification method based on the lightweight residual attention enhancement network (LRAE-Unet) according to an embodiment of the present invention may include: inputting image data to be classified; training the lightweight residual attention enhancement network (LRAE-Unet) model, specifically including designing the LRAE-Unet model. The design of the LRAE-Unet model includes a shallow feature extraction pathway and a deep feature extraction pathway. The model training process includes initializing the model weights, adjusting the model weights, and setting a termination condition for training; and outputting the image classification result, specifically including inputting the image block to be classified into the trained LRAE-Unet model to correctly classify the image data to be tested.

[0113] According to the image classification method of the lightweight residual attention enhancement network LRAE-Unet proposed in an embodiment of the present invention, the image classification model of the lightweight residual attention enhancement network LRAE-Unet is obtained through convolutional neural network theory and deep learning network model training method, so that end-to-end classification can be achieved, and can be used to fully automatically classify images to achieve pixel-level classification purposes. It is not restricted by the category of the image to be classified, has strong applicability, good model performance, and is robust, making classification not only more convenient and effective, but also simple to operate and with better performance, thereby effectively improving the accuracy and convenience of image classification.

[0114] In order to implement the above embodiment, the present invention also proposes an image classification device based on a lightweight residual attention enhancement network LRAE-Unet model.

[0115] Figure 10 A structural diagram of an image classification device based on a lightweight residual attention enhancement network LRAE-Unet model provided in an embodiment of the present invention.

[0116] like Figure 10 As shown, the image classification device based on the lightweight residual attention enhancement network LRAE-Unet model includes: an acquisition module 100, a construction module 200, a training module 300, and a classification module 400, wherein,

[0117] An acquisition module is used to obtain a training image dataset;

[0118] A construction module for constructing a lightweight residual attention enhancement network LRAE-Unet model, wherein the lightweight residual attention enhancement network LRAE-Unet model includes a shallow feature extraction path and a deep feature extraction path;

[0119] A training module, configured to perform model training using the lightweight residual attention enhancement network LRAE-Unet model and the training image dataset to generate a classification model;

[0120] The classification module is used to obtain the image to be classified, input the image to be classified into the classification model, and obtain the image classification result.

[0121] To achieve the above-mentioned purpose, the third aspect of the present invention proposes a computer device, characterized in that it includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, it implements the image classification method based on the lightweight residual attention enhancement network LRAE-Unet model as described above.

[0122] To achieve the above-mentioned purpose, the fourth aspect of the present invention proposes a computer-readable storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, it implements the image classification method based on the lightweight residual attention enhancement network LRAE-Unet model as described above.

[0123] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0124] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0125] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limiting the present invention. A person skilled in the art may change, modify, replace, and modify the above embodiments within the scope of the present invention.

Claims

1. An image classification method based on the lightweight residual attention enhancement network LRAE-Unet model, characterized by: The following steps are involved: Get a training image dataset; Constructing a lightweight residual attention enhancement network (LRAE-Unet) model, wherein the lightweight residual attention enhancement network (LRAE-Unet) model includes a shallow feature extraction path, a deep feature extraction path, and a classification layer; Performing model training using the lightweight residual attention enhancement network (LRAE-Unet) model and the training image dataset to generate a classification model; Obtaining an image to be classified, inputting the image to be classified into the classification model, and obtaining an image classification result; The input size of the lightweight residual attention enhancement network LRAE-Unet model is M×M×3, where M×M represents the image block scale and 3 represents the three channels of the image block, namely R red, G green, and B blue. The shallow feature extraction pathway includes a single convolutional layer, a first convolutional block, a second convolutional block, a third convolutional block, and a fourth convolutional block. All convolutional blocks in the shallow feature extraction pathway contain three convolutions. The first and second convolutional blocks are paired with a maximum pooling layer, and the third and fourth convolutional blocks are paired with an enhanced field of view average pooling layer. The FMs scale of each convolutional module output in the shallow feature extraction pathway is calculated as: C=M*2, Among them, C×C is the FMs scale of the convolution layer output, M×M is the FMs scale of the convolution layer input, The FMs output of the convolutional layer are connected to each other; The deep feature extraction path includes a first convolution block, a second convolution block, a third convolution block, and a fourth convolution block, each of which includes seven consecutive convolution layers. The FMs scale of each convolution block output of the deep feature extraction path is calculated as: C=M / 2, Where C×C is the FMs scale output by the convolutional layer, and M×M is the FMs scale input to the convolutional layer. The lightweight residual attention enhancement network LRAE-Unet model is constructed, further comprising: In the shallow feature extraction path, batch normalization is applied to the weights of each convolutional layer of the shallow feature extraction path, wherein, Among them, y is a batch of FMs with a mean of 0 and a variance of 1, x represents the input, mean[x] represents the average value of the input, Var[x] represents the variance of the input, eps represents the stability coefficient to prevent the denominator from being 0, γ and β are adjustments to the pixels based on BN, the initial value of γ is 1, which is used to adjust the variance, and the initial value of β is 0, which is used to adjust the mean.

2. The method according to claim 1, characterized in that The method of performing model training using the lightweight residual attention enhancement network LRAE-Unet model and the training image dataset includes: The lightweight residual attention enhancement network LRAE-Unet model is trained using the stochastic gradient descent method, and the model weights are adjusted using the learning rate function, and the Pytorch deep learning framework is used, wherein, The parameter setting form of the stochastic gradient descent method is as follows: sgd=SGD(lr=lr_base, momentum=0.9, nesterov=True), The learning rate function is as follows: Where SGD(·) is the stochastic gradient descent method in the Pytorch deep learning framework, momentum is the momentum parameter, nesterov = True indicates the use of nesterov momentum, lr is the learning rate for each round of training, lr_base indicates the learning rate at the beginning of training, epoch is the number of training rounds, epoches is the total number of training rounds, and lr_power is the exponential parameter.

3. The method according to claim 1, characterized in that The method of performing model training using the lightweight residual attention enhancement network LRAE-Unet model and the training image dataset further includes: Use EarlyStopping to control the training process. When the accuracy of the validation set no longer improves or the number of training rounds reaches the maximum, the model training is terminated.

4. An image classification device based on a lightweight residual attention enhancement network LRAE-Unet model, characterized in that: Includes the following modules: An acquisition module is used to obtain a training image dataset; A construction module for constructing a lightweight residual attention enhancement network LRAE-Unet model, wherein the lightweight residual attention enhancement network LRAE-Unet model includes a shallow feature extraction path and a deep feature extraction path; A training module, configured to perform model training using the lightweight residual attention enhancement network LRAE-Unet model and the training image dataset to generate a classification model; A classification module is used to obtain an image to be classified, input the image to be classified into the classification model, and obtain an image classification result; Among them, the input size of the lightweight residual attention enhancement network LRAE-Unet model is M×M×3, M×M represents the image block scale, and 3 represents the three channels of the image block, namely R red, G green, and B blue; The shallow feature extraction pathway includes a single convolutional layer, a first convolutional block, a second convolutional block, a third convolutional block, and a fourth convolutional block. All convolutional blocks in the shallow feature extraction pathway contain three convolutions. The first and second convolutional blocks are paired with a maximum pooling layer, and the third and fourth convolutional blocks are paired with an enhanced field of view average pooling layer. The FMs scale of each convolutional module output in the shallow feature extraction pathway is calculated as: C=M*2, Among them, C×C is the FMs scale of the convolution layer output, M×M is the FMs scale of the convolution layer input, The FMs output of the convolutional layer are connected to each other; The deep feature extraction path includes a first convolution block, a second convolution block, a third convolution block, and a fourth convolution block, each of which includes seven consecutive convolution layers. The FMs scale of each convolution block output of the deep feature extraction path is calculated as: C=M / 2, Where C×C is the FMs scale output by the convolutional layer, and M×M is the FMs scale input to the convolutional layer. The lightweight residual attention enhancement network LRAE-Unet model is constructed, further comprising: In the shallow feature extraction path, batch normalization is applied to the weights of each convolutional layer of the shallow feature extraction path, wherein, Among them, y is a batch of FMs with a mean of 0 and a variance of 1, x represents the input, mean[x] represents the average value of the input, Var[x] represents the variance of the input, eps represents the stability coefficient to prevent the denominator from being 0, γ and β are adjustments to the pixels based on BN, the initial value of γ is 1, which is used to adjust the variance, and the initial value of β is 0, which is used to adjust the mean.

5. A computer device, characterized in that: The present invention comprises a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the image classification method based on the lightweight residual attention enhancement network LRAE-Unet model as described in any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image classification method based on the lightweight residual attention enhancement network LRAE-Unet model as described in any one of claims 1 to 3 is implemented.

Images