Disease image recognition method based on CNN and MLP
By introducing parallel branch structure and feature pyramid fusion in CNN and MLP models, the problem of insufficient disease recognition accuracy in field environments is solved, higher recognition accuracy and parameter efficiency are achieved, and attention to prospective lesions is enhanced .
Patent Information
- Application Number
- CN202411355228.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2025-01-14
AI Technical Summary
In the identification of crop diseases and pests, existing CNN models are difficult to effectively identify complex backgrounds in the field growth environment, resulting in insufficient recognition accuracy and generalization ability.
Using a disease image recognition model based on parallel branch structures of CNN and MLP, local and local and Global features are fused in the pyramid structure to ultimately realize image classification.
It significantly improves the recognition ability of the model, reduces interference from field background, improves the recognition accuracy and parameter efficiency, and can pay more attention to the foreground lesions area, with fewer parameters and higher accuracy.
Smart Images
Figure CN119314045A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of disease image recognition, and in particular to a disease image recognition method based on CNN and MLP. Background Art
[0002] Convolutional neural network is one of the important tools for processing large amounts of image data. Its structural system has been widely studied and applied to the identification of crop pests and diseases. Compared with traditional machine learning methods, this method does not require complex preprocessing of data, can automatically extract features from raw data, and has higher accuracy and efficiency.
[0003] Although deep learning has achieved certain results in the detection of crop diseases and insect pests, most of the experimental data are taken in a controllable laboratory environment, with a single image background, which cannot effectively simulate the real growth environment of crops. As a result, when the model trained on laboratory background data is transferred to real field image data, the recognition effect is greatly reduced. For example, the existing CNN model has shown good results in the detection of crop diseases and insect pests, but there is still much room for improvement in the recognition effect in the field growth environment.
[0004] Therefore, it is necessary to further study the impact of the complex growth environment background in the image on the recognition accuracy in order to improve the recognition accuracy and generalization ability of the model. Summary of the invention
[0005] The purpose of the present invention is to solve the defect that it is difficult to accurately identify disease images in the prior art, and to provide a disease image recognition method based on CNN and MLP to solve the above problem.
[0006] In order to achieve the above object, the technical solution of the present invention is as follows:
[0007] The disease image recognition method based on CNN and MLP includes the following steps:
[0008] Acquisition and preprocessing of disease images: Acquisition of crop disease images and preprocessing;
[0009] Construction of disease image recognition model: Construction of disease image recognition model based on parallel CNN branch and MLP branch;
[0010] Training of disease image recognition model: input the preprocessed crop disease images into the disease image recognition model for training;
[0011] Acquisition of disease images to be identified: Acquisition of disease images to be identified and preprocessing;
[0012] Obtaining the disease image recognition result: input the preprocessed disease image to be identified into the trained disease image recognition model to obtain the disease image recognition result.
[0013] The construction of the disease image recognition model includes the following steps:
[0014] The first part of the disease image recognition model is set as a convolution pooling operation, the input is the preprocessed disease image, and the output is the initial feature map;
[0015] The second part of the disease image recognition model is set to be a parallel CNN branch and an MLP branch. The input is the initial feature map after dimension transformation. The CNN branch outputs local features, and the MLP branch outputs global features. Both the CNN branch and the MLP branch adopt a feature pyramid structure, and the local features and the global features are fused at the end of each layer of the pyramid to output the final features.
[0016] The third part of the disease image recognition model is set as a fully connected layer, which includes three fully connected operations and a Softmax. The fully connected layer maps the final features to the corresponding categories, thereby realizing the classification of the image;
[0017] Setting CNN branches: Setting CNN branches to consist of cascaded residual modules;
[0018] The residual module consists of four steps: first, a down-projection convolution operation with a convolution kernel size of 1×1 is used to reduce the input dimension; second, a spatial convolution kernel with a convolution kernel size of 3×3 is used for feature extraction; then, an up-projection convolution with a convolution kernel size of 1×1 is used to restore the dimension; finally, a jump connection is used between the input and output for identity mapping;
[0019] Setting the MLP branch: The MLP branch is composed of a cascade of MLP-Mixer modules. The MLP-Mixer module takes a sequence of image patches as input and projects the image patches onto the corresponding dimensions through a linear mapping.
[0020] The MLP-Mixer module consists of two sub-modules with the same structure: Token-Mixing MLP and Channel-Mixing MLP. Both Token-Mixing MLP and Channel-Mixing MLP are composed of two fully connected layers and an activation function GELU.
[0021] The Token-Mixing MLP module works between image blocks, extracts and encodes features of the input image blocks, and has global spatial characteristics. The Channel-Mixing MLP module works on channels and models the relationship between channels. Both Token-Mixing MLP and Channel-Mixing MLP are normalized using LayerNorm, and both add jump connections between input and output.
[0022] The training of the disease image recognition model includes the following steps:
[0023] The crop diseased leaf image with a resolution of 224×224×3 is input into the first part of the disease image recognition model for convolution and pooling operations. The input image is first subjected to initial feature extraction through a convolution with a kernel size of 7×7×64 and a stride of 2 and a maximum pooling operation of 3×3 and a stride of 2, obtaining an initial feature map with a resolution of 56×56×64.
[0024] The initial feature maps are input into the second part of the disease image recognition model, and enter the CNN branch and the MLP branch in parallel.
[0025] The CNN branch first downsamples the initial feature map to reduce its size, then expands it into a two-dimensional image block sequence, and fuses the image block sequence with the image block sequence obtained by the MLP branch.
[0026] The initial feature map obtained by the MLP branch is divided into 4×4 non-overlapping small image blocks using a convolution operation with a convolution kernel size of 4×4×256 and a stride of 4. The image block sequence is upsampled by interpolation to increase the image size and reshaped into a three-dimensional image. Then, the reshaped image is provided to the CNN branch for fusion;
[0027] By continuously passing the global information of the MLP branch to the convolutional feature map, the global perception and long-distance modeling capabilities of the CNN branch are enhanced. At the same time, the local features of the CNN branch are fed back to the MLP branch to enrich the local details of the MLP branch, and finally the final features after fusion are obtained.
[0028] The fully connected layer maps the fused features to different categories to complete the classification task.
[0029] The setting of CNN branches includes the following steps:
[0030] The first layer of the CNN branch is set to consist of three cascaded residual modules, where the last residual module compresses the image size and expands the dimension. The input size of this layer is 56×56×64, and the output size is 56×56×194. The convolution kernel sizes of the first three residual structures are 1×1×16, 3×3×16, and 1×1×64, respectively, and the last convolution kernel size of the last residual module is 1×1×196.
[0031] The second layer of the CNN branch is set to consist of four cascaded residual modules. The dimension is expanded in the last residual module. The input size of this layer is 56×56×196 and the output size is 28×28×196.
[0032] The third layer of the CNN branch is set to consist of three cascaded residual modules. The dimension is expanded in the last residual module. The input size of this layer is 28×28×196 and the output size is 14×14×196.
[0033] The fourth layer of the CNN branch is set to extract the final feature map through a residual module. The input size of this layer is 14×14×196 and the output size is 7×7×196.
[0034] Beneficial Effects
[0035] Compared with the prior art, the disease image recognition method based on CNN and MLP of the present invention uses a deep learning model with parallel convolutional neural network and multi-layer perceptron to realize the recognition of crop diseases and pests in field background, shows good effect in reducing the interference of field background on the model, and can significantly improve the recognition ability of the model.
[0036] In the present invention, local features are extracted by a convolutional neural network branch, and global information and channel information are supplemented by a multi-layer perceptron branch; the two branches are separated at the beginning of the pyramid and merged at the end. This structure fully inherits the structural advantages of the two branches, while retaining the representation capabilities of local and global features and the relationship modeling between channels to the greatest extent. Experimental results show that compared with other models, the model described in the present invention has fewer parameters, higher accuracy, and can maintain a higher degree of attention to the foreground lesion area. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 is a method sequence diagram of the present invention;
[0038] Figure 2 This is a structural diagram of the disease image recognition model involved in the present invention;
[0039] Figure 3 This is a structural diagram of the residual module involved in the present invention;
[0040] Figure 4 This is a diagram of the MLP branch structure involved in the present invention;
[0041] Figure 5a It is a curve diagram of the accuracy of the disease image recognition model and MLP described in the present invention;
[0042] Figure 5b This is the accuracy curve of the existing CNN model. DETAILED DESCRIPTION
[0043] In order to have a further understanding and recognition of the structural features and the effects achieved by the present invention, a preferred embodiment and accompanying drawings are used for detailed description as follows:
[0044] like Figure 1 As shown, the disease image recognition method based on CNN and MLP of the present invention comprises the following steps:
[0045] The first step is to obtain and preprocess disease images: obtain crop disease images and preprocess them.
[0046] The second step is to build a disease image recognition model: Figure 2 As shown in the figure, a disease image recognition model is constructed based on parallel CNN branches and MLP branches.
[0047] (1) The first part of the disease image recognition model is set as a convolution pooling operation, the input is the preprocessed disease image, and the output is the initial feature map.
[0048] (2) The second part of the disease image recognition model is set to be a parallel CNN branch and an MLP branch. The input is the initial feature map after dimensional transformation. The CNN branch outputs local features, and the MLP branch outputs global features. Both the CNN branch and the MLP branch adopt a feature pyramid structure, and the local features and global features are fused at the end of each layer of the pyramid to output the final features to improve the feature modeling effect of the model.
[0049] The CNN branch consists of cascaded residual modules. This structure has a strong inductive bias capability and can effectively extract local feature information and implicitly encode position information. However, for long-distance relationships, which are extremely important in computational vision tasks, the global modeling ability of this structure is weak. Therefore, more global information needs to be supplemented to improve the performance of the model. In contrast, the MLP branch consists of cascaded MLP-Mixer modules. By applying MLP-Mixer modules in spatial positions and feature channels, global feature information can be effectively obtained, thereby enhancing the expression of semantic information. By fusing the cross-information of the residual module and the MLP-Mixer module, global modeling information can be continuously supplemented to reduce the weight of the background while extracting local features, so that the model can focus more on high semantic feature areas.
[0050] (3) The third part of the disease image recognition model is set as a fully connected layer. The fully connected layer includes three fully connected operations and one Softmax. The fully connected layer maps the final features to the corresponding categories, thereby realizing the classification of the image.
[0051] (4) Setting CNN branches: The CNN branches are set to consist of cascaded residual modules.
[0052] Shallow neural networks can only extract limited image features. In order to improve the feature extraction effect, a common improvement method is to increase the network depth and increase the network width. However, although this improvement method can improve the feature extraction effect to a certain extent, a deeper model may cause gradient disappearance or gradient explosion, resulting in a decrease in recognition effect.
[0053] In the convolutional neural network branch, each stage consists of Figure 3 The structure of the residual module consists of four steps. First, a down-projection convolution operation with a convolution kernel size of 1×1 is used to reduce the input dimension, thereby reducing the amount of calculation and improving the efficiency of the model. Secondly, a spatial convolution kernel with a convolution kernel size of 3×3 is used for feature extraction to capture spatial information. Then, an up-projection convolution with a convolution kernel size of 1×1 is used to restore the dimension. Finally, a jump connection is used between the input and output for identity mapping to avoid gradient loss.
[0054] Setting up a CNN branch involves the following steps:
[0055] A1) The first layer of the CNN branch is set to consist of three cascaded residual modules, where the last residual module compresses the image size and expands the dimension. The input size of this layer is 56×56×64, and the output size is 56×56×194. The convolution kernel sizes of the first three residual structures are 1×1×16, 3×3×16, and 1×1×64 respectively, and the last convolution kernel size of the last residual module is 1×1×196;
[0056] A2) The second layer of the CNN branch is set to consist of four cascaded residual modules. The dimension is expanded in the last residual module. The input size of this layer is 56×56×196 and the output size is 28×28×196.
[0057] A3) The third layer of the CNN branch is set to consist of three cascaded residual modules. The dimension is expanded in the last residual module. The input size of this layer is 28×28×196 and the output size is 14×14×196.
[0058] A4) Set the fourth layer of the CNN branch to extract the final feature map through a residual module. The input size of this layer is 14×14×196 and the output size is 7×7×196.
[0059] (5) Setting the MLP branch: The MLP branch is set to consist of a cascade of MLP-Mixer modules. The MLP-Mixer module takes the image block sequence as input and projects the image blocks to the corresponding dimension through linear mapping.
[0060] like Figure 4 As shown in the figure, the MLP-Mixer module consists of two sub-modules with the same structure: Token-Mixing MLP and Channel-Mixing MLP. It can effectively model the long-distance image block relationship, thereby reducing the model's attention to the background and making the model pay more attention to the foreground area. Both Token-Mixing MLP and Channel-Mixing MLP are composed of two fully connected layers and an activation function GELU, thereby achieving more efficient information transmission;
[0061] The Token-Mixing MLP module works between image blocks, extracts and encodes features of the input image blocks, and has global spatial characteristics. The Channel-Mixing MLP module works on channels and models the relationship between channels. Both Token-Mixing MLP and Channel-Mixing MLP are normalized using LayerNorm, and both add jump connections between input and output.
[0062] The third step is to train the disease image recognition model: input the preprocessed crop disease images into the disease image recognition model for training;
[0063] (1) The diseased leaf image of a crop with a resolution of 224×224×3 is input into the first part of the disease image recognition model for convolution and pooling operations. The input image is first subjected to initial feature extraction through a convolution with a kernel size of 7×7×64 and a stride of 2 and a maximum pooling operation of 3×3 and a stride of 2, obtaining an initial feature map with a resolution of 56×56×64.
[0064] (2) The initial feature maps are input into the second part of the disease image recognition model and enter the CNN branch and the MLP branch in parallel.
[0065] The CNN branch first downsamples the initial feature map to reduce its size, then expands it into a two-dimensional image block sequence, and fuses the image block sequence with the image block sequence obtained by the MLP branch.
[0066] The initial feature map obtained by the MLP branch is divided into 4×4 non-overlapping small image blocks using a convolution operation with a convolution kernel size of 4×4×256 and a stride of 4. The image block sequence is upsampled by interpolation to increase the image size and reshaped into a three-dimensional image. Then, the reshaped image is provided to the CNN branch for fusion;
[0067] By continuously passing the global information of the MLP branch to the convolutional feature map, the global perception and long-distance modeling capabilities of the CNN branch are enhanced. At the same time, the local features of the CNN branch are fed back to the MLP branch to enrich the local details of the MLP branch, and finally the final features after fusion are obtained.
[0068] (3) The fully connected layer maps the fused features to different categories to complete the classification task.
[0069] The fourth step is to obtain the image of the disease to be identified: obtain the image of the disease to be identified and perform preprocessing.
[0070] Step 5: Obtaining the disease image recognition result: Input the preprocessed disease image to be identified into the trained disease image recognition model to obtain the disease image recognition result.
[0071] The current mainstream CNN model, MLP model and the model proposed in this invention are compared on various crop disease datasets. The experimental results are shown in Table 1, where the best results are marked in bold.
[0072] Table 1 Performance comparison of different models on various crop disease classification datasets
[0073]
[0074] According to the data in Table 1, it can be seen that since the model proposed in the present invention adopts a parallel structure, the number of parameters can be effectively reduced while maintaining its relatively reasonable depth and width. Moreover, the model has better accuracy than the current mainstream CNN model and the original MLP model when the number of parameters is minimal. For example, the ResNet50 model achieved an accuracy rate of 89.90% with a parameter number of 25.0M. The model proposed in the present invention has a parameter number of only 9.5M, which is much lower than the parameter number of the ResNet50 model, and the accuracy rate is 2.6% higher, reaching 92.5%. This result of the model proposed in the present invention shows that the dual-branch model proposed in the present invention can inherit the structural advantages of CNN and MLP and enhance the representation ability of the network.
[0075] In addition, as shown in Figures 5(a) and 5(b), the accuracy curves of each model during the training process are shown. As can be seen from Figures 5(a) and 5(b), the dual-branch model proposed in the present invention does not perform as well as ResNet50 and the original MLP model in the early stages of training, but as the training progresses, its accuracy gradually exceeds that of other models. This is because the inductive bias of the CNN branch is provided to the MLP-Mixer branch as a feature result, while for the MLP-Mixer itself, more iterations are required to learn the feature information. The final experimental results also show that the dual-branch model has better feature learning and generalization capabilities.
[0076] In order to verify the effectiveness of the dual-branch structure, an ablation experiment was conducted on the cases containing only the CNN branch, only the MLP branch, and the coexistence of the two branches. The experimental results are shown in Table 2.
[0077] Table 2. Ablation experiment comparison table
[0078]
[0079] According to the results in Table 2, the performance of the model using the CNN branch and the MLP branch alone is not good. In contrast, the dual-branch parallel model performs better. This is because the single-branch model cannot aggregate global feature information and local feature information at the same time, while the dual-branch parallel model can fuse local features and global representations at different resolutions in an interactive manner. This approach can maintain attention to local details while not neglecting the relationship modeling between local information.
[0080] In the present invention, local features are extracted by a convolutional neural network branch, and global information and channel information are supplemented by a multi-layer perceptron branch. The two branches are separated at the beginning of the pyramid and merged at the end. This structure fully inherits the structural advantages of the two branches, while retaining the representation capabilities of local features and global features, as well as the relationship modeling between channels to the greatest extent. Experimental results show that compared with other models, this model has fewer parameters, higher accuracy, and can maintain a higher degree of attention to the foreground lesion area.
[0081] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.
Claims
1. A disease image recognition method based on CNN and MLP, characterized in that: The following steps are involved: 11) Acquisition and preprocessing of disease images: Acquisition of crop disease images and preprocessing; 12) Construction of disease image recognition model: The disease image recognition model is constructed based on parallel CNN branches and MLP branches; 13) Training of disease image recognition model: inputting the preprocessed crop disease images into the disease image recognition model for training; 14) Acquisition of the disease image to be identified: Acquisition of the disease image to be identified and preprocessing; 15) Obtaining the disease image recognition result: Input the preprocessed disease image to be identified into the trained disease image recognition model to obtain the disease image recognition result.
2. The disease image recognition method based on CNN and MLP according to claim 1, characterized in that: The construction of the disease image recognition model includes the following steps: 21) The first part of the disease image recognition model is set as a convolution pooling operation, the input is the preprocessed disease image, and the output is the initial feature map; 22) The second part of the disease image recognition model is set to be a parallel CNN branch and an MLP branch, the input is the initial feature map after dimension transformation, the CNN branch outputs local features, and the MLP branch outputs global features. Both the CNN branch and the MLP branch adopt a feature pyramid structure, and the local features and the global features are fused at the end of each layer of the pyramid to output the final features; 23) The third part of the disease image recognition model is set as a fully connected layer, which includes three fully connected operations and one Softmax. The fully connected layer maps the final features to the corresponding categories to achieve image classification; 24) Setting CNN branches: Setting CNN branches to consist of cascaded residual modules; The residual module consists of four steps: first, a down-projection convolution operation with a convolution kernel size of 1×1 is used to reduce the input dimension; second, a spatial convolution kernel with a convolution kernel size of 3×3 is used for feature extraction; then, an up-projection convolution with a convolution kernel size of 1×1 is used to restore the dimension; finally, a jump connection is used between the input and output for identity mapping; 25) Set the MLP branch: Set the MLP branch to consist of a cascade of MLP-Mixer modules. The MLP-Mixer module takes a sequence of image blocks as input and projects the image blocks to the corresponding dimensions through linear mapping. The MLP-Mixer module consists of two sub-modules with the same structure: Token-Mixing MLP and Channel-Mixing MLP. Both Token-Mixing MLP and Channel-Mixing MLP are composed of two fully connected layers and an activation function GELU. The Token-Mixing MLP module works between image blocks, extracts and encodes features of the input image blocks, and has global spatial characteristics. The Channel-Mixing MLP module works on channels and models the relationship between channels. Both Token-Mixing MLP and Channel-Mixing MLP are normalized using LayerNorm, and both add jump connections between input and output.
3. The disease image recognition method based on CNN and MLP according to claim 1, characterized in that: The training of the disease image recognition model includes the following steps: 31) Input the crop diseased leaf image with a resolution of 224×224×3 into the first part of the disease image recognition model for convolution and pooling operation. The input image is first subjected to initial feature extraction through a convolution with a kernel size of 7×7×64 and a stride of 2 and a maximum pooling operation of 3×3 and a stride of 2, obtaining an initial feature map with a resolution of 56×56×64. 32) The initial feature maps are input into the second part of the disease image recognition model, and enter the CNN branch and the MLP branch in parallel. The CNN branch first downsamples the initial feature map to reduce its size, then expands it into a two-dimensional image block sequence, and fuses the image block sequence with the image block sequence obtained by the MLP branch. The initial feature map obtained by the MLP branch is divided into 4×4 non-overlapping small image blocks using a convolution operation with a convolution kernel size of 4×4×256 and a stride of 4. The image block sequence is upsampled by interpolation to increase the image size and reshaped into a three-dimensional image. Then, the reshaped image is provided to the CNN branch for fusion; By continuously passing the global information of the MLP branch to the convolutional feature map, the global perception and long-distance modeling capabilities of the CNN branch are enhanced. The local features of the CNN branch are fed back to the MLP branch to enrich the local details of the MLP branch, and finally the final features after fusion are obtained. 33) The fully connected layer maps the fused features to different categories to complete the classification task.
4. The disease image recognition method based on CNN and MLP according to claim 2, characterized in that: The setting of CNN branches includes the following steps: 41) The first layer of the CNN branch is set to consist of three cascaded residual modules, where the last residual module compresses the image size and expands the dimension. The input size of this layer is 56×56×64, and the output size is 56×56×194. The convolution kernel sizes of the first three residual structures are 1×1×16, 3×3×16, and 1×1×64, respectively, and the last convolution kernel size of the last residual module is 1×1×196; 42) The second layer of the CNN branch is set to consist of four cascaded residual modules. The dimension is expanded in the last residual module. The input size of this layer is 56×56×196 and the output size is 28×28×196. 43) The third layer of the CNN branch is set to consist of three cascaded residual modules. The dimension is expanded in the last residual module. The input size of this layer is 28×28×196 and the output size is 14×14×196. 44) Set the fourth layer of the CNN branch to extract the final feature map through a residual module. The input size of this layer is 14×14×196 and the output size is 7×7×196.