Field rice disease recognition system based on multi-model fusion algorithm and design method
By adopting multi-model fusion algorithm and data enhancement technology in the field rice disease identification system, the problems of low identification efficiency, lack of data sets and difficult to balance model stability in the existing technology are solved, efficient and accurate field rice disease identification is achieved, and the difficulty of promoting equipment is reduced.
Patent Information
- Application Number
- CN202411701329.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-03-04
AI Technical Summary
The prior art has problems in the identification of field rice diseases with low identification efficiency, high labor costs, lack of data sets, difficult to balance model stability and generalization capabilities, and lack of dedicated identification equipment.
A field rice disease recognition system based on multi-model fusion algorithm is adopted, and images are collected through mobile phone cameras, preliminary processing and data augmentation are performed, multiple convolutional neural network models are trained, and the final recognition results are calculated through multi-model fusion algorithm, and the field rice leaf disease recognition system is formed.
It improves the recognition efficiency and effect of rice diseases in the field, realizes real-time automatic identification, ensures the stability and generalization capabilities of the model, and reduces the difficulty of promoting equipment and the threshold for use.
Smart Images
Figure CN119559440A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of smart agricultural disease identification, and in particular to a field rice disease identification system and a design method based on a multi-model fusion algorithm. Background Art
[0002] Rice cultivation has long occupied an important position in agricultural development. As an important research in the field of plant disease resistance breeding, efficient disease identification is a major challenge faced by breeders. Rice diseases are the main factor affecting rice yield and quality, and their prevention and control effects depend on the accuracy and efficiency of disease identification.
[0003] Traditional identification methods rely on manual diagnosis by agronomists or comparison of disease maps. However, due to the complexity and similarity of the junction of rice lesions, manual identification requires rich experience and agricultural knowledge. In addition, this method has extremely high requirements for breeders and requires a lot of human resources and time costs. Therefore, smart planting that combines computer vision, image processing, machine learning and deep learning algorithms has attracted much attention. Early machine learning algorithms brought automatic identification into people's vision, but they relied too much on manual feature extraction. The automatic feature extraction and reasoning capabilities of deep learning broke this deadlock. Compared with the recognition mode based on artificial vision and machine learning, it automatically identifies rice diseases by learning the phenotypic characteristics of rice diseases, helping farmers to detect, diagnose and treat diseases in time at the early stage of the disease, and seize the best prevention and control time.
[0004] At present, in actual scenarios, the method adopted more often is still the method of manual on-site analysis to identify diseases. Manual on-site sampling may also be used, and then combined with image processing technology, machine learning and deep learning algorithms for further analysis; in addition, in some scenarios, drones and remote sensing technology will be used to collect on-site data. However, some existing methods still have the following four problems: 1) Disease characteristics are complex and difficult to distinguish, manual identification is subjective, identification efficiency is low and labor costs are high; 2) There are few public rice data sets collected from field backgrounds, there are many types of diseases and the disease period spans a long period, the field environment is complex and changeable, and it is difficult to build a complete field rice disease data set; 3) The stability and generalization ability of deep neural network models are difficult to balance on small data sets, and different models have different capabilities for extracting different features, making it difficult to introduce them into the identification of rice diseases in real scenarios; 4) There is a lack of dedicated rice disease identification equipment used in the field. Summary of the invention
[0005] The purpose of the present invention is to overcome the defects of the prior art, provide a field rice disease recognition system and design method based on a multi-model fusion algorithm, and build a field rice disease recognition device based on a multi-model result fusion algorithm to improve the recognition efficiency and effect, and realize real-time automatic recognition of field rice diseases.
[0006] The purpose of the present invention is achieved in one aspect as follows: a design method for a field rice disease identification system based on a multi-model fusion algorithm comprises the following steps:
[0007] S1) using a mobile phone camera to take pictures to obtain rice leaf disease images with complex backgrounds in the field, and preliminarily processing the images;
[0008] S2) performing data enhancement on the field rice leaf disease images after preliminary processing, expanding the original data, and constructing a field rice leaf disease dataset;
[0009] S3) training multiple convolutional neural network models based on the field rice leaf disease dataset, verifying and optimizing the performance of each model, and confirming the optimal parameters;
[0010] S4) calculating a weight parameter applicable to the whole world according to the cross entropy loss value of each model in the validation set, assigning corresponding weights to each model, calculating the final recognition result by using a multi-model fusion algorithm, and establishing a universal recognition model for rice leaf diseases in the field;
[0011] S5) The trained general recognition model is deployed to the Jetson Orin Nano development board, and the camera, display screen and power supply are combined to form a field rice leaf disease recognition system.
[0012] As a further limitation of the present invention, the preliminary processing of the image in step S1) is specifically as follows: based on the collected field rice leaf disease images, cleaning the rice disease images that are blurred, jittered, distorted, have unknown diseases or missing features; converting the storage format of the screened rice leaf disease images into a uniform JPG format; using Photoshop to perform multiple local cropping on each rice leaf disease image to expand the number of disease images, and according to the disease atlas, the cropped rice leaf disease images are divided into five categories: white leaf blight, brown spot disease, rice blast, dongru disease and health, and each category is saved in a folder.
[0013] As a further limitation of the present invention, in step S2), data enhancement of the field rice leaf disease images after preliminary processing specifically includes: data enhancement of the original white leaf blight, brown spot, rice blast, donggro disease and healthy category images stored in each folder by flipping, rotating, color enhancement, brightness enhancement, contrast enhancement and random color change operations, so that the data volume is expanded to eight times the original, and a five-category field rice leaf disease data set including four diseases and health is constructed.
[0014] As a further limitation of the present invention, the step S3) of training multiple convolutional neural network models based on the field rice leaf disease dataset is specifically: selecting four convolutional neural network models with less homogeneity, less performance difference and lightweight structure as basic models; using Train.py to adopt the transfer learning method to train the model.
[0015] As a further limitation of the present invention, the verification and optimization of the performance of each model in step S3) is specifically as follows: using val.py to verify the performance of each basic model and adjust the parameters, repeatedly verifying and adjusting the comparison to obtain the best parameter group; using label smoothing technology and cosine annealing to optimize the learning rate; the number of training iterations is set to 110, the initial learning rate is 0.0005, the batch size is set to 16, Adam is selected as the optimizer, and the optimal basic model is obtained through training.
[0016] As a further limitation of the present invention, the step 4) specifically includes: calculating the average cross entropy loss value of each basic model through the validation set, using the exponential function and the power function to amplify the difference between the cross entropy loss values and then performing normalization calculation to obtain a weight parameter applicable to the global situation; using the basic model to identify the target object respectively, assigning the weight parameters to the results of the corresponding basic models respectively, and calculating to obtain the final result.
[0017] As a further limitation of the present invention, the step 5) specifically includes: running the predict program on the terminal, selecting the model to be used by inputting the serial number, and after loading the model, the device turns on the camera to capture the picture, and the picture is transmitted to the Jetson Orin Nano development board through the camera sensor; after receiving the picture, the Jetson Orin Nano development board calls the classify_image function to load the selected model, resizes, normalizes and fills the picture before preprocessing and inputs the image to the model, the model extracts the features of the input image through the convolution layer to generate a feature map, uses the activation function to introduce nonlinearity, reduces the dimension of the feature map through the pooling layer, reduces the number of parameters and the amount of calculation, and finally classifies through the fully connected layer to obtain the final output result.
[0018] Another aspect of the present invention is to achieve the purpose as follows: A field rice disease identification system based on a multi-model fusion algorithm, characterized in that it includes a collection module, a display module, and a processing module;
[0019] The acquisition module has a built-in sensor for capturing images of rice leaf diseases in the field. The image of rice leaf diseases in the field is captured by adjusting the macro distance and focal length by rotating the lens, and the image is transmitted to the processing module in real time through the sensor;
[0020] The processing module is used for identifying rice leaf diseases in the field, and performs resizing, normalization and filling preprocessing on the received images; extracts features from the processed images using the trained basic models; and then uses a multi-model fusion algorithm to identify the final disease category;
[0021] The display module is used to display the results of rice leaf disease identification in the field, print the disease category and the corresponding confidence level on the captured picture, and then display it in real time;
[0022] The acquisition module and the display module are electrically connected to the processing module.
[0023] As a further limitation of the present invention, the acquisition module is a camera, the display module is a display screen, and the processing module is a Jetson Orin Nano development board; the camera is connected to the Jetson Orin Nano development board using a USB cable; the display screen is connected to the Jetson Orin Nano development board using an HDMI cable and a Micro-USB cable; the Jetson OrinNano development board is configured with a PyTorch deep learning framework for deploying the trained basic model and multi-model fusion algorithm to the development board.
[0024] As a further limitation of the present invention, it also includes a storage module and a heat dissipation module; the storage module is used to store the field rice leaf disease data collected by the collection module and the disease identification data extracted after the processing module processes the image; the heat dissipation module is used to absorb and dissipate the internal heat of the processing module, the display module and the storage module.
[0025] The present invention adopts the above technical scheme, and compared with the prior art, the beneficial effects are as follows: 1) The present invention uses a multi-model fusion algorithm to process images, which can efficiently and accurately obtain the category and confidence of rice leaf diseases, while ensuring the stability and generalization ability of the model; 2) The present invention uses a camera, Jetson Orin Nano and a display screen to realize real-time rice leaf disease recognition in the field. The equipment is light, easy to assemble and low-cost, which effectively reduces the difficulty of promotion and the threshold of use. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 Flowchart of the method of the present invention.
[0027] Figure 2 Schematic diagram of the basic model of the field rice leaf disease training set of the present invention.
[0028] Figure 3 A schematic diagram of the structure of the system of the present invention. DETAILED DESCRIPTION
[0029] like Figure 1 The design method of a field rice disease identification system based on a multi-model fusion algorithm shown in the figure comprises the following steps:
[0030] S1) using a mobile phone camera to take pictures to obtain rice leaf disease images with complex backgrounds in the field, and preliminarily processing the images;
[0031] The images of rice leaf diseases in the field were collected by mobile phones in the experimental field, including images of rice leaf diseases and health in HEIC format, as RGB images, including leaf color, shape, partial discoloration, curling and other symptoms.
[0032] The collected rice images were screened, and images with blur, shaky images, excessive occlusion, and other images that were not valuable for learning were removed. The screened images were transferred to the computer, and the HEIC format images were converted using Photoshop software. The rice leaf disease categories were identified by comparing the disease atlas and saved as JPG format, and then saved in corresponding folders according to categories. The rice leaf images included five categories, namely, bacterial leaf blight, brown spot, rice blast, dongru disease, and health.
[0033] S2) performing data enhancement on the field rice leaf disease images after preliminary processing, expanding the original data, and constructing a field rice leaf disease dataset;
[0034] Use Pycharm to call the PIL library to perform batch data enhancement on the preliminarily processed field rice leaf disease images; call the transpose function to flip the image; call the rotate function and set the rotation angle to 279 and 23 to rotate the image; call the Sharpness function to jitter the image color; call the Contrast function to enhance the image contrast; call the Brightness function to enhance the image brightness; call the Color function to enhance the image color; the above data enhancement technology enhances each image and saves it to the corresponding category folder, expanding the original image to eight times the original. Finally, a field rice leaf disease dataset containing 6090 images was constructed. The field rice leaf disease dataset is divided into a training set, a validation set, and a test set, with 4200, 1050, and 840 images, respectively.
[0035] S3) training multiple convolutional neural network models based on the field rice leaf disease dataset, verifying and optimizing the performance of each model, and confirming the optimal parameters;
[0036] like Figure 2 As shown, use Pycharm to configure the PyTorch deep learning framework, select Yolov5s-cls, EfficientNet_b0, MobileNet_v3, and ShuffleNet_v2 as the basic models; call the train function to train the model: load the field rice leaf disease training set, perform preprocessing operations such as resizing, normalization, and padding to make the image meet the input requirements of the model; load the weights of the basic model trained on the ImageNet dataset, and use transfer learning training; after one round of training, call the val function to load the verification set to verify the performance of each model; repeat the training for the set number of iterations and save the results in a local folder; compare and analyze the training results, and continuously optimize and improve until the best model is determined and saved;
[0037] The steps of calling the val function to verify the performance of the model are as follows: draw the cross entropy loss value curve of the training set and the validation set, continuously optimize the parameters to make the validation set loss value tend to be flat while fitting the training set loss value; calculate the accuracy of the model on the validation set and draw the curve, optimize the parameters to improve the recognition accuracy of the model;
[0038] The optimization and improvement techniques include using label smoothing regularization method and cosine annealing strategy to adjust the learning rate; label smoothing technology, setting the parameter to 0.1, this method adds noise to the true label probability distribution to soften the label, reduce the weight of the true label category when calculating the loss function, and achieve the effect of suppressing model overfitting and calibrating errors; after using label smoothing technology, the mathematical expression of the cross entropy loss value is as follows:
[0039]
[0040] Among them, Loss is the loss value without using label smoothing technology, ε is a very small hyperparameter, i represents the category, and y represents the true label;
[0041] The cosine annealing strategy adjusts the learning rate method, setting the initial learning rate to 0.0005, momentum to 0.9, and decay to 1e-5. This method adjusts the learning rate more gently, balancing the convergence speed and stability of the model; the mathematical expression of the learning rate is:
[0042]
[0043] Among them, η i is the learning rate used in a certain iteration, η min , η max are the upper and lower limits of the learning rate adjustment, T i is the current number of iterations, T max is the total number of iterations.
[0044] S4) calculating a weight parameter applicable to the whole world according to the cross entropy loss value of each model in the validation set, assigning corresponding weights to each model, calculating the final recognition result by using a multi-model fusion algorithm, and establishing a universal recognition model for rice leaf diseases in the field;
[0045] Use the softmax function to calculate the recognition prediction probability results of each basic model, use the smartCrossEntropyLoss function to calculate the cross entropy loss value of each basic model and accumulate until all images in the recognition validation set are recognized, calculate the average cross entropy loss value of each model, use the exponential function and power function to amplify the gap between the average loss values of each model, take the inverse and perform normalization calculation to obtain the global weight parameter corresponding to each model. The mathematical expression of the global weight parameter is:
[0046]
[0047] Among them, Loss i is the loss function value of the i-th model, C is the number of basic models, Loss j is the loss function value of the j-th model.
[0048] The global weight parameter is used when identifying unknown target objects. The specific steps are as follows: Use softmax to calculate the recognition prediction probability results of each basic model, multiply the weight parameters corresponding to each model with the prediction probability results and add them bit by bit to obtain a set of weighted prediction probability results, and use the max function to obtain the category corresponding to the maximum probability value, which is the final recognition result.
[0049] S5) The trained general recognition model is deployed to the Jetson Orin Nano development board, and the camera, display screen and power supply are combined to form a field rice leaf disease recognition system.
[0050] like Figure 3 As shown, a field rice disease identification system based on a multi-model fusion algorithm includes a collection module, a display module, a processing module; a storage module and a heat dissipation module;
[0051] The acquisition module has a built-in sensor for capturing images of rice leaf diseases in the field. The macro and focal length are adjusted by rotating the lens to capture images of rice leaf diseases in the field, and the images are transmitted to the processing module in real time through the sensor.
[0052] The processing module is used to identify rice leaf diseases in the field. It resizes, normalizes and fills the received images for preprocessing; uses the trained basic model to extract features from the processed images; and then uses the multi-model fusion algorithm to identify the final disease category;
[0053] The display module is used to display the results of rice leaf disease identification in the field, printing the disease category and corresponding confidence level on the captured image and then displaying it in real time;
[0054] The storage module is used to store the field rice leaf disease data collected by the collection module and the disease identification data extracted by the processing module after processing the image;
[0055] The heat dissipation module is used to absorb and dissipate the internal heat of the processing module, display module and storage module.
[0056] The acquisition module is a camera, the display module is a display screen, and the processing module is a Jetson Orin Nano development board; the camera is connected to the Jetson Orin Nano development board using a USB cable; the display screen is connected to the Jetson Orin Nano development board using an HDMI cable and a Micro-USB cable, and is equipped with a battery power supply to form a portable device for identifying rice leaf diseases in the field; the Jetson Orin Nano development board is configured with the PyTorch deep learning framework, WinSCP is used to remotely connect to the Jetson OrinNano development board, port 22, IP 192.168.55.1, and user name jetson are set, and the trained basic model and multi-model fusion algorithm are deployed to the development board.
[0057] When the present invention is working, the predict program is run in the terminal, and the model to be used is selected by inputting the serial number. After the model is loaded, the device turns on the camera to capture the picture, and the picture is transmitted to the Jetson Orin Nano development board through the sensor of the camera; after receiving the picture, the Jetson Orin Nano development board calls the classify_image function to load the selected model, resizes, normalizes and fills the picture before inputting the image into the model, and the model extracts the features of the input image through the convolution layer to generate a feature map, uses the activation function to introduce nonlinearity, reduces the dimension of the feature map through the pooling layer, reduces the number of parameters and the amount of calculation, and finally obtains the final output result through the fully connected layer for classification.
[0058] The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solution disclosed in the present invention, technicians in this field can make some substitutions and deformations to some technical features therein according to the disclosed technical content without creative labor, and these substitutions and deformations are all within the protection scope of the present invention.
Claims
1. A design method for a field rice disease identification system based on a multi-model fusion algorithm, characterized in that: The following steps are involved: S1) Use a mobile phone camera to take pictures to obtain rice leaf disease images with complex backgrounds in the field, and perform preliminary processing on the images; S2) Performing data enhancement on the initially processed field rice leaf disease images, expanding the original data, and constructing a field rice leaf disease dataset; S3) Train multiple convolutional neural network models based on the field rice leaf disease dataset, verify the performance of each model and optimize it to confirm the optimal parameters; S4) According to the cross entropy loss value of each model in the validation set, the weight parameters applicable to the whole world are calculated, and the corresponding weights are assigned to each model respectively. The final recognition result is calculated by using the multi-model fusion algorithm, and a universal recognition model for rice leaf diseases in the field is established; S5) Deploy the trained general recognition model to the Jetson Orin Nano development board, and combine the camera, display, and power supply to form a field rice leaf disease recognition system.
2. The method for identifying field rice diseases based on a multi-model fusion algorithm according to claim 1, characterized in that: The preliminary processing of the image in step S1) is specifically as follows: based on the collected field rice leaf disease images, the rice disease images that are blurred, jittered, distorted, have unknown diseases or are missing features are cleaned; the storage format of the screened rice leaf disease images is uniformly converted into a JPG format; Photoshop is used to perform multiple local cropping on each rice leaf disease image to expand the number of disease images, and the cropped rice leaf disease images are divided into five categories according to the disease atlas, namely, white leaf blight, brown spot, rice blast, dongru disease and health, and each category is saved in a folder.
3. The method for identifying field rice diseases based on a multi-model fusion algorithm according to claim 1, characterized in that: In step S2), data enhancement is performed on the field rice leaf disease images after preliminary processing, specifically including: data enhancement is performed on the original white leaf blight, brown spot, rice blast, donggro disease and healthy category images stored in each folder by flipping, rotating, color enhancement, brightness enhancement, contrast enhancement and random color change operations, so that the data volume is expanded to eight times the original amount, and a five-category field rice leaf disease dataset containing four diseases and health is constructed.
4. The method for identifying field rice diseases based on a multi-model fusion algorithm according to claim 1, characterized in that: The step S3) of training multiple convolutional neural network models based on the field rice leaf disease dataset is specifically as follows: four convolutional neural network models with less homogeneity, less performance difference and lighter structure are selected as basic models; and Train.py is used to train the models using a transfer learning method.
5. The method for identifying field rice diseases based on a multi-model fusion algorithm according to claim 1, characterized in that: The verification and optimization of the performance of each model in step S3) specifically includes: using val.py to verify the performance of each basic model and adjust the parameters, repeatedly verifying and adjusting and comparing to obtain the best parameter group; using label smoothing technology and cosine annealing to optimize the learning rate; The number of training iterations was set to 110, the initial learning rate was 0.0005, the batch size was set to 16, Adam was selected as the optimizer, and the optimal basic model was obtained through training.
6. The method for identifying field rice diseases based on a multi-model fusion algorithm according to claim 1, characterized in that: The step 4) specifically includes: calculating the average cross entropy loss value of each basic model through the validation set, using the exponential function and the power function to amplify the difference between the cross entropy loss values and then performing normalization calculation to obtain a weight parameter applicable to the global situation; using the basic model to identify the target object respectively, assigning the weight parameters to the results of the corresponding basic model respectively, and calculating to obtain the final result.
7. The method for identifying field rice diseases based on a multi-model fusion algorithm according to claim 1, characterized in that: The step 5) specifically includes: running the predict program on the terminal, selecting the model to be used by inputting the serial number, and after loading the model, the device turns on the camera to capture the picture, and the picture is transmitted to the Jetson Orin Nano development board through the camera sensor; after receiving the picture, the Jetson Orin Nano development board calls the classify_image function to load the selected model, resizes, normalizes and fills the picture before inputting the image into the model, the model extracts the features of the input image through the convolution layer to generate a feature map, uses the activation function to introduce nonlinearity, reduces the dimension of the feature map through the pooling layer, reduces the number of parameters and the amount of calculation, and finally classifies through the fully connected layer to obtain the final output result.
8. A field rice disease identification system based on a multi-model fusion algorithm, characterized in that: It includes acquisition module, display module and processing module; The acquisition module has a built-in sensor for capturing images of rice leaf diseases in the field. The image of rice leaf diseases in the field is captured by adjusting the macro distance and focal length by rotating the lens, and the image is transmitted to the processing module in real time through the sensor; The processing module is used for identifying rice leaf diseases in the field, and performs resizing, normalization and filling preprocessing on the received images; and extracts features from the processed images using the trained basic model; Then use the multi-model fusion algorithm to identify the final disease category; The display module is used to display the results of rice leaf disease identification in the field, print the disease category and the corresponding confidence level on the captured picture, and then display it in real time; The acquisition module and the display module are electrically connected to the processing module.
9. The field rice disease identification system based on multi-model fusion algorithm according to claim 8, characterized in that: The acquisition module is a camera, the display module is a display screen, and the processing module is a Jetson Orin Nano development board; the camera is connected to the Jetson Orin Nano development board using a USB cable; the display screen is connected to the Jetson Orin Nano development board using an HDMI cable and a Micro-USB cable; the Jetson Orin Nano development board is configured with a PyTorch deep learning framework for deploying the trained basic model and multi-model fusion algorithm to the development board.
10. The field rice disease identification system based on multi-model fusion algorithm according to claim 8, characterized in that: It also includes a storage module and a heat dissipation module; the storage module is used to store the field rice leaf disease data collected by the collection module and the disease identification data extracted after the processing module processes the image; the heat dissipation module is used to absorb and dissipate the internal heat of the processing module, display module and storage module.