Plant disease identification method and system based on image generation

Abstract

The invention relates to the technical field of plant disease recognition, in particular to a plant disease recognition method and system based on image generation, and the method comprises the steps: constructing a plant leaf three-dimensional model, and generating plant leaf simulation image data of the plant leaf three-dimensional model under different conditions; the diversity is ensured while the sample data is expanded, so that the generalization ability of the recognition model is enhanced while the sample data acquisition problem is solved. Subsequently, the plant leaf simulation image data is subjected to realistic processing through the generative adversarial network, so that the recognition accuracy of the recognition model can be ensured.

Description

Technical Field

[0001] The present application relates to the technical field of plant disease identification, and in particular to a plant disease identification method and system based on image generation. Background Art

[0002] Plant disease identification is of great significance in agricultural production and landscape gardening. Traditional plant disease identification methods rely on manual observation and empirical judgment, which are inefficient and easily affected by human factors. In recent years, plant disease detection methods based on computer vision and deep learning have gradually emerged, using a large amount of image data to train deep learning models to achieve efficient and accurate disease identification. However, deep learning methods rely on a large amount of annotated image data, and obtaining these sample data is often time-consuming and laborious. In addition, due to environmental changes and different image acquisition conditions, there are certain differences in the performance of the actual collected image samples in the data set, which limits the generalization ability of the model. Summary of the invention

[0003] In order to at least to some extent overcome the problems in the related art of plant disease identification through deep learning methods, such as difficulty in obtaining sample data and limited generalization ability of the model, the present application provides a plant disease identification method and system based on image generation.

[0004] The scheme of this application is as follows: According to a first aspect of an embodiment of the present application, a plant disease identification method based on image generation is provided, comprising: Acquire real image data of plant leaves; the real image data of plant leaves at least includes: real image data of plant leaves in a healthy state and real image data of plant leaves in a diseased state; Constructing a three-dimensional model of a plant leaf according to the real image data of the plant leaf, and generating simulated image data of the plant leaf under different conditions for the three-dimensional model of the plant leaf; Training a generative adversarial network based on the real image data of the plant leaves; Performing realistic processing on the simulated image data of the plant leaves through a trained generative adversarial network; Combining the simulated image data of plant leaves after the authenticity processing with the real image data of the plant leaves to obtain training data; The recognition model is trained according to the training data; the recognition model is used to output the state recognition result of the plant leaves when the plant leaf image is input; the state of the plant leaves at least includes a healthy state and a disease state.

[0005] Preferably, the method further comprises: Annotating the real image data of the plant leaves; Generating corresponding annotations in the simulated image data of the plant leaves after the authenticity processing according to the annotation results of the real image data of the plant leaves; The annotation includes: the status of plant leaves.

[0006] Preferably, when the state of the plant leaves is a disease state, the label also includes: disease type, location and severity.

[0007] Preferably, obtaining the real image data of plant leaves includes: Obtain real image data of plant leaves in different seasons, growth stages, and environmental conditions.

[0008] Preferably, generating plant leaf simulation image data of the plant leaf three-dimensional model under different conditions includes: Generate plant leaf 3D models from simulated image data of plant leaves under different lighting conditions, angles and backgrounds.

[0009] Preferably, training a generative adversarial network based on the real plant leaf image data comprises: Training a generator and a discriminator of a generative adversarial network according to the real image data of the plant leaves; The simulated image generated by the generator is judged by the discriminator; Perform adversarial training of the generator and the discriminator based on the judgment results to optimize the generation quality of the generator; When the generation quality of the generator meets the preset conditions, the training of the generative adversarial network is completed.

[0010] Preferably, the method further comprises: Performing a data enhancement operation on the training data; The data enhancement includes at least: rotation, scaling, flipping and color adjustment.

[0011] Preferably, the training process of the recognition model includes: Forward propagation, backpropagation and parameter update; The training parameters of the recognition model are adjusted in real time according to the training data and the model structure.

[0012] Preferably, the method further comprises: Evaluate the performance of the recognition model based on a variety of evaluation indicators using a preset test data set; Adjust the parameters of the recognition model according to the evaluation results until the evaluation results of the recognition model meet the standards, and complete the training of the recognition model; The evaluation indicators include at least: accuracy, recall and F1-score indicators.

[0013] According to a second aspect of an embodiment of the present application, a plant disease identification system based on image generation is provided, comprising: An acquisition module, used for acquiring real image data of plant leaves; the real image data of plant leaves at least includes: real image data of plant leaves in a healthy state and real image data of plant leaves in a diseased state; A three-dimensional modeling module is used to construct a three-dimensional model of a plant leaf according to the real image data of the plant leaf, and to generate simulated image data of the plant leaf under different conditions of the three-dimensional model of the plant leaf; A first training module, used for training a generative adversarial network based on the real image data of the plant leaves; A reality processing module, used for performing reality processing on the simulated image data of the plant leaves through a trained generative adversarial network; A data combining module, used for combining the simulated image data of plant leaves after the realization process with the real image data of the plant leaves to obtain training data; The second training module is used to train the recognition model according to the training data; the recognition model is used to output the state recognition result of the plant leaves when the plant leaf image is input; the state of the plant leaves at least includes a healthy state and a diseased state.

[0014] The technical solution provided by this application may have the following beneficial effects: The plant disease identification method based on image generation in the present application includes: obtaining real image data of plant leaves; the real image data of plant leaves at least includes: real image data of plant leaves in a healthy state and real image data of plant leaves in a diseased state; constructing a three-dimensional model of plant leaves according to the real image data of plant leaves, and generating plant leaf simulation image data of the three-dimensional model of plant leaves under different conditions; training a generative adversarial network according to the real image data of plant leaves; performing realistic processing on the simulated image data of plant leaves through the trained generative adversarial network; combining the simulated image data of plant leaves after the realistic processing with the real image data of plant leaves to obtain training data; training a recognition model according to the training data; the recognition model is used to output the state recognition result of the plant leaves when the plant leaf image is input; the state of the plant leaves includes at least a healthy state and a diseased state.

[0015] In this technical solution, by constructing a three-dimensional model of plant leaves and generating plant leaf simulation image data of the three-dimensional model of plant leaves under different conditions, the sample data is expanded while ensuring diversity, thereby solving the problem of sample data acquisition and enhancing the generalization ability of the recognition model. Subsequently, the plant leaf simulation image data is realistically processed through a generative adversarial network to ensure the recognition accuracy of the recognition model.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] Figure 1 This is a flow chart of a plant disease identification method based on image generation provided by an embodiment of the present application; Figure 2 It is a flowchart of another plant disease identification method based on image generation provided by an embodiment of the present application; Figure 3 This is a flow chart of another method for identifying plant diseases based on image generation provided by an embodiment of the present application; Figure 4 This is a structural schematic diagram of a plant disease identification system based on image generation provided by an embodiment of the present application.

[0019] Reference numerals: acquisition module-21; three-dimensional modeling module-22; first training module-23; reality processing module-24; data combination module-25; second training module-26. DETAILED DESCRIPTION Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0020] Embodiment 1 Figure 1 is a flow chart of a plant disease identification method based on image generation provided by an embodiment of the present application, referring to Figure 1 , a plant disease recognition method based on image generation, comprising: S11: Acquire real image data of plant leaves; the real image data of plant leaves at least includes: real image data of plant leaves in a healthy state and real image data of plant leaves in a diseased state; In actual practice, real image data of plant leaves are obtained through high-definition cameras or professional image acquisition equipment to ensure the clarity and resolution of the image.

[0021] It should be noted that plant leaves generally exist in two states: healthy state and diseased state, and the present technical solution is to identify diseases of plant leaves. Therefore, in this embodiment, when obtaining the real image data of plant leaves, it is necessary to simultaneously obtain the real images of plant leaves in these two states.

[0022] Preferably, in this embodiment, real image data of plant leaves in different seasons, different growth stages and different environmental conditions are obtained.

[0023] Acquiring real image data of plant leaves in different seasons includes: acquiring real image data of plant leaves in spring, summer, autumn and winter.

[0024] Acquire real image data of plant leaves at different growth stages, including: acquire real image data of plant leaves at bud stage, generator stage, and mature stage.

[0025] Acquire real image data of plant leaves under different environmental conditions, including: acquire real image data of plant leaves under normal sunlight conditions, intermittent sunlight conditions, rainy weather, stormy weather, and normal weather.

[0026] In this embodiment, by acquiring real image data of plant leaves in different seasons, different growth stages, and different environmental conditions, the diversity of sample data can be further ensured.

[0027] S12: constructing a three-dimensional model of a plant leaf according to the real image data of the plant leaf, and generating plant leaf simulation image data of the three-dimensional model of the plant leaf under different conditions; In specific practice, a 3D model of a plant leaf is built using 3D modeling software (such as Blender, Speedtree, GrowFX, etc.). The constructed 3D model should at least include the morphological characteristics, texture, and color of the plant leaf.

[0028] Generate plant leaf 3D models and plant leaf simulation image data under different conditions, including: Generate plant leaf 3D models from simulated image data of plant leaves under different lighting conditions, angles and backgrounds.

[0029] In the modeling software, different lighting conditions, angles, and backgrounds are set to generate diverse plant leaf simulation image data to ensure the diversity of sample data.

[0030] Specifically, different lighting conditions include: 30% lighting, 60% lighting, and 100% lighting.

[0031] Different angles include: front, back, left, right, up, down, left front, right front, left back, right back, left front up, right front up, left back up, right back up, left front down, right front down, left back down, right back down, etc.

[0032] The background is more diverse, such as sunshine background, rainy background, thunderstorm background, breeze background and so on.

[0033] S13: Train a generative adversarial network based on real plant leaf image data; It should be noted that the generative adversarial network consists of a generator and a discriminator. The generator is responsible for capturing the distribution of sample data, while the discriminator is generally a binary classifier that determines whether the input is real data or a generated sample. The optimization process of this model is a "binary minimax game" problem. During training, one of the parties (the discriminator network or the generator network) is fixed, and the parameters of the other model are updated. The iterations are repeated alternately, and finally, the generator can estimate the distribution of sample data. The emergence of generative adversarial networks has greatly promoted the research on unsupervised learning and image generation.

[0034] It should be noted that the training of the generative adversarial network based on the real image data of plant leaves includes: The generator and discriminator of the generative adversarial network are trained based on real plant leaf image data; The simulated image generated by the generator is judged by the discriminator; Perform adversarial training of the generator and the discriminator based on the judgment results to optimize the generation quality of the generator; When the generation quality of the generator meets the preset conditions, the training of the generative adversarial network is completed.

[0035] In this embodiment, the real image data of plant leaves is input into the generator and discriminator of the generative adversarial network for training. During the training process, the generator generates simulated images and the discriminator judges the authenticity of the images. Both parties continuously improve the quality of the generated images through adversarial training. When the training is completed, the simulated images generated by the generator should be very close to the real image samples visually.

[0036] S14: Realizing the simulated image data of plant leaves through the trained generative adversarial network; The trained generative adversarial network has the ability to generate high-quality plant leaf images. By processing the plant leaf simulation image data with the trained generative adversarial network, a large amount of plant leaf simulation image data with high authenticity can be obtained.

[0037] It should be noted that the high-quality plant leaf image here refers to a simulated plant leaf image with high authenticity that is basically the same as the real plant leaf image.

[0038] S15: combining the simulated image data of the plant leaves after the authenticity processing with the real image data of the plant leaves to obtain training data; In this embodiment, the simulated image data of plant leaves after the authenticity processing and the real image data of plant leaves are combined to form a larger and richer training data set.

[0039] It should be noted that the method also includes: Perform data augmentation operations on training data; Data augmentation includes at least: rotation, scaling, flipping, and color adjustment.

[0040] In this embodiment, the training data set is further enriched by performing data enhancement operations (such as rotation, scaling, flipping, color adjustment, etc.) on the training data.

[0041] S16: training a recognition model according to the training data; the recognition model is used to output a state recognition result of the plant leaves when a plant leaf image is input; the state of the plant leaves includes at least a healthy state and a diseased state.

[0042] It should be noted that the training process of the recognition model includes: Forward propagation, backpropagation and parameter update; The training parameters of the recognition model are adjusted in real time according to the training data and model structure.

[0043] In specific practice, convolutional neural networks (such as ResNet, VGG, etc.) are selected to build recognition models. The enhanced training data is used for model training, and the model parameters are optimized to improve its ability to identify plant diseases.

[0044] It should be noted that, refer to Figure 2 , the method further comprises: Label the real image data of plant leaves; Generate corresponding annotations in the simulated image data of the plant leaves after the authenticity processing according to the annotation results of the real image data of the plant leaves; Labels include: Status of plant leaves.

[0045] Furthermore, when the state of the plant leaves is a disease state, the label also includes: disease type, location and severity.

[0046] It should be noted that this technical solution is to identify diseases on plant leaves, so all images in the training data need to be labeled. The images in the training data include real image data of plant leaves and simulated image data of plant leaves after realistic processing. The real image data of plant leaves need to be labeled when they are acquired, and the simulated image data of plant leaves also need to generate corresponding annotations after realistic processing.

[0047] In actual practice, professional annotation tools are used to annotate images to indicate the type, location and severity of the disease.

[0048] The labeling process can be completed by the system and then inspected by professional plant pathology experts to ensure the accuracy and consistency of the labeled data.

[0049] It should be noted that, refer to Figure 3 , the method further comprises: S17: Evaluate the performance of the recognition model based on a variety of evaluation indicators using a preset test data set; S18: adjusting the parameters of the recognition model according to the evaluation result until the evaluation result of the recognition model meets the standard, and completing the training of the recognition model; Evaluation indicators include at least: accuracy, recall and F1-score indicators.

[0050] After completing the training of the recognition model, the trained recognition model is deployed in the actual production environment, and the plant leaves are monitored in real time through image acquisition equipment to perform disease detection and early warning.

[0051] Preferably, the system also provides accurate disease prevention and control suggestions based on the test results to help farmers or garden managers take timely measures.

[0052] In this technical solution, by constructing a three-dimensional model of plant leaves and generating plant leaf simulation image data of the three-dimensional model of plant leaves under different conditions, the sample data is expanded while ensuring diversity, thereby solving the problem of sample data acquisition and enhancing the generalization ability of the recognition model. Subsequently, the plant leaf simulation image data is realistically processed through a generative adversarial network to ensure the recognition accuracy of the recognition model.

[0053] Embodiment 2 Figure 4 is a schematic diagram of a plant disease identification system based on image generation provided by an embodiment of the present application, referring to Figure 4 , a plant disease recognition system based on image generation, comprising: The acquisition module 21 is used to acquire the real image data of plant leaves; the real image data of plant leaves at least includes: the real image data of plant leaves in a healthy state and the real image data of plant leaves in a diseased state; The three-dimensional modeling module 22 is used to construct a three-dimensional model of a plant leaf according to the real image data of the plant leaf, and to generate simulated image data of the plant leaf under different conditions of the three-dimensional model of the plant leaf; A first training module 23 is used to train a generative adversarial network based on real plant leaf image data; A realisation processing module 24, used for realising the plant leaf simulation image data through a trained generative adversarial network; The data combining module 25 is used to combine the simulated image data of plant leaves after the realization process with the real image data of plant leaves to obtain training data; The second training module 26 is used to train the recognition model according to the training data; the recognition model is used to output the state recognition result of the plant leaves when the plant leaf image is input; the state of the plant leaves at least includes the healthy state and the diseased state.

[0054] The acquisition module 21 is specifically used to acquire real image data of plant leaves in different seasons, different growth stages and different environmental conditions.

[0055] The three-dimensional modeling module 22 is specifically used to generate plant leaf simulation image data of a three-dimensional model of a plant leaf under different lighting conditions, angles and backgrounds.

[0056] The first training module 23 is specifically used to train the generator and discriminator of the generative adversarial network based on the real image data of plant leaves; judge the simulated image generated by the generator through the discriminator; perform adversarial training of the generator and the discriminator according to the judgment result to optimize the generation quality of the generator; when the generation quality of the generator meets the preset conditions, complete the training of the generative adversarial network.

[0057] It should be noted that in this technical solution, by constructing a three-dimensional model of plant leaves and generating plant leaf simulation image data of the three-dimensional model of plant leaves under different conditions, the sample data is expanded while ensuring diversity, thereby solving the problem of sample data acquisition and enhancing the generalization ability of the recognition model. Subsequently, the plant leaf simulation image data is realistically processed through a generative adversarial network to ensure the recognition accuracy of the recognition model.

[0058] It should be noted that the plant disease recognition system based on image generation also includes: Annotation module, used to annotate the real image data of plant leaves; Generate corresponding annotations in the simulated image data of the plant leaves after the authenticity processing according to the annotation results of the real image data of the plant leaves; Labels include: Status of plant leaves.

[0059] Furthermore, when the state of the plant leaves is a disease state, the label also includes: disease type, location and severity.

[0060] It should be noted that the plant disease recognition system based on image generation also includes: Data enhancement module, used to perform data enhancement operations on training data; Data augmentation includes at least: rotation, scaling, flipping, and color adjustment.

[0061] It should be noted that the plant disease recognition system based on image generation also includes: The model testing module is used to evaluate the performance of the recognition model based on a variety of evaluation indicators using a preset test data set; adjust the parameters of the recognition model according to the evaluation results until the evaluation results of the recognition model meet the standards, completing the training of the recognition model; Evaluation indicators include at least: accuracy, recall and F1-score indicators.

[0062] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0063] It should be noted that, in the description of this application, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0064] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0065] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0066] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0067] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0068] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0069] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0070] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A plant disease identification method based on image generation, characterized in that: include: Obtain real image data of plant leaves; The plant leaf real image data at least includes: plant leaf real image data in a healthy state and plant leaf real image data in a diseased state; Constructing a three-dimensional model of a plant leaf according to the real image data of the plant leaf, and generating simulated image data of the plant leaf under different conditions for the three-dimensional model of the plant leaf; Training a generative adversarial network based on the real image data of the plant leaves; Performing realistic processing on the simulated image data of the plant leaves through a trained generative adversarial network; Combining the simulated image data of plant leaves after the authenticity processing with the real image data of the plant leaves to obtain training data; The recognition model is trained according to the training data; the recognition model is used to output the state recognition result of the plant leaves when the plant leaf image is input; the state of the plant leaves at least includes a healthy state and a disease state.

2. The method according to claim 1, characterized in that The method further comprises: Annotating the real image data of the plant leaves; Generating corresponding annotations in the simulated image data of the plant leaves after the authenticity processing according to the annotation results of the real image data of the plant leaves; The annotation includes: the status of plant leaves.

3. The method according to claim 2, characterized in that When the plant leaf is in a diseased state, the label further includes: disease type, location and severity.

4. The method according to claim 1, characterized in that Obtain real image data of plant leaves, including: Obtain real image data of plant leaves in different seasons, growth stages, and environmental conditions.

5. The method according to claim 1, characterized in that Generate plant leaf 3D models and plant leaf simulation image data under different conditions, including: Generate plant leaf 3D models from simulated image data of plant leaves under different lighting conditions, angles and backgrounds.

6. The method according to claim 1, characterized in that Training a generative adversarial network according to the real plant leaf image data includes: Training a generator and a discriminator of a generative adversarial network according to the real image data of the plant leaves; The simulated image generated by the generator is judged by the discriminator; Perform adversarial training of the generator and the discriminator based on the judgment results to optimize the generation quality of the generator; When the generation quality of the generator meets the preset conditions, the training of the generative adversarial network is completed.

7. The method according to claim 1, characterized in that The method further comprises: Performing a data enhancement operation on the training data; The data enhancement includes at least: rotation, scaling, flipping and color adjustment.

8. The method according to claim 1, characterized in that The training process of the recognition model includes: Forward propagation, backpropagation and parameter update; The training parameters of the recognition model are adjusted in real time according to the training data and the model structure.

9. The method according to claim 1, characterized in that: The method further comprises: Evaluate the performance of the recognition model based on a variety of evaluation indicators using a preset test data set; Adjust the parameters of the recognition model according to the evaluation results until the evaluation results of the recognition model meet the standards, and complete the training of the recognition model; The evaluation indicators include at least: accuracy, recall and F1-score indicators.

10. A plant disease identification system based on image generation, characterized in that: include: An acquisition module, used for acquiring real image data of plant leaves; The plant leaf real image data at least includes: plant leaf real image data in a healthy state and plant leaf real image data in a diseased state; A three-dimensional modeling module is used to construct a three-dimensional model of a plant leaf according to the real image data of the plant leaf, and to generate simulated image data of the plant leaf under different conditions of the three-dimensional model of the plant leaf; A first training module, used for training a generative adversarial network based on the real image data of the plant leaves; A reality processing module, used for performing reality processing on the simulated image data of the plant leaves through a trained generative adversarial network; A data combining module, used for combining the simulated image data of plant leaves after the realization process with the real image data of the plant leaves to obtain training data; The second training module is used to train the recognition model according to the training data; the recognition model is used to output the state recognition result of the plant leaves when the plant leaf image is input; the state of the plant leaves at least includes a healthy state and a diseased state.

Images