Crop leaf disease detection method in complex scene based on style migration and mixed learning

Abstract

The invention relates to a crop leaf disease detection method in a complex scene based on style migration and mixed learning. Comprising the following steps: 1, designing a disease leaf style migration model based on a generative adversarial network, and training the style migration model by using marked source domain data; 2, designing a disease leaf detection model based on a feature extraction network, and connecting two modules used for supervised learning and unsupervised learning respectively behind the feature extraction network; wherein the classification module is designed for supervised learning, and the measurement module is designed for unsupervised learning; and 3, designing a dynamic weight function based on a fuzzy rule to dynamically adjust the weights of the two learning modes so as to dynamically train the feature extraction network and finally complete a leaf disease detection task in a complex scene. According to the method, a supervised learning module and an unsupervised learning module are designed, and data of a source domain and a target domain are combined to train a leaf disease detection model. The invention further designs a dynamic weighting function which is dynamically combined with two learning modes.

Description

Technical field:

[0001] The invention belongs to the field of crop disease detection, and in particular relates to a crop leaf disease detection method based on style transfer and hybrid learning. Background technology:

[0002] my country is a large agricultural country. The sustained stability of crop yields is related to my country's food security and has a significant impact on national destiny and people's livelihood. Therefore, crop disease prevention and control is of vital importance. Traditionally, crop disease detection and its severity are judged by visual inspection of crop tissues by well-trained experts or experienced growers, which leads to strong subjectivity, high professional labor costs, and low overall efficiency. With the popularization of digital camera equipment and the widespread application and development of artificial intelligence technology in the agricultural field, technologies such as image recognition, pattern recognition, machine learning, and deep learning have been widely used in crop leaf disease detection, enabling early detection and prevention of crop diseases, greatly improving planting efficiency and production capacity.

[0003] In recent years, target detection algorithms based on machine learning and deep learning have performed well in many fields and can monitor, screen and detect more intelligently and quickly. In order to improve the accuracy and speed of diagnosis results, scholars have studied automatic diagnosis of crop diseases based on traditional machine learning algorithms, such as random forests, k-nearest neighbors and support vector machines (SVM). However, since the selection and adoption of classification features are based on human experience, these methods improve recognition accuracy, but the recognition rate is still not high enough and is easily affected by artificial feature selection.

[0004] Disease patterns vary significantly (high intra-class variation) due to factors such as leaf morphology, uneven background, age of infected cells, leaf color, and differences in light during imaging. On the other hand, sometimes the visual symptoms of different diseases may appear similar (low inter-class variation) due to factors such as different lighting conditions and aging. Therefore, it is possible for scholars to study the use of image processing technology in CV (computer vision) technology to detect the infection of plant diseased leaves.

[0005] my country attaches great importance to the combination of artificial intelligence and traditional agriculture. The number of people participating in the research of agricultural automation is increasing, and great progress has been made. Shi Bingying, Li Jiaqi and other scholars used Fourier transform + infrared spectroscopy technology to analyze tomato leaves and effectively identify tomato diseases in the training set. Zhou Feiyan and others designed a tobacco recognition model and achieved good recognition results based on a 6-layer convolutional neural network. Wei Chao, Fan Zizhu, Zhang Hong and other scholars used the HOG+SVM machine learning algorithm to achieve a recognition rate of up to 0.95 for grape leaves and weeds. Zhang Lumei and others first separated the background and the target in the processing of cucumber disease detection in complex backgrounds, paving the way for the accurate extraction of cucumber feature attributes. The separation of segmentation and detection greatly improved the efficiency of target detection and classification. Wang Shulian and others used computer vision technology to sample a crop leaf with disease spots, and obtained the difference in the reflectance spectrum curves between the normal part and the diseased part, thus proposing a highly robust method for determining the cause of crop disease.

[0006] Although different machine vision methods have emerged to improve the overall efficiency of crop / crop disease analysis, multiple factors, such as the lighting conditions of crop images, shooting angles, multimodal lenses, temperature changes and other complex scenes, have brought great challenges to the accuracy of crop leaf disease detection. There is an urgent need for model algorithms with better performance that can adapt to disease detection in complex scenes. Summary of the invention:

[0007] In view of the fact that other crop leaf disease detection methods cannot adapt to complex scenes, the present invention proposes a leaf detection method based on style transfer and hybrid learning to solve the problem of leaf image disease detection in complex environments such as insufficient lighting and large shooting angles. It is characterized by comprising the following steps:

[0008] Step 1: Design a diseased leaf style transfer model based on a generative adversarial network and train the style transfer model using labeled source domain data;

[0009] Step 2: Design a diseased leaf detection model based on a feature extraction network, and connect two modules for supervised learning and unsupervised learning respectively after the feature extraction network. The present invention designs a classification module for supervised learning and a measurement module for unsupervised learning;

[0010] Step 3: Design a dynamic weight function based on fuzzy rules to dynamically adjust the weights of the two learning methods, so as to dynamically train the feature extraction network and finally complete the leaf disease detection task in complex scenarios.

[0011] The step 1 comprises the following steps:

[0012] Step 1.1: Use the existing model to preprocess the source data and target data. First, in order to reduce the influence of the complex background in the source image, the present invention uses AL-STSGAN to preprocess the source image and target data. s The complex background has been weakened:

[0013] p f =G l (p s )

[0014] Among them G l represents the processing of AL-STSGAN, p s is the source leaf image, p f ∈P f This is the leaf image after the background is weakened.

[0015] Step 1.2: Design a style transfer model based on generative adversarial networks, which includes a three-condition generator, a domain style discriminator, and a disease classifier. The three are trained alternately. For the three inputs of the generator, they first pass through three encoders with the same structure but no shared parameters: E1, E2, and E3. After that, the three sets of features are connected and input into the decoder F. It is worth noting that among the three inputs of the generator, p x and p f Must be paired, that is, each group of p x and p f All come from the same source image p s .

[0016] The triple conditional mapping completed by the generator can be expressed as:

[0017] p g =G(p x ,p f ,p b )

[0018] Where G represents a parameterized generator, x g To generate an image, it has both the background of the target scene and the main body of the leaf image of the source scene. In addition, it also retains the original leaf image features.

[0019] The discriminator is essentially a binary classifier composed of a convolutional network. The purpose of designing the discriminator is to make the generator generate images with the target style. Similar to the original GAN, the present invention also uses the minimax opposition method to train the two:

[0020]

[0021] Where D and G represent the parameterized discriminator and generator, respectively. adv The purpose is to reduce the gap between the generated data distribution and the target data distribution.

[0022] The step 2 comprises the following steps:

[0023] Step 2.1: In order to obtain stronger ability to extract image macro information, a relatively large convolution kernel is designed.

[0024] Step 2.2: Use AlexNet pre-trained on ImageNet as the feature extraction model. Specifically, remove the high-dimensional fully connected layer and the Softmax activation layer after the average pooling layer, and normalize the high-dimensional vector output by the average pooling layer.

[0025] Step 2.2: After AlexNet, design an architecture called Cascade Inception, which includes two maximum pooling layers and two classic Inception network structures. The first maximum pooling layer is used to filter the noise of the feature map generated by AlexNet, and then the two Inceptions extract the best discriminative features from multidimensional analysis.

[0026] Step 2.3: Use prior knowledge to select negative sample pairs in the source image and the target classified image. Use feature distance and comparative feature distance to mine positive sample pairs in the target domain.

[0027] Step 2.4: Design a metric-based unsupervised learning module, train the metric module based on positive and negative sample pairs, and adjust the feature extraction model at the same time.

[0028] The step 3 comprises the following steps:

[0029] Step 3.1: Designate the classification loss used in the supervised learning module as the main loss, then input the fuzzy logic to calculate its weight value according to the rate of change of the main loss in the current iteration, and finally determine the weight of the unsupervised module according to the weight of the main loss.

[0030] Step 3.2: Design a dynamic weight function based on fuzzy rules to dynamically adjust the weights of the two learning methods, thereby dynamically training the feature extraction network.

[0031] Step 3.3 Stochastic gradient descent (SGD) is used to update the weights of the convolutional neural network, but it may lead to the "local optimal" problem; to solve this problem, the accelerated gradient algorithm NAG is designed and applied to train the proposed model. The algorithm calculates the updated weights based on the previous iteration, and the specific formula is expressed as follows:

[0032] d i =βd i-1 +αg(θ-βd i-1 )

[0033] θ i=θ i-1 -d i

[0034] where d i represents the current update vector, d i-1 represents the last update vector, θ i is the parameter currently updated, g(θ) represents the θ gradient in the objective function, β is the momentum term, and α represents the learning rate.

[0035] Step 3.4: Use the converged feature extraction network to complete the diseased leaf identification task.

[0036] Beneficial effects of the present invention: Considering that the performance of the current plant disease leaf image detection classification model is far below expectations and that the system resources occupied in model training are too large, the present invention designs a crop disease leaf classification detection method based on style transfer and hybrid learning. The method uses a style transfer model to generate images with the style of the target domain for supervised learning, and then mines positive and negative sample pairs for unsupervised learning. The present invention improves the classification quality by suppressing the noise background of the original leaf image, extracting the diseased subject of the original image, and filtering similar units of the image. The present invention also designs a dynamic weight function based on fuzzy logic, which dynamically adjusts the weight between the two tasks by calculating the rate of change of the loss during training. Description of the drawings:

[0037] Figure 1 It is a flow chart of a diseased leaf classification and detection method based on style transfer and hybrid learning.

[0038] Figure 2 The overall structure diagram of the present invention is

[0039] Figure 3 This is a style transfer model diagram based on the generation of adversarial networks

[0040] Figure 4 This is a disease recognition model diagram based on hybrid learning Specific implementation method:

[0041] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Figure 1 It is a specific flow chart of the implementation of the present invention, Figure 2 It is the overall structure diagram of the present invention, Figure 3This is a style transfer model diagram based on the generative adversarial network. Figure 4 This is a model diagram of crop disease leaf classification detection based on style transfer and hybrid learning, such as Figure 1 As shown, the method comprises the following steps:

[0043] Step 1: Design a diseased leaf style transfer model based on a generative adversarial network and train the style transfer model using labeled source domain data;

[0044] Step 2: Design a diseased leaf detection model based on a feature extraction network, and connect two modules for supervised learning and unsupervised learning respectively after the feature extraction network. The present invention designs a classification module for supervised learning and a measurement module for unsupervised learning;

[0045] Step 3: Design a dynamic weight function based on fuzzy rules to dynamically adjust the weights of the two learning methods, so as to dynamically train the feature extraction network and finally complete the leaf disease detection task in complex scenarios.

[0046] The step 1 comprises the following steps:

[0047] Step 1.1: Use the existing model to preprocess the source data and target data. First, to reduce the influence of the background in the source image, the present invention uses SBSGAN to preprocess the source image and target data. s Background weakening has been carried out:

[0048] x f =S1(x s )

[0049] Where S1 represents the processing of SBSGAN, x f ∈X f This is the image after background reduction.

[0050] Afterwards, the present invention uses the disease estimation model to s Disease extraction from leaves:

[0051] x p =S2(x s )

[0052] Where S2 represents the processing of the disease estimation model, x p ∈X p This is an image of a diseased leaf.

[0053] Finally, in order to prevent the leaves in the target domain from affecting the generation process, the leaf segmentation model is used to obtain the leaf mask, and then the mask is used to filter the target leaves:

[0054]

[0055] in It is a Hadamard product; m t is x t The mask of the leaves; x b ∈X b For x t background.

[0056] Step 1.2: Design a style transfer model based on generative adversarial networks, which includes a three-condition generator, a domain style discriminator, and an image classifier. The three are trained alternately. For the three inputs of the generator, they first pass through three encoders with the same structure but no shared parameters: E1, E2, and E3. After that, the three sets of features are connected and input to the decoder F. It is worth noting that among the three inputs of the generator, x p and xf must be paired, that is, each group of x p and x f All come from the same source image x s .

[0057] The triple conditional mapping completed by the generator can be expressed as:

[0058] x g =G(x p ,x f ,x b )

[0059] Where G represents a parameterized generator, x g To generate an image, it has both the background of the target domain and the leaf body of the source domain, in addition to which it retains the original image.

[0060] The discriminator is essentially a binary classifier composed of a convolutional network. The purpose of designing the discriminator is to make the generator generate images with the target style. Similar to the original GAN, the present invention also uses the minimax opposition method to train the two:

[0061]

[0062] Where D and G represent the parameterized discriminator and generator, respectively. adv The purpose is to reduce the gap between the generated data distribution and the target data distribution

[0063] To generate image x g In order to keep the original image, the present invention adds an image classifier to the model. Before the introduction, the present invention completes the pre-training on the source domain. Use image classification loss to constrain x g The leaf image in:

[0064] L id = -logp(ys |x g )

[0065] where y s is x g The corresponding original image. id The purpose is to make the generated image retain the original image label.

[0066] The step 2 comprises the following steps:

[0067] Step 2.1: Use ResNet-50 pre-trained on ImageNet as the feature extraction model. Specifically, remove the 1000-dimensional fully connected layer and softmax activation layer after the average pooling layer, and normalize the 2048-dimensional vector output by the average pooling layer.

[0068] Step 2.2: Generate a large number of leaf images with target domain style and source domain images using the converged generator and preprocessed data. Design a supervised learning module based on the image classification model, which is trained using the generated images and the feature extraction network is trained at the same time.

[0069] Step 2.3: Use prior knowledge (leaf images in the source domain and target domain do not overlap) to select negative sample pairs in the source domain and target domain. Use feature distance and comparative feature distance to mine positive sample pairs in the target domain.

[0070] Step 2.4: Design a metric-based unsupervised learning module, train the metric module based on positive and negative sample pairs, and adjust the feature extraction model at the same time.

[0071] The step 3 comprises the following steps:

[0072] Step 3.1: Design a dynamic weight function to dynamically adjust the training process of the two modules, thereby dynamically training the feature extraction network. The present invention designates the classification loss used in the supervised learning module as the main loss and calculates the change rate of the main loss in the current iteration:

[0073]

[0074] Where L i Represents the value of the main loss at the i-th iteration. Afterwards, the present invention defines the dynamic weight of the main loss:

[0075]

[0076] where λ i is the weight of the main loss in the i-th iteration, and γ is the focusing strength. Finally, the weight of the unsupervised module is determined according to the weight of the main loss.

[0077] Step 3.2: Use the converged feature extraction network to complete the cross-domain leaf re-identification task.

[0078] It should be understood that parts not elaborated in detail in this specification belong to the prior art.

[0079] The above description in combination with the accompanying drawings is only a specific implementation method and process of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art should understand that this is only an example, and various changes and substitutions can be made to this implementation method without departing from the essence of the present invention. The scope of the present invention is limited only by the attached claims.

[0080] The embodiments of the present invention described with reference to the accompanying drawings are exemplary and are only used to explain the present invention. They should not be understood as limiting the present invention. The specific scope of the embodiments of the present invention is not limited thereto. On the contrary, all embodiments of the present invention include all changes and modifications that fall within the spirit and connotation of the appended claims.

Claims

1. A cross-domain person re-identification method based on hybrid learning, characterized in that: The steps include: Step 1: Design a diseased leaf style transfer model based on a generative adversarial network and train the style transfer model using labeled source domain data; Step 2: Design a diseased leaf detection model based on a feature extraction network, and connect two modules for supervised learning and unsupervised learning respectively after the feature extraction network; wherein the present invention designs a classification module for supervised learning, and designs a metric module for unsupervised learning; Step 3: Design a dynamic weight function based on fuzzy rules to dynamically adjust the weights of the two learning methods, so as to dynamically train the feature extraction network and finally complete the leaf disease detection task in complex scenarios; The step 1 comprises the following steps: Step 1.1: Use the existing model to preprocess the source data and target data; First, in order to reduce the influence of the complex background in the source image, the present invention uses AL-STSGAN to weaken the complex background: p f =G l (p s ) Among them G l represents the processing of AL-STSGAN, p s is the source leaf image, p f ∈P f This is the leaf image after the background is weakened; Step 1.2: Design a style transfer model based on generative adversarial networks, which includes a three-condition generator, a domain style discriminator, and a disease classifier; the three are trained alternately; for the three inputs of the generator, they first pass through three encoders with the same structure but no shared parameters: E1, E2, and E3; then, the three sets of features are connected and input into the decoder F; it is worth noting that among the three inputs of the generator, p x and p f must be paired, i.e. each set and comes from the same source image p s ; The triple conditional mapping completed by the generator can be expressed as: p g G(p x ,p f ,p b ) Where G represents a parameterized generator, x g To generate an image, it has both the background of the target scene and the main body of the leaf image of the source scene, and in addition, it also retains the original leaf image features; The discriminator is essentially a binary classifier composed of a convolutional network; The purpose of designing the discriminator is to make the generator generate images with the target style. Similar to the original GAN, the present invention also uses the minimax opposition method to train the two: Where D and G represent the parameterized discriminator and generator respectively; L adv The purpose is to reduce the gap between the generated data distribution and the target data distribution; The step 2 comprises the following steps: Step 2.1: In order to obtain stronger ability to extract image macro information, design a relatively large convolution kernel; Step 2.2: Use AlexNet pre-trained on ImageNet as the feature extraction model. Specifically, remove the high-dimensional fully connected layer and the Softmax activation layer after the average pooling layer, and normalize the high-dimensional vector output by the average pooling layer. Step 2.2: After AlexNet, design an architecture called Cascade Inception, which includes two maximum pooling layers and two classic Inception network structures; the first maximum pooling layer is used to filter the noise of the feature map generated by AlexNet, and then the two Inception layers extract the best discriminative features from the multidimensional analysis; Step 2.3: Using prior knowledge, select negative sample pairs in the source image and the target classification image; use feature distance and comparative feature distance to mine positive sample pairs in the target domain; Step 2.4: Design a metric-based unsupervised learning module, train the metric module based on positive and negative sample pairs, and adjust the feature extraction model; The step 3 comprises the following steps: Step 3.1: Designate the classification loss used in the supervised learning module as the main loss, then input the fuzzy logic to calculate its weight value according to the change rate of the main loss in the current iteration, and finally determine the weight of the unsupervised module according to the weight of the main loss; Step 3.2: Design a dynamic weight function based on fuzzy rules to dynamically adjust the weights of the two learning methods, thereby dynamically training the feature extraction network; Step 3.3 Stochastic gradient descent (SGD) is used to update the weights of the convolutional neural network, but it may lead to the "local optimum" problem; to solve this problem, the accelerated gradient algorithm NAG is designed and applied to train the proposed model; the algorithm calculates the updated weights according to the previous iteration, and the specific formula is expressed as follows: d i =βd i-1 +ag(θ-βd i-1 ) i i =θ i-1 -d i where d i represents the current update vector, d i-1 represents the last update vector, is the parameter of the current update, g(θ) represents the θ gradient in the objective function, β is the momentum term, and α represents the learning rate; Step 3.4: Use the converged feature extraction network to complete the diseased leaf identification task; The beneficial effects of the present invention are as follows: Taking into account that the performance of current plant disease leaf image detection and classification models is far below expectations and that the system resources occupied in model training are too large, the present invention designs a crop disease leaf classification and detection method based on style transfer and hybrid learning. The method uses a style transfer model to generate images with target domain style for supervised learning, and then mines positive and negative sample pairs for unsupervised learning. The present invention improves the classification quality by suppressing the noise background of the original leaf image, extracting the diseased subject of the original image, and filtering similar units of the image. The present invention also designs a dynamic weight function based on fuzzy logic, which dynamically adjusts the weight between the two tasks by calculating the rate of change of the loss during training.

Images