Method for generating pest data for a specified area based on understanding and true scene

Abstract

The application discloses a specified area pest data generation method based on understanding and real scene, and comprises the following steps: S1, collecting pest images; S2, pest labeling; S3, extracting the high-frequency detail map of the reference map to generate the detail token; S4, using DINOv2 to extract the identity features of the reference map to generate the identity token; S5, sending the Detail Token and the ID Token into the disease spot generation model to learn image generation; and S6, using the disease spot generation model obtained through the training of S5 to generate the disease spot image through smearing operation. The method realizes the generation of a large number of high-quality controllable pest data sets, can be widely applied to the pest recognition scenes of crops such as tomatoes, corns and rice, generates high-quality agricultural pest data through AI, and enables model training and intelligent detection.

Description

Technical Field

[0001] This invention belongs to the field of agricultural data and relates to a method for generating pest and disease data, specifically a method for generating pest and disease data for a specified area based on understanding and real-world scenarios. Background Technology

[0002] Agricultural pests and diseases affect crop yields, and traditional methods of pest and disease control mainly rely on chemical agents. In recent years, with continuous breakthroughs in science and technology, new pest control methods such as laser pest control have begun to be applied in the agricultural field. Most existing pest control methods rely on manual visual judgment, which suffers from low accuracy, time-consuming processes, and high levels of expertise required. Image recognition technology in the field of computer vision has also been applied in agriculture in recent years, but using the Transformer architecture for pest and disease classification requires a large number of data samples. However, agricultural pest and disease data is characterized by long collection periods, low incidence rates, and complex collection environments, resulting in a severe shortage of relevant training data. Summary of the Invention

[0003] To address the challenges of collecting and sparse data on specific agricultural crop diseases and pests during classification training, this invention integrates existing image data and utilizes techniques such as diffusion models and high-frequency detail maps to provide a method for generating disease and pest data for specified areas based on understanding and real-world scenarios. This method enables the generation of large-scale, high-quality, and controllable disease and pest datasets, which can be widely applied to disease and pest identification scenarios for crops such as tomatoes, corn, and rice. By generating high-quality agricultural disease and pest data through AI, it empowers model training and intelligent detection.

[0004] The objective of this invention is achieved through the following technical solution:

[0005] A method for generating pest and disease data for a specified area based on understanding and real-world scenarios includes the following steps:

[0006] Step S1: Collect images of pests and diseases:

[0007] In an open environment of real farmland, disease spots were directly captured using photographic equipment. The specific steps are as follows: Assume the point of contact between the diseased plant and the ground is the origin of the coordinate system, and the position of the leaf is... The shooting equipment is placed in a certain location in space, denoted as... At certain intervals Collect images of lesions;

[0008] Step S2, Pest and Disease Labeling:

[0009] Step S21: Use SAM technology to separate the plant leaves from the natural environment;

[0010] Step S22: Process the segmented leaf image using a color-based threshold filtering method to obtain a mask image: Assume the input image is Color gamut is For any pixel in the image Convert it from RGB space to HSV space, the converted vector is , It is a pixel The transformed vector, It is Pixel conversion The method It is a pixel The converted color tone, It is a pixel The converted saturation Yes, pixels The converted brightness The subscript is used, and then a threshold range is set based on the lesions. The lower limit vector is set as follows. The upper limit vector is , will satisfy The pixels with threshold conditions are set as masks. By iterating through and processing each pixel of the image, the final mask image is obtained.

[0011] Step S23: Extract the lesion area by overlaying the mask image and the original image;

[0012] Step S3: Extract high-frequency detail images from the reference image and generate detail tokens:

[0013] The mask image is subjected to Fourier transform, and then a Sobel high-pass filter is used to extract high-frequency detail feature maps. Finally, the extracted line contour map is pasted onto the background image to form a Collage puzzle sticker.

[0014] Step S4: Using DINOv2, extract identity features from the reference image to generate identity tokens:

[0015] The open-source model DINOv2 is introduced, and the identity feature extraction module is used for feature extraction to remove disease spots in the field background. The output is a global token and multiple patch blocks. A simple linear layer is used to concatenate the global token and these patch blocks to obtain a new token.

[0016] Step S5: Feed the Detail Token and ID Token into the lesion generation model to learn image generation:

[0017] The processed Detail Token and ID Token are fed into the UNet and ControlNet networks of the Stable Diffusion model. The outputs of the ControlNet network and the UNet network are superimposed to generate lesion images. During training, the parameters are continuously adjusted to achieve the minimum loss accuracy.

[0018] Step S6: Using the lesion generation model trained in step S5, generate lesion images through a smearing operation:

[0019] Step S6-1: Perform an arbitrary shape smearing operation on a healthy leaf image, set the pixel value of the smeared area to 1, and the background area to 0, and finally save the processing result as a PNG format mask0 mask.

[0020] Step S6-2: For each closed region of the recognition mask, apply an inflation factor to the mask1 mask; in the mask1 region where the pixel value is 1 after inflation, record the maximum and minimum horizontal coordinates respectively. , Maximum and minimum longitudinal coordinates Mark, and form a rectangular area accordingly. Its four vertex coordinates are , , , ;

[0021] Step S6-3: In the aforementioned rectangular area The diffusion process is performed internally: using an image containing pests and diseases as a reference image, the pest and disease areas are painted over to obtain a reference image; this reference image is then input into the diffusion model to generate pests and diseases on healthy leaves that correspond to the painted areas in the reference image.

[0022] Step S6-4: Apply mask0 to the diffused image and retain the area with a mask0 pixel value of 1; paste this area back onto the smeared position on the healthy leaf and perform feathering processing, so that the target pests and diseases are finally formed in the smeared area of ​​the healthy leaf.

[0023] Compared with the prior art, the present invention has the following advantages:

[0024] 1. This invention integrates the Stable Diffusion model to obtain the pre-trained diffusion model's understanding of lighting and texture, which can quickly generate lesions of a certain quality. The model has a good starting point, strong generalization ability, and high upper limit threshold.

[0025] 2. The lesion generation model of the present invention obtains Detail Tokens from the high-frequency detail images of the reference image and ID Tokens from the concatenation of DINOv2 during training. Combining the two provides feature vectors of lesions from two perspectives. The trained model can perceive the overall outline and also pay attention to the semantic details of the object.

[0026] 3. In the fine-tuning stage of this invention, ControlNet is introduced, which makes the images generated by the diffusion model more personalized, has better performance when generating the expected lesions, weakens the influence of irrelevant feature space on the generated lesions, and accurately achieves controllable lesion generation.

[0027] 4. Due to the limitations of the diffusion model principle, the traditional diffusion model can only diffuse within a rectangular frame. This invention introduces an expansion factor to expand the smeared area mask, and then uses an outer rectangle for diffusion generation, achieving the generation of lesions with a higher proportion of the smeared area without changing the quality. Attached Figure Description

[0028] Figure 1 This is a training method based on pest and disease datasets;

[0029] Figure 2 This refers to the steps involved in generating pest and disease data based on a trained model.

[0030] Figure 3 To train using real healthy images and real images of lesions;

[0031] Figure 4 To generate pests and diseases in a designated area by applying a paste;

[0032] Figure 5 Healthy leaves free from pests and diseases;

[0033] Figure 6 Leaves with lesions forming in the smeared area. Detailed Implementation

[0034] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0035] This invention provides a method for generating pest and disease data for a specified area based on understanding and real-world scenarios. The method includes the following steps:

[0036] Step S1: Collect images of pests and diseases:

[0037] To enable computers to capture the characteristics of lesions, a large amount of image data is needed. This invention selects a real farmland environment and uses photographic equipment to directly capture the lesions. This method simulates real lighting conditions, preserving the characteristics of the lesions to the greatest extent possible. Assuming the point of contact between the diseased plant and the ground is the origin of the coordinate system, and the position of the leaf is... The shooting equipment is placed in a certain location in space, denoted as... At certain intervals Collect images of the lesions. During the imaging process, it is important to ensure... This method ensures that the lesions captured in the photograph are on the upper surface of the leaf, not the lower surface. This is because it's more difficult to collect images of the lower surface of the leaf using the photographic equipment, and it also simulates the human visual perspective. Multiple angles of leaf lesion features can be collected, and multiple samples can be taken from the same lesion. During sampling, [the method involves selecting...]. The coordinates of the point.

[0038] Step S2, Pest and Disease Labeling:

[0039] After collecting images of pests and diseases, since the shooting location is outdoors, objects such as the ground and stems will be captured. These contents are interference with image learning. Therefore, this invention uses SAM technology to segment the plant leaves from the natural environment. The segmented leaves include both lesions and the entire leaf. During training, because local images of lesions are needed, the segmented leaf images need to be further processed. This invention uses a "color-based threshold filtering" method, assuming the input image is... Color gamut is For any pixel in the image Convert it from RGB space to HSV space, the converted vector is , It is a pixel The transformed vector, It is Pixel conversion The method It is a pixel The converted color tone, It is a pixel The converted saturation Yes, pixels The converted brightness Subscript It's the color tone. It's saturation. It is brightness, and then a threshold range is set according to the lesions, with the lower limit vector set as follows. The upper limit vector is , will satisfy The pixels with threshold conditions are set as masks. By traversing and processing each pixel of the image, the final mask image is obtained. Finally, by overlaying the mask image and the original image, the lesion area can be extracted.

[0040] Step S3: Extract high-frequency detail images from the reference image (i.e., the image containing the lesion area extracted in Step 2) and generate a Detail Token.

[0041] During generative training, the shape of the lesion is very important. Under ideal conditions, the shape of the lesion satisfies the Fisher-KPP equation:

[0042]

[0043] in: The location of the pathogen and time concentration, It is the diffusion coefficient (which depends on leaf surface humidity and pathogen activity). It is the Laplace operator, representing spatial diffusion. It is the inherent growth rate of pathogens.

[0044] Under these conditions, lesions spread outwards at a uniform degree and speed, gradually deepening in the center. However, in reality, lesion formation is influenced by factors such as stem veins, rainwater, and gravity. For example, in the early stages of lesion spread, they avoid stem veins, only covering them later. Rainwater promotes lesion spread along the direction of the water flow. If a leaf is vertical, the downward spread of lesions is stronger than the upward spread, leading to preferential spread on the outer periphery of the leaf. These factors result in irregular lesion shapes and uneven severity. To enable the model to learn the true shape of lesions during lesion generation, this invention uses high-frequency images for training. Specifically, the image undergoes a Fourier transform, and then a Sobel high-pass filter is used to extract high-frequency detail feature maps. Finally, the extracted line contour map is pasted onto the background image to form a Collage mosaic sticker.

[0045] Step S4: Using DINOv2, extract identity features from the reference image and generate an ID Token.

[0046] When generating the object, it is necessary to know the characteristics of the lesion, such as its type, approximate color, and core structure. Then, it is mapped to a feature space that is not affected by geometric deformation or lighting changes, in order to ensure that the generated object and the original object are semantically the same thing.

[0047] This invention introduces the powerful open-source model DINOv2 and develops an identity feature extraction module. This module uses a self-supervised attention mechanism to extract features from disease spots that have had their field background removed. The identity feature extraction module then outputs a global token and multiple patch blocks. A simple linear layer is used to concatenate the global token and these patch blocks to obtain a new token.

[0048] Step S5: Input the Detail Token and ID Token into the Unet network together for image generation learning.

[0049] The processed Detail Token and ID Token are fed into the UNet network. The StableDiffusion UNet network already possesses the ability to represent the overall image structure. To optimize the detail generation of UNet, a ControlNet network is introduced. It modifies the output of UNet to impose detail constraints on pixels without violating prior knowledge. The output tensor of ControlNet is initially set to 0, thus avoiding the large gradient noise introduced by random initialization. During training, parameters are continuously adjusted through a feedforward neural network to achieve minimal loss accuracy.

[0050] Step S6: Using the trained lesion generation model, generate lesion images through a smearing operation:

[0051] Through training, a lesion generation model was obtained, which can generate lesions in any region. This invention involves applying an arbitrary shape of paint to a healthy leaf. The painted area is stored as 1, and the background area as 0. The result is stored as a PNG format mask0. The system identifies each closed region of the mask and processes the mask1 using an expansion factor. Because the Stable Diffusion model generates lesions within a rectangular region, it approximates the actual lesion shape in the outer rectangular region. Without expansion, the painted area might have a large proportion of healthy area and a small proportion of lesion area. Using an expansion factor to process the mask effectively solves this problem. The maximum and minimum lateral coordinates of the expanded mask1 region with a pixel value of 1 are... , The maximum and minimum vertical coordinates are: , Then a rectangular area is formed. The coordinates of the four points are: , , , In the rectangular area The diffusion process involves using a reference image containing pests and diseases as a reference. The affected areas are then painted over and fed into the diffusion model, which generates pest and disease patterns on healthy leaves corresponding to the painted areas in the reference image. The diffused image is then processed using a mask0 mask, leaving areas with a mask0 pixel value of 1. These areas are then pasted back onto the painted areas on the healthy leaves. After some feathering, this process creates a function that generates high-quality lesions on the painted areas of healthy leaves. The specific process is as follows: Figure 3 and Figure 4 As shown, the effect is as follows Figure 5 and Figure 6 As shown.

Claims

1. A method for generating pest and disease data for a specified area based on understanding and real-world scenarios, characterized in that... The method includes the following steps: Step S1: Collect images of pests and diseases: In the open environment of real farmland, disease spots were directly captured using photographic equipment; Step S2, Pest and Disease Labeling: Step S21: Use SAM technology to separate the plant leaves from the natural environment; Step S22: Process the segmented leaf image using a color-based threshold filtering method to obtain a mask image; Step S23: Extract the lesion area by overlaying the mask image and the original image; Step S3: Extract high-frequency detail images from the reference image and generate detail tokens; Step S4: Using DINOv2, extract identity features from the reference graph and generate identity tokens (ID Tokens). Step S5: Input the Detail Token and ID Token into the lesion generation model to learn image generation; Step S6: Using the lesion generation model trained in step S5, generate lesion images through a smearing operation.

2. The method for generating pest and disease data for a specified area based on understanding and real-world scenarios according to claim 1, characterized in that... The specific steps of step S1 are as follows: Assuming the point of contact between the diseased plant and the ground is the origin, and the position of the leaf is... The shooting equipment is placed in a certain location in space, denoted as... At certain intervals Images of lesions were collected.

3. The method for generating pest and disease data for a specified area based on understanding and real-world scenarios according to claim 1, characterized in that... The specific steps of step S22 are as follows: Assume the input image is Color gamut is For any pixel in the image Convert it from RGB space to HSV space, the converted vector is , It is a pixel The transformed vector, It is Pixel conversion The method, It is a pixel The converted color tone, It is a pixel The converted saturation Yes, pixels The converted brightness The subscript is used, and then a threshold range is set based on the lesions. The lower limit vector is set as follows. The upper limit vector is , will satisfy The pixels with threshold conditions are set as masks. By traversing and processing each pixel of the image, the final mask image is obtained.

4. The method for generating pest and disease data for a specified area based on understanding and real-world scenarios according to claim 1, characterized in that... The specific steps of step S3 are as follows: The mask image is subjected to Fourier transform, and then a Sobel high-pass filter is used to extract high-frequency detail feature maps. Finally, the extracted line contour map is pasted onto the background image to form a Collage puzzle sticker.

5. The method for generating pest and disease data for a specified area based on understanding and real-world scenarios according to claim 1, characterized in that... The specific steps of step S4 are as follows: The open-source model DINOv2 is introduced, and the identity feature extraction module is used for feature extraction to remove disease spots in the field background. The output is a global token and multiple patch blocks. A simple linear layer is used to concatenate the global token and these patch blocks to obtain a new token.

6. The method for generating pest and disease data for a specified area based on understanding and real-world scenarios according to claim 1, characterized in that... The specific steps of step S5 are as follows: The processed Detail Token and ID Token are fed into the UNet and ControlNet networks of the Stable Diffusion model. The outputs of the ControlNet network and the UNet network are superimposed to generate lesion images. During training, the parameters are continuously adjusted to achieve the minimum loss accuracy.

7. The method for generating pest and disease data for a specified area based on understanding and real-world scenarios according to claim 1, characterized in that... The specific steps of step S6 are as follows: Step S6-1: Perform an arbitrary shape smearing operation on a healthy leaf image, set the pixel value of the smeared area to 1, set the background area to 0, and finally save the processing result as a PNG format mask0 mask. Step S6-2: For each closed region of the identification mask, apply an expansion factor to the mask1 mask; In the mask1 region where the pixel value is 1 after dilation, record the maximum and minimum horizontal coordinates respectively. , Maximum and minimum longitudinal coordinates Mark, and form a rectangular area accordingly. Its four vertex coordinates are , , , ; Step S6-3: In the aforementioned rectangular area The diffusion process is performed internally: using an image containing pests and diseases as a reference image, the pest and disease areas are painted over to obtain a reference image; this reference image is then input into the diffusion model to generate pests and diseases on healthy leaves that correspond to the painted areas in the reference image. Step S6-4: Apply mask0 to the diffused image and retain the areas where the mask0 pixel value is 1; The affected area is then reattached to the area on the healthy leaf and subjected to feathering treatment, ultimately resulting in the formation of the target pest or disease in the area on the healthy leaf.

Images