A corn disease and insect pest monitoring method based on image processing

Abstract

This paper provides a corn disease and insect pest monitoring method based on image processing, which belongs to the technical field of applying computer vision technology to plant disease prevention. Specifically, by analyzing corn images with an LSTM model, the method explores the temporal changes in corn status and predicts the most likely lesion type to occur when a certain change in plant appearance occurs at a certain time. Furthermore, the method integrates an attention network to focus on key time points and locations, thus resolving the bottleneck of current corn disease and insect pest research, which is the confusion and difficulty in distinguishing between multiple diseases and insect pests.

Description

Technical Field

[0001] The present application relates to the technical field of application of computer vision technology in plant disease prevention, and specifically to a corn disease and insect pest monitoring method based on image processing. Background Art

[0002] The complexity and monitoring difficulty of corn pests and diseases lies in the fact that corn is susceptible to multiple pests and diseases during its growth cycle, and the growth stages and parts of corn attacked by multiple pests and diseases are different. For example, powdery mildew generally occurs more frequently during the heading stage, attacking female ears and male ears. Corn borers eat stalks during the jointing stage of corn, and adults damage leaves during the heading stage. Corn root rot (caused by multiple pathogens) mainly occurs in the seedling stage, damaging the root system. The above situation is already very complicated. What makes it more complicated is that corn borers and cotton bollworms generally attack corn in the larval stage, usually drilling into and boring into corn stalks and cobs. Aphids, thrips, and corn spider mites generally attack corn in the adult stage. Aphids feed on heart leaves, filaments, and male ears. Thrips suck the juice of corn heart leaves or leaves, causing the leaves to lose green and spots. Corn spider mites suck the juice of the host leaf back tissue, and the affected leaves turn from yellow to white and die.

[0003] Currently, the discovery of corn pests and diseases mainly relies on the following methods, but all of them have serious shortcomings:

[0004] (1) Manual field survey has the following disadvantages:

[0005] It is inefficient and requires agricultural experts to check the symptoms of pests and diseases on each plant, which is time-consuming and labor-intensive, and cannot achieve real-time monitoring over a large area. Since it relies on a single expert, the accuracy of the diagnosis depends on the expert's subjective experience, and similar symptoms are prone to misjudgment.

[0006] (2) Traditional image recognition technology, for example, uses static model-based CNN recognition algorithms (such as ResNet and YOLO) to classify corn images collected by drones in corn fields to identify pests and diseases. However, due to the high shooting distance of drones and the difficulty in changing the shooting angle, the image shows a single part of the corn. The occurrence of corn pests and diseases is complex, and different pests and diseases affect different parts of the corn. Therefore, it is difficult to accurately determine the exact pest and disease, and thus it is impossible to provide an effective prevention and control plan.

[0007] (3) Sensor-assisted monitoring has the disadvantage of being susceptible to environmental interference. Systems based on multispectral or thermal infrared sensors are easily affected by field light and humidity, and have a high false alarm rate.

[0008] (4) The growth information of corn is taken into account in the model calculation to predict the probability of pest and disease threats at different growth stages. However, there are too many factors affecting the invasion of corn by pests and diseases. Integrating only the growth information will provide the model with a single directional information, resulting in poor generalization of the prediction model.

[0009] It can be seen that the current methods of dealing with corn diseases and insect pests cannot effectively discover the causes of plant diseases. Summary of the Invention

[0010] In view of the shortcomings of the above-mentioned related technologies, the present invention provides a corn disease and insect pest monitoring method based on image processing to solve the technical bottleneck encountered in image recognition of corn diseases and insect pests.

[0011] This application provides a corn disease and insect pest monitoring method based on image processing, including:

[0012] Use drones to collect multiple continuous images of the entire corn plant;

[0013] Multiple continuous local images of corn plants are collected by a fixed camera;

[0014] Extracting image features of the whole image of the corn plant and image features of the local image of the corn plant;

[0015] The image features of the extracted whole image of the corn plant are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs them into the next memory unit, and outputs the predicted pest and disease category.

[0016] Furthermore, before extracting the image features of the whole image of the corn plant and the image features of the local image of the corn plant, the following process may be performed:

[0017] Using near-infrared to enhance the contrast between healthy and damaged tissues in images;

[0018] Create a binary mask to separate vegetation from non-vegetation background pixels in the image.

[0019] Furthermore, before extracting the image features of the whole image of the corn plant and the image features of the local image of the corn plant, the following process may be performed:

[0020] Mark the damaged areas caused by accidental factors in the image and record them as noise;

[0021] The whole image of the corn plant and the local image of the corn plant carrying the noise are input into the UNet network to generate a noise prediction map;

[0022] The noise prediction image is subtracted from the whole image of the corn plant and the local image of the corn plant without noise to obtain the difference image.

[0023] The differential image is returned to the UNet network to generate the next round of noise prediction image and the next round of differential image, until the differential image output by the UNet network for the Nth time is close to the real image, and the network weights are updated through the back propagation algorithm.

[0024] Furthermore, the corn disease and insect pest monitoring method based on image processing also includes:

[0025] The long short-term memory network outputs the hidden state of the overall image output by each memory unit of the image features of the overall image of the corn plant;

[0026] The extracted image features of the local image of the corn plant are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs the next memory unit, and outputs the hidden state of the local image output by each memory unit;

[0027] The image features of the whole corn plant image and the image features of the local corn plant image are input into the spatiotemporal attention network, and the multi-channel features of the whole image are weighted by time steps.

[0028] The weighted overall graph hidden state and partial graph hidden state are then returned to the last memory unit, and the last memory unit updates the output and predicts the pest and disease category based on the input information.

[0029] Furthermore, the corn disease and insect pest monitoring method based on image processing also includes:

[0030] The long short-term memory network outputs the hidden state of the overall image output by each memory unit of the image features of the overall image of the corn plant;

[0031] The extracted image features of the local image of the corn plant are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs the next memory unit, and outputs the hidden state of the local image output by each memory unit;

[0032] Expand the overall hidden state and the partial hidden state to obtain the overall multi-channel features and the partial multi-channel features;

[0033] The multi-channel features of the overall image and the multi-channel features of the local image are input into the spatiotemporal attention network to perform spatial weighting on the multi-channel features of the overall image;

[0034] The weighted multi-channel features are then returned to the last memory unit of the length memory network, and the last memory unit updates the output to predict the pest and disease category.

[0035] Furthermore, the steps of training the long short-term memory network include the following process:

[0036] Use drones to collect multiple continuous images of the entire corn plant;

[0037] Multiple continuous local images of corn plants are collected by a fixed camera;

[0038] Annotate the whole image of corn plants and the local images of corn plants;

[0039] Extracting image features of the whole image of the corn plant and image features of the local image of the corn plant;

[0040] The extracted image features of the whole corn plant image are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs them into the next memory unit, and outputs the predicted pest and disease category;

[0041] Calculate the loss values ​​of the predicted pest and disease categories and the marked pest and disease categories, and adjust the model parameters until convergence.

[0042] Furthermore, the steps of training the long short-term memory network include:

[0043] Use drones to collect multiple continuous images of the entire corn plant;

[0044] Multiple continuous local images of corn plants are collected by a fixed camera;

[0045] Annotate the whole image of corn plants and the local images of corn plants;

[0046] Extracting image features of the whole image of the corn plant and image features of the local image of the corn plant;

[0047] The extracted image features of the whole corn plant image are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs them into the next memory unit, and outputs the predicted pest and disease category;

[0048] The long short-term memory network outputs the hidden state of the overall image output by each memory unit of the image features of the overall image of the corn plant;

[0049] The extracted image features of the local image of the corn plant are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs the next memory unit, and outputs the hidden state of the local image output by each memory unit;

[0050] The image features of the whole corn plant image and the image features of the local corn plant image are input into the spatiotemporal attention network, and the multi-channel features of the whole image are weighted by time steps.

[0051] Expand the overall hidden state and the partial hidden state to obtain the overall multi-channel features and the partial multi-channel features;

[0052] The multi-channel features of the overall image and the multi-channel features of the local image are input into the spatiotemporal attention network to perform spatial weighting on the multi-channel features of the overall image;

[0053] Then, the time-weighted overall graph hidden state, partial graph hidden state, and spatially weighted multi-channel features are returned to the last memory unit of the length memory network, and the last memory unit updates and outputs the predicted pest and disease category;

[0054] Calculate the updated predicted pest and disease category and the loss value of the marked pest and disease category, and adjust the model parameters until convergence.

[0055] As described above, the beneficial effects of the present invention are: by setting up the calculation of the long-short memory network, the evolution law of corn suffering from diseases and pests during the growth stage is explored, and the various factors that cause corn to become pathological are converted into the state evolution of corn over the growth time. By identifying the morphological changes, color changes, and shape changes of various parts of the plant, the types of diseases and pests that have occurred are predicted.

[0056] In addition, the information of the long-short memory network is optimized by combining the spatiotemporal attention network, and the growth information of pests and diseases that cause disease is integrated into the timeline of corn growth. According to the pest period that causes corn disease, the hidden information of the memory unit corresponding to each time step of the long-short memory network is weighted. The model pays more attention to the outbreak period of special pests, etc., so as to predict the types of pests and diseases more accurately at a specified time. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Schematic diagram of the steps of performing corn pest and disease monitoring based on image processing according to the present invention;

[0058] Figure 2 is a schematic diagram of a long short-term memory network used in an example of the present invention;

[0059] Figure 3 The diagram shows an example of a corn pest and disease classification network structure based on a long short-term memory network. DETAILED DESCRIPTION

[0060] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments.

[0061] Corn is one of the world's important food crops, but it is often attacked by various diseases and pests during its growth, which seriously affects its yield and quality.

[0062] With the advancement of agricultural modernization, intelligent monitoring technology is playing an increasingly important role in corn pest and disease control. Traditional manual field survey methods are inefficient and highly subjective, while chemical control methods bring environmental pollution and pesticide resistance.

[0063] By integrating a variety of advanced technical means, intelligent monitoring technology provides a reliable solution for the early identification, accurate warning and scientific prevention and control of corn diseases and pests.

[0064] Current intelligent corn pest and disease monitoring technologies combine drones, computer vision, remote sensing, and infrared technology. Deep learning models (such as improved ResNet and YOLO series) are used to identify and locate pests and diseases, while multispectral / hyperspectral imaging can detect early-stage latent diseases. Remote sensing uses drones and satellites to acquire multi-scale field data, coupled with IoT sensor networks to monitor environmental parameters in real time. Big data and artificial intelligence technologies analyze time series data through models such as graph neural networks to establish predictive and early warning systems. The integrated application of these technologies has significantly improved monitoring efficiency, enabling earlier detection of pests and diseases compared to manual inspections and reducing pesticide use.

[0065] However, due to the wide variety of corn diseases and pests, and the complex and changeable ways in which different diseases and pests invade corn, the current intelligent monitoring of corn diseases and pests lacks in-depth research on the internal correlations of corn diseases and pests. Computer vision technology is directly used to integrate various features of big data, and corn image features are analyzed to classify corn diseases and pests. The model calculation process relies solely on features, without exploring the internal correlations of various features. The efficiency of capturing key features is low, resulting in low model robustness.

[0066] The method proposed by the present invention can effectively solve the above problems. Figure 1 A corn pest and disease monitoring method based on image processing comprises the following steps:

[0067] S1: Use a drone to collect multiple continuous images of the entire corn plant.

[0068] For example, it can be the entire growth stage of the diseased plant, such as from the beginning of cultivation to the growth process, to being damaged by some kind of damage, and then to the growth process after treatment. The image a1 is collected at time T1, the image a2 is collected at time T2, and the image a3 is collected at time T4. n Time acquisition image a n And so on.

[0069] S2: Collect multiple continuous local images of corn plants through a fixed camera.

[0070] For example, the image b1 can be acquired at time T1, the image b2 can be acquired at time T2, and the image b3 can be acquired at time T1. n Time acquisition image b n And so on.

[0071] S3: Label the whole image of the corn plant and the local image of the corn plant.

[0072] The annotation content includes the corn growth stage, pest type, damaged part, the time of pest onset, and the time point of special pest infestation. The local image of the corn plant corresponding to the damaged part is then annotated with a relationship label.

[0073] For example, thrips can damage corn leaves by causing chlorosis, which can also damage the heart leaves. In this case, the heart leaf area should be marked with a focus label in the image area where chlorosis occurs.

[0074] If the corn plants are specifically affected by insect pests during image acquisition, the specific damage will also be labeled. For example, if the corn plants are affected by cotton bollworm larvae during image acquisition, the specific damage label indicating an outbreak of cotton bollworm larvae will be added.

[0075] The types of pests and diseases are not limited to corn borers, cotton bolls, aphids, thrips, corn spider mites, powdery mildew, corn root rot, etc. Sometimes corn plants are also attacked by a combination of several pests and diseases.

[0076] For example, it is usually the larvae of cotton bollworms that cause damage to corn plants, so the time when cotton bollworm larvae appear on corn plants can be regarded as the time when special pests and diseases appear.

[0077] During the training phase, the collected and labeled whole corn plant images and partial corn plant images are used as training samples. During the application phase, step S3 is not performed, and the collected whole corn plant images and partial corn plant images are directly input into the network for image processing.

[0078] For example, during the training phase, 6,000 images of corn plants were collected, 100 images each of corn borer damage, cotton bolls, aphids, thrips, and corn spider mites were collected, of which 200 were used as test samples. 2,400 images of corn plants were collected, 40 images each of corn borer damage, cotton bolls, aphids, thrips, and corn spider mites were collected, of which 120 were used as test samples to train the long-short memory network.

[0079] S4: Extract the image features of the whole image of the corn plant (x1, x2...x n ), image features of the local image of the corn plant (y1, y2...y n ).

[0080] Feature extraction can be done using networks such as ResNet and VGG. X1 corresponds to the feature set extracted from the first corn plant local image, and Y n represents the feature set extracted from the nth local image of a corn plant, and so on.

[0081] Before extracting the image features of the whole image of the corn plant, or before extracting the image features of the local image of the corn plant, the following steps may also be performed:

[0082] Near-infrared (wavelength 700-1000nm) is used to enhance the contrast between healthy and damaged tissues in the image.

[0083] S12: Use NDVI to create a binary mask to separate vegetation and non-vegetation background pixels in the image.

[0084] NDVI is the abbreviation of Normalized Difference Vegetation Index.

[0085] Pests and diseases can damage corn plants, causing changes in the state, shape, and color of some parts or areas. However, these changes in corn plants are not necessarily caused by pests and diseases. They may be caused by accidental factors such as drought, animal bites, and extreme climate.

[0086] The following processing can be performed on the collected whole corn plant images and partial corn plant images:

[0087] S21: Mark the damaged areas caused by accidental factors in the image and record them as noise.

[0088] S22: Inputting the whole image of the corn plant and the local image of the corn plant carrying the noise into the UNet network to generate a noise prediction map.

[0089] S23: Subtracting the noise prediction image from the whole image of the corn plant and the local image of the corn plant without noise to obtain a differential image.

[0090] S24: Return the differential image to the UNet network and continue steps S22-S23 until the output differential image is close to the real image, and update the network weights through the back propagation algorithm.

[0091] For example, we calculate the SSIM value between a difference image and an image of a whole corn plant with artificial noise removed. If the SSIM value is less than 0.000, the two images are considered to be infinitely close. SSIM (Structural Similarity Index) is a metric used to measure image quality and is widely used in image processing, computer vision, and image compression.

[0092] S25: Denoising the images collected in steps S1 and S2 using the adjusted UNet network.

[0093] S5: The image features of the whole image of the corn plant extracted in step S4 are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs them into the next memory unit, and outputs the predicted pest and disease category.

[0094] Figure 2 This is a schematic diagram of a long short-term memory network used in an example of this application. Figure 2As shown, the long short-term memory network includes several memory units, memory unit A, memory unit B, and memory unit C, and each memory unit inputs features corresponding to time.

[0095] Take memory unit B as an example, (x1, x2...x n ) t Enter memory unit B, and the calculation process can be seen as:

[0096] K11: Combined with the hidden information h of the previous memory unit A t-1 ,Controlling the forgetting of historical information through formula (1) can be seen as ,screening out the information based on the corn growth stage that is irrelevant to the ,information that causes the pest and disease at the current time, and forgetting it.

[0097] (1), in formula (1) is the weight matrix of the forget gate, is the bias term of the forget gate, is the activation function.

[0098] K12: Combined with the hidden information h of the previous memory unit A t-1 The supplementary memory is generated by formula (2) and formula (3), and based on the growth stage of corn, the relevant information of the estimated corn pests and diseases at the current time is calculated.

[0099] (2) To control the image characteristics collected at time t Information storage; in formula (2) is the weight matrix of the input gate, is the bias term of the input gate, is the activation function.

[0100] =tanh (3), The candidate state of corn is obtained based on the feature information extracted from the current image combined with the historical information of corn. In formula (3) is the correlation matrix for generating candidate states, is the corresponding bias term.

[0101] K13: Combine the candidate state obtained after memory supplement based on the current input feature and the corn pest state of the corn at time t-1 output by the previous memory unit A, and update the state through formula (4) to obtain the unit state of the current memory unit In this invention, the unit state represents the state of corn pests, storing the temporal pattern of pest evolution. Computer analysis of the corn pest state can reveal information such as whether the corn plant has been infested with pests and diseases up to the current moment, and the extent and type of pests infested, according to pest evolution patterns.

[0102] = * + * (4); Indicates the cell state of the previous memory cell.

[0103] K14: Processes the corn pest status through the output gate. Based on the corn status information at time t, the output gate filters and outputs more accurate prediction-related information for prediction calculation. The relevant formulas for the output gate are formula (5) and formula (6).

[0104] = (5);

[0105] (6).

[0106] is the hidden state of memory unit B, which is used for predictive calculation or transfer to the next memory unit. is the matrix of output gates, is the bias term of the output gate.

[0107] K15: Performs classification calculations on the hidden state of the memory cell corresponding to the time step to be predicted, predicting the pest category. Generally, the memory cell corresponding to the time step to be predicted is the last memory cell.

[0108] Z = Softmax( ), is the classification layer weight, is the classification layer bias term.

[0109] During model training, after the predicted pest and disease category is output through the long short-term memory network, the following steps can be performed to calculate the loss value of the predicted pest and disease category and the labeled pest and disease category, and adjust the model parameters until convergence.

[0110] For example, the cross entropy loss function can be used to calculate the cross entropy loss value of the predicted category and the labeled category, and the back propagation algorithm can be used to update the model parameters to minimize the loss value, and then the optimization algorithm such as gradient descent can be used to adjust the model parameters.

[0111] During the model application process, step S3 is not executed. After the image is collected, step S4 is executed to extract image features, and subsequent steps are performed to predict the type of pests and diseases.

[0112] When S4 extracts features, a multi-channel feature map is extracted using a CNN network. During the labeling process, the local image of the corn plant corresponding to the damaged part is annotated with a relationship label. Therefore, some channels of the multi-channel feature map store information of a special area, and the special area is associated with the local area image of the corn.

[0113] The marking process also marks the special damage caused by pests. Other channels of the multi-channel feature map store information about special damage, such as the channel information corresponding to the leaf edge and the area of ​​morphological changes caused by larvae.

[0114] The above characteristics can be used to expand the hidden state output by the long-short-term memory network and restore it to a multi-channel state diagram. This carries information from different channels, that is, spatial information, which can be used for spatial weighting. At the same time, because the hidden state carries the temporal information of different memory units, it can also be used for temporal weighting. Therefore, after the long-short-term memory network of the present invention outputs the pest species, it can continue to perform the following spatiotemporal memory network calculation steps to optimize the entire output content:

[0115] Executing step S5 can also output the overall hidden state of the image output by each memory unit (h1, h2...h t-1 、h t ……h n ).

[0116] S6: The image features of the local image of the corn plant extracted in step S4 are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs the next memory unit, and outputs the local hidden state of the output of each memory unit (h`1, h`2...h` t-1 、h` t ...h` n ).

[0117] S7: Expand the global image hidden state and the partial image hidden state to obtain the global image multi-channel features and the partial image multi-channel features.

[0118] S8: Input the image features of the hidden state of the overall graph and the hidden state of the local graph into the spatiotemporal attention network, and perform time-step weighting on the multi-channel features of the overall graph to focus on key time points; that is, weight the channel features corresponding to the special infestation time of special pests in the mark to focus on the information stored in the hidden state or cell state corresponding to the special infestation time of the pests.

[0119] For example, if cotton bollworm larvae damage corn plants, the time of their appearance is the key time point. Originally, the plant's growth and the growth of a particular insect were two separate timelines. However, the present invention collects images of corn along a time series of plant growth, establishes a timeline of memory units in a long-short-term memory network (LSM). Then, by labeling the larvae that damage corn in the corn growth time series, the LSM network learns the associations between features in the corn growth time series, embedding the cotton bollworm's evolutionary timeline into the corn growth time series. Through LSM training, the network is then used to predict the appearance of cotton bollworm larvae during a specific time period during corn growth.

[0120] The spatiotemporal attention network performs time-step weighting on the hidden state, which can be expressed by the following formulas (7) and (8):

[0121] (7);

[0122] (8).

[0123] =(h1、h2……h t-1 、h t ……h n ) or (h`1, h`2...h` t-1 、h` t ...h` n ), As the input of the attention mechanism, through linear transformation Hidden layer representation of temporal features in attention network. In turn Each element of is weighted, The time step with a large value is the key time point when the pests discovered by the model through learning begin to spread and develop, or the environmental conditions (temperature, humidity) are suitable for the development of pests and diseases.

[0124] 、 are learnable weights, is the bias term. It is the hidden feature after time attention weighting.

[0125] S9: Input the multi-channel features of the overall image and the multi-channel features of the local image into the spatiotemporal attention network, and perform spatial weighting on the multi-channel features of the overall image, that is, weight the channel features corresponding to the special infestation and damaged parts of the pests in the mark.

[0126] S8 and S9 are executed in parallel.

[0127] The latent sequence is reshaped to produce a spatial feature map F with dimensions W*H*C, where C represents the number of channels, W represents the image width, and H represents the image height. F-avg and F-max are calculated. F-avg is the mean along the channel dimension, reflecting the global importance of each spatial location. High-value regions may correspond to persistent lesions, such as rust spreading over several days.

[0128] F-max calculates the maximum value along the channel dimension to capture local salient features, such as the heart leaf area corresponding to thrips infestation and the dotted chlorotic areas on the leaf.

[0129] Perform channel splicing calculations [F-avg, F-max] and apply convolution kernel calculations to the spliced ​​results. For example, dilated convolution can be used to expand the receptive field for smaller borers like corn borers. Use larger convolution kernels to capture the transition area between diseased and healthy tissue.

[0130] The above calculation formula can be expressed by formula (9) and formula (10).

[0131] (9);

[0132] (10);

[0133] represents the convolution operation, is the activation function, Indicates channel splicing calculation.

[0134] S10: The weighted multi-channel features are then returned to the last memory unit of the length memory network, and the last memory unit updates and outputs the predicted pest and disease category.

[0135] S11: Then add the weighted overall hidden state and partial hidden state, and finally update the memory unit to output the predicted pest and disease category.

[0136] S12: Then add the weighted multi-channel features, the overall hidden state of the graph, the partial hidden state, and finally update the memory unit to output the predicted pest and disease category.

[0137] S10 may be executed, or S11 may be executed, or S12 may be executed.

[0138] Will Return to the last memory unit of the length memory network, perform the calculation process of the memory unit, obtain the new hidden state, and then perform classification calculation based on the new hidden state to predict the category of pests and diseases.

[0139] For example, reference Figure 3The figure shows an example of the network structure for corn pest and disease classification based on the long short-term memory network. The long short-term memory network used in the above process is based on the original one. A feature extraction network based on VGG is added to the head of the long short-term memory network, and a classification network and spatiotemporal attention network based on CNN convolution are added to the tail of the long short-term memory network.

[0140] During the model training process, step S5 may be executed to output the hidden state of the overall graph, and steps S6-S12 may be executed to update the output predicted pest and disease category, calculate the loss value based on the updated predicted pest and disease category, and adjust the model parameters.

Claims

1. A corn disease and insect pest monitoring method based on image processing, characterized in that: Includes methods: Use drones to collect multiple continuous images of the entire corn plant; Multiple continuous local images of corn plants are collected by a fixed camera; Extracting image features of the whole image of the corn plant and image features of the local image of the corn plant; The extracted image features of the whole corn plant image are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs them into the next memory unit, and outputs the predicted pest and disease category; Corn pest and disease monitoring methods based on image processing also include: The long short-term memory network outputs the hidden state of the overall image output by each memory unit of the image features of the overall image of the corn plant; The extracted image features of the local image of the corn plant are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs the next memory unit, and outputs the hidden state of the local image output by each memory unit; The image features of the whole corn plant image and the image features of the local corn plant image are input into the spatiotemporal attention network, and the multi-channel features of the whole image are weighted by time steps. The weighted overall graph hidden state and partial graph hidden state are then returned to the last memory unit, and the last memory unit updates the output and predicts the pest and disease category based on the input information.

2. The method according to claim 1, characterized in that Before extracting the image features of the whole image of the corn plant and the image features of the local image of the corn plant, the following process is performed: Using near-infrared to enhance the contrast between healthy and damaged tissues in images; Create a binary mask to separate vegetation from non-vegetation background pixels in the image.

3. The method according to claim 1, characterized in that Before extracting the image features of the whole image of the corn plant and the image features of the local image of the corn plant, the following process is performed: Mark the damaged areas caused by accidental factors in the image and record them as noise; The whole image of the corn plant and the local image of the corn plant carrying the noise are input into the UNet network to generate a noise prediction map; The noise prediction image is subtracted from the whole image of the corn plant and the local image of the corn plant without noise to obtain the difference image. The differential image is returned to the UNet network to generate the next round of noise prediction image and the next round of differential image, until the differential image output by the UNet network for the Nth time is close to the real image, and the network weights are updated through the back propagation algorithm.

4. The method according to claim 1, wherein Corn pest and disease monitoring methods based on image processing also include: Expand the overall hidden state and the partial hidden state to obtain the overall multi-channel features and the partial multi-channel features; The multi-channel features of the overall image and the multi-channel features of the local image are input into the spatiotemporal attention network to perform spatial weighting on the multi-channel features of the overall image; The weighted multi-channel features are then returned to the last memory unit of the length memory network, and the last memory unit updates the output to predict the pest and disease category.

5. The method according to claim 1, wherein The steps for training a long short-term memory network include the following process: Use drones to collect multiple continuous images of the entire corn plant; Multiple continuous local images of corn plants are collected by a fixed camera; Annotate the whole image of corn plants and the local images of corn plants; Extracting image features of the whole image of the corn plant and image features of the local image of the corn plant; The extracted image features of the whole corn plant image are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs them into the next memory unit, and outputs the predicted pest and disease category; Calculate the loss values ​​of the predicted pest and disease categories and the marked pest and disease categories, and adjust the model parameters until convergence.

6. The method according to claim 1, wherein The steps to train a long short-term memory network include: Use drones to collect multiple continuous images of the entire corn plant; Multiple continuous local images of corn plants are collected by a fixed camera; Annotate the whole image of corn plants and the local images of corn plants; Extracting image features of the whole image of the corn plant and image features of the local image of the corn plant; The extracted image features of the whole corn plant image are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs them into the next memory unit, and outputs the predicted pest and disease category; The long short-term memory network outputs the hidden state of the overall image output by each memory unit of the image features of the overall image of the corn plant; The extracted image features of the local image of the corn plant are input into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features, inputs the next memory unit, and outputs the hidden state of the local image output by each memory unit; The image features of the whole corn plant image and the image features of the local corn plant image are input into the spatiotemporal attention network, and the multi-channel features of the whole image are weighted by time steps. Expand the overall hidden state and the partial hidden state to obtain the overall multi-channel features and the partial multi-channel features; The multi-channel features of the overall image and the multi-channel features of the local image are input into the spatiotemporal attention network to perform spatial weighting on the multi-channel features of the overall image; Then, the time-weighted overall graph hidden state, partial graph hidden state, and spatially weighted multi-channel features are returned to the last memory unit of the length memory network, and the last memory unit updates and outputs the predicted pest and disease category; Calculate the updated predicted pest and disease category and the loss value of the marked pest and disease category, and adjust the model parameters until convergence.

Images