Corn disease and pest monitoring method based on image processing

Abstract

The invention provides a corn disease and insect pest monitoring method based on image processing, and belongs to the technical field of application of a computer vision technology to plant disease prevention. The method specifically comprises the following steps: analyzing a corn image through an LSTM model, mining a change rule of a corn state along with time, predicting a lesion type appearing at the maximum probability when a certain change of plant appearance appears at a certain time, and combining an attention network, paying attention to a key time point and a key position to determine the change of the plant appearance. The bottleneck that multiple diseases and pests are disordered and difficult to distinguish in current corn disease and pest research is solved.

Description

Technical Field

[0001] This application relates to the technical field of applying computer vision technology to plant disease prevention, and specifically relates to a method for monitoring maize pests and diseases based on image processing. Background Art

[0002] The complexity and monitoring difficulties of maize pests and diseases lie in that maize is vulnerable to various pests and diseases during its growth cycle, and the growth stages and parts of maize affected by various pests and diseases are different. For example, head smut generally occurs in the heading stage and affects the female and male spikes, the Asian corn borer bores into the stem during the jointing stage of maize, and the adult worms damage the leaves during the heading stage. Maize root rot (caused by various pathogens) mainly occurs in the seedling stage and harms the roots. The above situation is already very complex. What's more complex is that the Asian corn borer and cotton bollworm generally damage maize in the larval stage, usually drilling into the maize stem and ear axis. Aphids, thrips, and maize spider mites generally damage maize in the adult stage. Aphids gather on the heart leaves, silk, and male spikes to feed, thrips suck the sap of maize heart leaves or leaves, causing punctate chlorosis of the leaves and the appearance of spots, etc. Maize spider mites suck the sap of the host leaf back tissue, and the affected leaves turn yellow and wither from white.

[0003] Currently, the discovery of maize pests and diseases mainly relies on the following methods, but all of them have serious deficiencies: (1) Manual field investigation, the disadvantages are: Low efficiency. Agricultural experts need to check the symptoms of pests and diseases plant by plant, which is time-consuming and laborious. Large-area real-time monitoring cannot be achieved. Since it depends on a certain expert, the diagnostic accuracy depends on the subjective experience of the expert, and it is easy to misjudge similar symptoms.

[0004] (2) Traditional image recognition technology. For example, a recognition algorithm based on CNN (such as ResNet, YOLO) with a static model is used to classify maize images in a maize field collected by a drone to identify pests and diseases. However, due to the high shooting distance of the drone and the difficulty in changing the shooting angle, the maize parts shown in the image are single. And the occurrence of maize pests and diseases is complex, and different pests and diseases affect different maize parts. Therefore, it is difficult to accurately judge the exact pests and diseases, and thus an effective prevention and control plan cannot be given.

[0005] (3) Sensor-assisted monitoring, the defect is that it is easily affected by the environment. Systems based on multi-spectral or thermal infrared sensors are easily affected by field light and humidity, and the false alarm rate is high. (4) Considering maize growth information in model calculation to predict the threat probability of pests and diseases in different growth stages. However, there are too many factors affecting maize pests and diseases. Only fusing growth information will give the model a single pointing information, resulting in poor generalization of the prediction model. It can be seen that the current methods for dealing with maize pests and diseases cannot efficiently discover the causes of plant diseases. Summary of the Invention

[0006] In view of the disadvantages of the above-mentioned related technologies, the present invention provides a method for monitoring maize diseases and pests based on image processing to solve the technical bottleneck encountered in image recognition of maize diseases and pests.

[0007] A method for monitoring maize diseases and pests based on image processing provided by this application includes: Using a drone to collect multiple consecutive overall images of maize plants; Collecting multiple consecutive partial images of maize plants through a fixed camera; Extracting the image features of the overall image of maize plants and the image features of the partial images of maize plants; Inputting the image features of the overall image of maize plants extracted in sequence into each memory unit of the long short-term memory network. Each memory unit of the long short-term memory network selects the image features and inputs them into the next memory unit, and outputs the predicted pest and disease categories.

[0008] Furthermore, before extracting the image features of the overall image of maize plants and the image features of the partial images of maize plants, the following processes can also be carried out: Using near-infrared to enhance the contrast between healthy tissues and damaged tissues in the image; Creating a binary mask to separate the vegetation and non-vegetation background pixel points in the image.

[0009] Furthermore, before extracting the image features of the overall image of maize plants and the image features of the partial images of maize plants, the following processes can also be carried out: Marking the damaged areas caused by accidental factors in the image as noise; Inputting the overall image of maize plants and the partial images of maize plants carrying the noise into the UNet network to generate a noise prediction map; Subtracting the noise prediction map from the overall image of maize plants and the partial images of maize plants without noise to obtain a difference image; Returning the difference image to the UNet network to generate the next round of noise prediction map and the next round of difference image until the difference image output by the UNet network for the Nth time approximates the real image, and updating the network weights through the backpropagation algorithm.

[0010] Furthermore, the method for monitoring maize diseases and pests based on image processing also includes: The overall map hidden states output by each memory unit that outputs the image features of the overall image of maize plants by the long short-term memory network; Inputting the image features of the partial images of maize plants extracted in sequence into each memory unit of the long short-term memory network. Each memory unit of the long short-term memory network selects the image features and inputs them into the next memory unit, and outputs the local map hidden states output by each memory unit; Input the image features of the overall corn plant image and the image features of the local corn plant image into the spatio-temporal attention network, and perform time-step weighting on the multi-channel features of the overall image; Then return the hidden state of the weighted overall image and the hidden state of the partial image to the last memory unit, and the last memory unit updates the output to predict the pest and disease category based on the input information.

[0011] Furthermore, the corn pest and disease monitoring method based on image processing further includes: The long short-term memory network outputs the hidden state of the overall image of the image features of the overall corn plant image output by each memory unit; Input the extracted image features of the local corn plant image into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features and inputs them into the next memory unit, and outputs the hidden state of the partial image output by each memory unit; Unfold the hidden state of the overall image and the hidden state of the partial image to obtain the multi-channel features of the overall image and the multi-channel features of the local image; The multi-channel features of the overall image and the multi-channel features of the local image are input into the spatio-temporal attention network to perform spatial weighting on the multi-channel features of the overall image; Then return the weighted multi-channel features to the last memory unit of the long-term memory network, and the last memory unit updates the output to predict the pest and disease category.

[0012] Furthermore, the steps of training the long short-term memory network include the process: Use a drone to collect multiple consecutive overall images of corn plants; Collect multiple consecutive local images of corn plants through a fixed camera; Label the overall images of corn plants and the local images of corn plants; Extract the image features of the overall corn plant image and the image features of the local corn plant image; Input the extracted image features of the overall corn plant image into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features and inputs them into the next memory unit, and outputs the predicted pest and disease category; Calculate the loss value between the predicted pest and disease category and the labeled pest and disease category, and adjust the model parameters until convergence.

[0013] Furthermore, the steps of training the long short-term memory network include: Use a drone to collect multiple consecutive overall images of corn plants; Collect multiple consecutive local images of corn plants through a fixed camera; Label the overall images of corn plants and the local images of corn plants; Extract the image features of the overall image of the corn plant and the image features of the local image of the corn plant; Input the image features of the overall image of the corn plant extracted in sequence into each memory unit of the long short-term memory network. Each memory unit of the long short-term memory network selects the image features and inputs them into the next memory unit, and outputs the predicted pest and disease categories; The long short-term memory network outputs the overall graph hidden states output by each memory unit of the image features of the overall image of the corn plant; Input the image features of the local image of the corn plant extracted in sequence into each memory unit of the long short-term memory network. Each memory unit of the long short-term memory network selects the image features and inputs them into the next memory unit, and outputs the local graph hidden states output by each memory unit; Input the image features of the overall image of the corn plant and the image features of the local image of the corn plant into the spatio-temporal attention network to perform time-step weighting on the multi-channel features of the overall graph; Unfold the overall graph hidden state and the local graph hidden state to obtain the multi-channel features of the overall graph and the multi-channel features of the local graph; Input the multi-channel features of the overall graph and the multi-channel features of the local graph into the spatio-temporal attention network to perform spatial weighting on the multi-channel features of the overall graph; Then return the time-weighted overall graph hidden state, the local graph hidden state, and the multi-channel features after spatial weighting to the last memory unit of the length memory network, and the last memory unit updates and outputs the predicted pest and disease categories; Calculate the loss value between the updated predicted pest and disease categories and the labeled pest and disease categories, and adjust the model parameters until convergence.

[0014] As described above, the beneficial effects of the present invention are as follows: By setting the calculation of the long short-term memory network, the evolution law of pests and diseases suffered during the corn growth stage is mined, simplified, and various factors causing corn lesions are converted into the state evolution of corn over growth time. By identifying the morphological changes, color changes, and shape changes of each part of the plant, the types of pests and diseases that have occurred are predicted.

[0015] And the spatio-temporal attention network is combined to optimize the information of the long short-term memory network, and the growth information of pests and diseases causing diseases is fused on the time line of corn growth. For the pest periods that will cause corn lesions, the hidden information of the corresponding memory units at each time step of the long short-term memory network is weighted, and the model pays more attention to the outbreak periods of special pests, etc., so as to predict more accurate pest and disease categories for a specified time. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the steps of the present invention for monitoring corn pests and diseases based on image processing; Figure 2 It is a schematic diagram of the long short-term memory network applied in an example of the present invention; Figure 3 The example diagram of the maize pest and disease classification network structure established based on the long short-term memory network is shown. Detailed implementation manners

[0017] The following will further illustrate the present invention with reference to the accompanying drawings and preferred embodiments.

[0018] Maize is one of the important food crops globally, but it is often attacked by various pests and diseases during its growth process, seriously affecting yield and quality.

[0019] With the advancement of the agricultural modernization process, intelligent monitoring technology plays an increasingly important role in the field of maize pest and disease control. Traditional manual field investigation methods are inefficient and subjective, while chemical control brings environmental pollution and drug resistance problems.

[0020] Intelligent monitoring technology provides a reliable solution for the early identification, precise warning and scientific control of maize pests and diseases by integrating a variety of advanced technical means.

[0021] Current maize pest and disease intelligent monitoring technology combines drones, computer vision technology, remote sensing technology, infrared technology, etc., and realizes pest and disease identification and positioning through deep learning models (such as improved ResNet, YOLO series). Combining multi-spectral / hyperspectral imaging can detect early latent diseases; remote sensing technology uses drones and satellites to obtain multi-scale data in the field and cooperates with the Internet of Things sensor network to monitor environmental parameters in real time; big data and artificial intelligence technology analyze time series data through models such as graph neural networks to establish a prediction and warning system. The integrated application of these technologies significantly improves the monitoring efficiency, can detect pests and diseases earlier than manual inspection, and has advantages such as reducing the amount of pesticide used.

[0022] However, due to the wide variety of maize pests and diseases and the complex and changeable ways in which different pests and diseases attack maize, the current intelligent monitoring of maize pests and diseases lacks in-depth research on the internal relationships of maize pests and diseases. It directly uses computer vision technology to fuse various features of big data, analyzes the maize image features to classify maize pests and diseases, and the model calculation process simply relies on features, without exploring the internal relationships of various features, and has a low efficiency in capturing key features, resulting in low model robustness.

[0023] The method proposed by the present invention can effectively solve the above problems. Referring to Figure 1 , a maize pest and disease monitoring method based on image processing, the method includes the following steps: S1: Use a drone to collect multiple consecutive overall images of maize plants.

[0024] For example, it can be for the entire growth stage of diseased plants, such as from the start of cultivation to the growth process, to being subjected to a certain infestation, and then to the growth process after treatment. Image a1 is collected at time T1, image a2 is collected at time T2, and image a is collected at time T n and so on. n And so on.

[0025] S2: Collect multiple consecutive partial images of corn plants through a fixed camera.

[0026] For example, it can be that image b1 is collected at time T1, image b2 is collected at time T2, and image b is collected at time T n and so on. n And so on.

[0027] S3: Label the overall image of the corn plant and the partial images of the corn plant.

[0028] The content of the label is the growth stage of the corn, the type of pests and diseases, the damaged part, the time when pests and diseases appear, and the time node of special pest infestations. Then, a relationship label is marked on the partial image of the corn plant corresponding to the damaged part.

[0029] For example, for thrips, the damage to the corn leaf is the leaf showing punctate chlorosis, and it will also damage the heart leaf. Then, a concern label for the heart leaf is marked in the image area of the leaf showing punctate chlorosis.

[0030] If there is a special infestation of pests on the corn plant when the image is collected, the special infestation of pests is also labeled. For example: If the corn plant is infested by cotton bollworm larvae when the image is collected, then a special infestation label for the outbreak of cotton bollworm larvae is marked.

[0031] The types of pests and diseases are not limited to corn borer, cotton boll, aphid, thrips, corn spider mite, smut, corn root rot, etc. Sometimes, the corn plant will also be infested by a combination of several pests and diseases.

[0032] For example, cotton bollworm generally causes damage to the corn plant by its larvae. Therefore, if the time when cotton bollworm larvae appear on the corn plant can be regarded as the time when special pests and diseases appear.

[0033] In the training stage, the overall images of the corn plants and the partial images of the corn plants collected and labeled are used as training samples. In the application stage, step S3 is not executed, and the overall images of the corn plants and the partial images of the corn plants collected are directly input into the network for image processing.

[0034] For example, during the training phase, 6000 overall images of corn plants were collected, along with 100 images each of corn borer infestations, cotton bolls, aphids, thrips, and spider mites on corn leaves. Among them, 200 images were used as test samples. Additionally, 2400 overall images of corn plants were collected, along with 40 images each of corn borer infestations, cotton bolls, aphids, thrips, and spider mites on corn leaves. Among them, 120 images were used as test samples, and the long short-term memory network was trained.

[0035] S4: Extract the image features (x1, x2... x n ) of the overall image of the corn plant and the image features (y1, y2... y n ) of the local image of the corn plant.

[0036] Feature extraction can be performed using networks such as ResNet and VGG. X1 corresponds to the feature set extracted from the first local image of the corn plant, and Y n represents the feature set extracted from the nth local image of the corn plant, and so on.

[0037] Before extracting the image features of the overall image of the corn plant, or before extracting the image features of the local image of the corn plant, the following steps can also be carried out: S11: Use near-infrared (wavelength 700 - 1000nm) to enhance the contrast between healthy and damaged tissues in the image.

[0038] S12: Create a binary mask using NDVI to separate the vegetation and non-vegetation background pixel points in the image.

[0039] NDVI is the abbreviation of Normalized Difference Vegetation Index.

[0040] Pests and diseases damage the corn plant, causing changes in the state, shape, and color of some parts or regions. However, these changes in the corn plant may not necessarily be caused by pests and diseases, and may be due to accidental factors such as drought, animal bites, and extreme weather.

[0041] For the collected overall images and local images of the corn plant, the following processing procedures can also be performed: S21: Mark the damage areas caused by accidental factors in the image and record them as noise.

[0042] S22: Input the overall image and local image of the corn plant carrying this noise into the UNet network to generate a noise prediction map.

[0043] S23: Subtract the noise prediction map from the overall image and local image of the corn plant without noise to obtain a differential image.

[0044] S24: Return the difference image to the UNet network, and continue with steps S22 - S23 until the output difference image approximates the real image, and update the network weights through the backpropagation algorithm.

[0045] For example, calculate the SSIM value between the difference image and the overall image of the corn plants with noise removed manually. When the SSIM value is less than or close to a certain value, it is considered that the two images are infinitely approaching. SSIM (Structural Similarity Index) is a metric for measuring image quality and is widely used in fields such as image processing, computer vision, and image compression.

[0046] S25: Denoise the images collected in steps S1 and S2 through the adjusted UNet network.

[0047] S5: Input the image features of the overall image of the corn plants extracted in step S4 into each memory unit of the long short - term memory network in sequence. Each memory unit of the long short - term memory network selects the image features and inputs them into the next memory unit, and outputs the predicted pest and disease category.

[0048] Figure 2 It is a schematic diagram of the long short - term memory network applied in an example of this application. As Figure 2 shown, the long short - term memory network includes several memory units, memory unit A, memory unit B, memory unit C, and each memory unit inputs the features corresponding to the time.

[0049] Taking memory unit B as an example, the x in (x1, x2...x n ) is input into memory unit B, and the calculation process can be regarded as: t K11: Combine the hidden information h of the previous memory unit A, and control the forgetting of historical information through formula (1). It can be regarded as screening out the information in the corn growth stage information that has nothing to do with suffering from pests and diseases at the current time and forgetting it. t-1

[0050] (1), where in formula (1), is the weight matrix of the forgetting gate, is the bias term of the forgetting gate, and is the activation function.

[0051] K12: Combine the hidden information h t-1 of the previous memory unit A, and generate supplementary memory through formula (2) and formula (3), and calculate the relevant information about the corn suffering from pests and diseases at the current time based on the corn growth stage.

[0052] ​(2) to store the information of the image features collected at time t in formula (2), is the weight matrix of the input gate, is the bias term of the input gate, is the activation function.

[0053] = tanh (3), The candidate state of the corn is obtained by combining the feature information extracted from the current image and the historical information of the corn. In formula (3), is the correlation matrix for generating the candidate state, is the corresponding bias term.

[0054] K13: Combine the candidate state obtained after memory supplementation based on the current input features with the corn pest state at time t - 1 output by the previous memory cell A, and update the cell state of the current memory cell through formula (4) , in the present invention, the cell state represents the corn pest state and stores the temporal pattern of the evolution of pests and diseases. The corn pest state can obtain information through computer analysis: whether the corn plant has suffered from pests and diseases, the degree and type of pests and diseases suffered according to the evolution law of pests and diseases at the current moment.

[0055] = * + * (4); represents the cell state of the previous memory cell.

[0056] K14: Process the corn pest state through the output gate. Based on the state information of the corn at time t, the output gate filters out more accurate information related to the output prediction for prediction calculation. The relevant formulas for the output gate are formulas (5) and (6).

[0057] = (5); (6).

[0058] is the hidden state of memory cell B, used for prediction calculation or passed to the next memory cell, is the matrix of the output gate, is the bias term of the output gate.

[0059] K15: Take the hidden state of the memory unit corresponding to the time step to be predicted for classification calculation to predict the category of the pest and disease. Generally, the memory unit corresponding to the time step to be predicted is the last memory unit.

[0060] Z = Softmax( ), is the weight of the classification layer, is the bias term of the classification layer.

[0061] During the model training process, after predicting the category of the pest and disease through the long short-term memory network, the following steps can also be executed to calculate the loss value between the predicted category of the pest and disease and the labeled category of the pest and disease, and adjust the model parameters until convergence.

[0062] For example, the cross-entropy loss function can be used to calculate the cross-entropy loss value between the predicted category and the labeled category, and the parameters of the model can be updated through the backpropagation algorithm to minimize the loss value, and then optimization algorithms such as gradient descent can be used to adjust the parameters of the model.

[0063] During the model application process, step S3 is not executed. After collecting the image, step S4 is executed to extract the image features and subsequent steps to predict the type of the pest and disease.

[0064] If a multi-channel feature map is extracted by the CNN network when extracting features in S4, and during the labeling process, relationship labels are marked on the local image of the corn plant corresponding to the damaged part, some channels of the multi-channel feature map store information of special regions, and the special regions are associated with the local image of the corn region.

[0065] During the labeling process, special pest infestations are also marked, then some other channels of the multi-channel feature map store information of the special infestations, such as the channel information corresponding to the morphological change regions caused by the leaf edges and larvae.

[0066] Then, using the above characteristics, the hidden state output by the long short-term memory network can be unfolded to restore a multi-channel state map, which carries information of different channels, that is, spatial information, and spatial weighting processing can be performed; at the same time, because the hidden state carries time information of different memory units, time weighting processing can be performed. Therefore, after the long short-term memory network of the present invention outputs the type of the pest and disease, the following steps of the spatio-temporal memory network calculation can be continued to optimize the entire output content: Executing step S5 can also output the overall graph hidden states (h1, h2... h t-1 , h t ... h n ) output by each memory unit.

[0067] S6: Input the image features of the partial images of corn plants extracted in step S4 into each memory unit of the long short-term memory network in sequence. Each memory unit of the long short-term memory network selects the image features and inputs them into the next memory unit, and outputs the hidden states of the partial images (h`1, h`2... h` t-1 、h` t ……h` n ).

[0068] S7: Unfold the hidden state of the overall image and the hidden state of the partial image to obtain the multi-channel features of the overall image and the multi-channel features of the partial image.

[0069] S8: Input the image features of the hidden state of the overall image and the hidden state of the partial image into the spatio-temporal attention network, and perform time-step weighting on the multi-channel features of the overall image to focus on the key time points; that is, weight the channel features corresponding to the special infestation time of special pest infestations in the labels to pay attention to the information stored in the hidden state or cell state corresponding to the special infestation time of the pest.

[0070] For example, if the larvae of Helicoverpa armigera cause damage to corn plants, then the time when the larvae of Helicoverpa armigera appear is the key time point marked. The original growth of the plant over time and the growth of a certain insect are two independent time lines. The present invention collects images according to the time sequence of corn growth of the plant, sets the time line of the memory units of the long short-term memory network, and then marks the larvae that damage the corn on the time sequence of corn growth. Through the associative learning of features by the long short-term memory network on the time sequence of corn growth, the time line of the evolution of Helicoverpa armigera is implanted into the time sequence of corn growth, and through training and learning by the long short-term memory network, it is predicted that the larvae of Helicoverpa armigera will appear in a certain period of corn growth.

[0071] The spatio-temporal attention network can perform time-step weighting on the hidden state through the following formulas (7) and (8): (7); (8).

[0072] =(h1, h2... h t-1 、h t ……h n ) or (h`1, h`2... h` t-1 、h` t ……h` n ), As the input of the attention mechanism, through linear transformation The hidden layer representation on the time sequence features in the attention network. Successively weight each element of , Time steps with large values are the key time points when the model has learned and discovered that pests start to spread and develop, or when environmental conditions (temperature, humidity) are suitable for the development of pests and diseases.

[0073] , is the learnable weight, is the bias term. is the hidden feature after time attention weighting.

[0074] S9: Input the multi-channel features of the overall graph and the multi-channel features of the local graph into the spatio-temporal attention network, and perform spatial weighting on the multi-channel features of the overall graph, that is, weight the channel features corresponding to the special damage and damaged parts where pests appear in the label.

[0075] S8 and S9 are executed in parallel.

[0076] Reshape the hidden sequence to obtain the spatial feature map F, with dimensions W*H*C, where C represents the number of channels, W represents the image width, and H represents the image height. Calculate F-avg and F-max. F-avg is the mean calculated along the channel dimension, reflecting the global importance of each spatial position. The high-value regions may correspond to continuously developing lesions, such as the continuous expansion of rust for several days.

[0077] F-max calculates the maximum value along the channel dimension to capture local significant features. For example, it may be the heart leaf area corresponding to thrips infestation and the leaf punctate chlorosis area.

[0078] Perform channel concatenation calculation [F-avg, F-max], and perform convolution kernel calculation on the concatenated result. For example, dilated convolution can be used for smaller boreholes such as those caused by the Asian corn borer to expand the receptive field. Use a larger-sized convolution kernel to capture the transition region between leaf spot disease and healthy tissue.

[0079] The above calculation formulas can be represented by formula (9) and formula (10).

[0080] (9); (10); represents the convolution operation, is the activation function, represents the channel concatenation calculation.

[0081] S10: Then return the weighted multi-channel features to the last memory unit of the length memory network, and the last memory unit updates and outputs the predicted pest and disease categories.

[0082] S11: Then, the weighted overall graph hidden state and sub-graph hidden state are used, and finally the memory unit is updated to output the predicted pest and disease category.

[0083] S12: Then, the weighted multi-channel features, overall graph hidden state and sub-graph hidden state are used, and finally the memory unit is updated to output the predicted pest and disease category.

[0084] S10 can be executed, or S11 or S12 can be executed.

[0085] Take Return the last memory unit of the length memory network, perform the calculation process of the memory unit to obtain a new hidden state, and then perform classification calculation based on the new hidden state to predict the pest and disease category.

[0086] For example, referring to Figure 3 the example diagram of the maize pest and disease classification network structure established based on the long short-term memory network shown above, the long short-term memory network used in the above process has, on the original basis, a feature extraction network based on VGG added to the head of the long short-term memory network, and a classification network and a spatio-temporal attention network based on CNN convolution added to the tail of the long short-term memory network.

[0087] During the model training process, step S5 can also be executed to output the overall graph hidden state, and S6 - S12 can be executed to update the output to predict the pest and disease category, calculate the loss value based on the updated predicted pest and disease category, and adjust the model parameters.

Claims

1. A method for monitoring corn diseases and pests based on image processing, characterized in that, Including methods: Using a drone to collect multiple consecutive overall images of corn plants; Collecting multiple consecutive partial images of corn plants through a fixed camera; Extracting the image features of the overall images of corn plants and the image features of the partial images of corn plants; Sequentially inputting the extracted image features of the overall images of corn plants into each memory unit of a long short-term memory network. Each memory unit of the long short-term memory network selects the image features and inputs them into the next memory unit, and outputs the predicted pest and disease categories.

2. The method according to claim 1, wherein Before extracting the image features of the overall images of corn plants and the image features of the partial images of corn plants, the following processes can also be carried out: Using near-infrared to enhance the contrast between healthy tissues and damaged tissues in the image; Creating a binary mask to separate the vegetation and non-vegetation background pixel points in the image.

3. The method according to claim 1, wherein Before extracting the image features of the overall images of corn plants and the image features of the partial images of corn plants, the following processes can also be carried out: Marking the damaged areas caused by accidental factors in the image as noise; Inputting the overall images of corn plants and the partial images of corn plants carrying the noise into a UNet network to generate a noise prediction map; Subtracting the noise prediction map from the overall images of corn plants and the partial images of corn plants without noise to obtain a difference image; Returning the difference image to the UNet network to generate the next round of noise prediction map and the next round of difference image until the difference image output by the UNet network for the Nth time approximates the real image, and updating the network weights through the backpropagation algorithm.

4. The method according to claim 1, wherein The method for monitoring corn pests and diseases based on image processing also includes: The overall map hidden states output by each memory unit of the long short-term memory network for the image features of the overall images of corn plants; Sequentially inputting the extracted image features of the partial images of corn plants into each memory unit of the long short-term memory network. Each memory unit of the long short-term memory network selects the image features and inputs them into the next memory unit, and outputs the local map hidden states output by each memory unit; Inputting the image features of the overall images of corn plants and the image features of the partial images of corn plants into a spatio-temporal attention network to perform time-step weighting on the multi-channel features of the overall map; Then returning the weighted overall map hidden state and the local map hidden state to the last memory unit, and the last memory unit updates the output to predict the pest and disease categories based on the input information.

5. The method according to claim 1, wherein The method for monitoring corn pests and diseases based on image processing also includes: The overall map hidden states output by each memory unit of the long short-term memory network for the image features of the overall images of corn plants; Sequentially inputting the extracted image features of the partial images of corn plants into each memory unit of the long short-term memory network. Each memory unit of the long short-term memory network selects the image features and inputs them into the next memory unit, and outputs the local map hidden states output by each memory unit; Unfolding the overall map hidden state and the local map hidden state to obtain the multi-channel features of the overall map and the multi-channel features of the local map; Inputting the multi-channel features of the overall map and the multi-channel features of the local map into a spatio-temporal attention network to perform spatial weighting on the multi-channel features of the overall map; Then, the weighted multi-channel features are returned to the last memory cell of the length memory network, and the last memory cell updates and outputs the predicted pest and disease category.

6. The method according to claim 1, wherein The steps of training the long short-term memory network include the process: Use a drone to collect multiple consecutive overall images of corn plants; Collect multiple consecutive partial images of corn plants through a fixed camera; Label the overall images of corn plants and the partial images of corn plants; Extract the image features of the overall images of corn plants and the image features of the partial images of corn plants; Input the image features of the overall images of corn plants extracted in sequence into each memory cell of the long short-term memory network. Each memory cell of the long short-term memory network selects the image features and inputs them into the next memory cell, and outputs the predicted pest and disease category; Calculate the loss value between the predicted pest and disease category and the labeled pest and disease category, and adjust the model parameters until convergence.

7. The method according to claim 1, characterized in that The steps of training the long short-term memory network include: Use a drone to collect multiple consecutive overall images of corn plants; Collect multiple consecutive partial images of corn plants through a fixed camera; Label the overall images of corn plants and the partial images of corn plants; Extract the image features of the overall images of corn plants and the image features of the partial images of corn plants; Input the image features of the overall images of corn plants extracted in sequence into each memory cell of the long short-term memory network. Each memory cell of the long short-term memory network selects the image features and inputs them into the next memory cell, and outputs the predicted pest and disease category; The long short-term memory network outputs the overall map hidden states output by each memory cell of the image features of the overall images of corn plants; Input the image features of the partial images of corn plants extracted in sequence into each memory cell of the long short-term memory network. Each memory cell of the long short-term memory network selects the image features and inputs them into the next memory cell, and outputs the partial map hidden states output by each memory cell; Input the image features of the overall images of corn plants and the image features of the partial images of corn plants into the spatio-temporal attention network to weight the multi-channel features of the overall map in time steps; Unfold the overall map hidden state and the partial map hidden state to obtain the multi-channel features of the overall map and the multi-channel features of the partial map; Input the multi-channel features of the overall map and the multi-channel features of the partial map into the spatio-temporal attention network to weight the multi-channel features of the overall map in space; Then, return the time-weighted overall map hidden state, the partial map hidden state, and the space-weighted multi-channel features to the last memory cell of the length memory network, and the last memory cell updates and outputs the predicted pest and disease category; Calculate the loss value between the updated predicted pest and disease category and the labeled pest and disease category, and adjust the model parameters until convergence.

Images