A method and system for predicting the spread of leaf spot on a plant

Abstract

The present application belongs to the field of plant disease prevention and treatment, and discloses a plant leaf spot diffusion prediction method and system based on a Markov model and a long short-term memory network, which comprises the following steps: continuously collecting grape leaf spot diffusion process image series, extracting leaf surface spot features related to each picture, constructing the features extracted from the series of images into a time sequence, training an LSTM prediction model so that it can predict subsequent spot feature regions, converting test pictures into gray scale images according to leaf surface features, regarding the changes of pixels of each leaf surface gray scale image as a discrete Markov process, calculating a probability transition matrix of the next step through leaf surface pixel features predicted by the LSTM network, calculating the next leaf surface gray scale image by using a discrete Markov model, and obtaining a final diffusion prediction image through image adjustment. The present application can predict and simulate the diffusion process of leaf surface diseases to a certain extent, the model has generalization, and is easy to popularize to spot diffusion research of other plant diseases.

Description

Technical Field

[0001] This invention belongs to, but is not limited to, the field of plant disease control technology, and particularly relates to a method and system for predicting the spread of plant leaf spots based on Markov models and long and short memory networks. Background Technology

[0002] Spots on plant leaves are often caused by pests and diseases, and the spread of spots is accompanied by leaf wilting. Therefore, predicting the speed and trend of leaf spot spread is helpful for studying the spread of plant diseases and has a certain auxiliary role in the research on disease prevention and control of agricultural and forestry crops.

[0003] This invention is a further expansion and development of leaf spot segmentation technology, based on leaf spot segmentation, and is intended for future application in the prevention and control of plant diseases and pests.

[0004] Related patents: A Transformer-based image segmentation method for grape leaf lesions; A method and system for classifying mature spots and lesions in flue-cured tobacco based on texture and shape features; A grape lesion detection method based on convolutional networks;

[0005] The aforementioned patents mainly focus on lesion segmentation, primarily studying lesion segmentation in images of different crops for further disease identification, without addressing lesion spread or providing predictive research. This invention, however, focuses on the spread of crop lesions, studying the speed and manner of lesion diffusion. Therefore, this invention has significant innovations, representing substantial improvements and innovations compared to existing patents.

[0006] Simulating the diffusion of spots on leaf images essentially studies the process by which healthy pixels in an image are converted into spot pixels. Through observation of a series of acquired images, some patterns in the diffusion of leaf spots were discovered:

[0007] (1) The degree of leaf spot lesions varies. They start as light-colored spots and then gradually darken. For example, on grape leaves, the spots first turn light yellow, then yellow, and finally brown until they wither.

[0008] (2) The pattern of spot diffusion is mainly based on the expansion of existing spots. That is, healthy areas adjacent to spot areas are more likely to be converted into spot areas, and some healthy areas may directly be converted into spot areas.

[0009] (3) As it is necessary to collect images of multiple leaf surfaces over several consecutive days, various uncertain factors affect the image. The angle, lighting, and distance of the lens vary each time, resulting in different sizes of the same leaf surface when photographed on different days, which poses a challenge for subsequent analysis.

[0010] Based on the above analysis, the urgent technical problems that need to be solved in the existing technology are:

[0011] Simulating the spread of spots on leaf images requires addressing issues such as varying degrees of leaf spot lesions, the increased susceptibility of healthy pixels adjacent to the spotted areas to infection, and the inconsistent size of the same leaf surface at different times due to shooting angles and distances. Summary of the Invention

[0012] To address the problems existing in the prior art, this invention provides a method and system for predicting the spread of plant leaf spots based on Markov models and long short-term memory networks.

[0013] This invention is implemented as follows: a method for predicting the spread of plant leaf spots based on Markov models and long short-term memory networks, characterized in that the method specifically includes:

[0014] S1: Image acquisition and preprocessing: Continuously acquire a series of images of the grape leaf spot diffusion process, extract leaf spot features related to each image, and construct a time series from the features extracted from the series of images.

[0015] S2: LSTM prediction model training: The LSTM model is trained using features extracted from leaf spot patterns in sequential images, enabling it to predict subsequent spot feature regions.

[0016] S3: Spot diffusion prediction based on discrete Markov chain. The test image is converted into a grayscale image according to the leaf surface features. The pixel change of each leaf surface grayscale image is regarded as a discrete Markov process. The leaf surface pixel features are predicted by the LSTM network and the probability transition matrix of the next step is calculated. The next leaf surface grayscale image is calculated by the discrete Markov model. Finally, the diffusion prediction image is obtained by image adjustment.

[0017] Furthermore, in S1, images of the same plant leaf with spots are periodically collected, for example, every 2-3 days, until the leaf surface withers or falls off over a large area.

[0018] Further, in S1, the collected leaf images are processed. The main steps of each image processing include background removal, lesion segmentation, image grayscale conversion, disease severity grading, and feature extraction. For a series of images with a disease cycle, each type of feature can be extracted once from each image to form a time series of features, which is used as training data for a time series model.

[0019] (1) Background removal was performed using 360 AI tools;

[0020] (2) Leaf spot segmentation is performed using a lesion segmentation algorithm (a mature algorithm).

[0021] (3) Classify the severity of spot color based on the k-means clustering algorithm;

[0022] (4) Convert the graded leaf images into grayscale images;

[0023] (5) Hp% represents the proportion of healthy pixel areas to the entire leaf surface, Sp1% represents the proportion of slightly spotted pixels to the leaf surface area, Sp2% represents the proportion of moderately spotted pixels, and Sp3% represents the proportion of severely spotted pixels. The calculation formula is as follows:

[0024]

[0025]

[0026]

[0027]

[0028] Among them, the above features satisfy the following constraints:

[0029] Hp%+Sp1%+Sp2%+Sp3%=1

[0030] Right now:

[0031] Hp%=1-Sp1%-Sp2%-Sp3%

[0032] Furthermore, in S2, an LSTM model is selected to predict the leaf spot feature values ​​of the next state. This model needs to be trained before it can be used for prediction. This invention uses three long short-term memory networks.

[0033] Furthermore, in S3, the prediction method involves first using the trained LSTM model to predict the leaf surface features of the next state one step at a time, comparing the predicted value with the current value, calculating the transition probability of various pixels on the leaf surface, then using a Markov model to calculate the probability of each pixel, and finally adjusting based on the calculation results to obtain the predicted grayscale image.

[0034] (1) Discrete Markov chain: Assume that a pixel on a leaf surface is in a state X at a certain moment. i Then its sequence over the entire period can be regarded as the following set of states: X = {X1, X2, ..., X} t ,X t+1 …}, whose conditional probability distribution p(X) t+1 |X t The probability p is called the state transition probability. Suppose a random variable changes from state 0 to state k, and at time t-1 it is in state j. Then the probability that it is in state i at time t is p. ij =(X t =i|Xt-1 If j), then the transition probability matrix can be set as follows:

[0035]

[0036] ST

[0037]

[0038] Discrete Markov state changes can be written in the following matrix form:

[0039] X t+1 =PX t

[0040] (2) Setting the leaf pixel transition probability matrix and solving the transition probability: Each pixel in the leaf spot diffusion model is regarded as a Markov process. Assuming that the leaf pixels in the grayscale model image have the following states: State 0: healthy state, State 1: mild spot, State 2: moderate spot, State 3: severe spot, p ij Let represent the probability that a pixel transitions from state j to state i. Then the probability transition matrix is ​​as follows:

[0041]

[0042] The probability transition matrix of the leaf pixels can be simplified to:

[0043]

[0044] Since the sum of each row of the probability transition matrix is ​​1, then:

[0045] p 22 =1-p 32

[0046] p 11 =1-p 21

[0047] p 00 =1-p 10

[0048] According to the formula, we only need to find p. 10 ,p 21 ,p 32 Then the probability transition matrix P can be determined. These three probability values ​​can be obtained by statistically analyzing the area of ​​each region from time t-1 to time t. That is, the transition probability of each state of the pixel can be calculated using the following method:

[0049] set up The areas of healthy, mildly spotted, moderately spotted, and severely spotted leaf surfaces at time t-1 are respectively. To predict the corresponding region surface at time t using the LSTM network, we have:

[0050]

[0051]

[0052]

[0053] Where, p 10 This represents the probability that a healthy pixel will become a mild blemish. It consists of two parts: the probability that a neighboring pixel will become a blemish pixel, denoted as p. 1X Second, the probability of a non-adjacent healthy pixel being converted into a speckle pixel, denoted as p. 1Y Then we have:

[0054] p 01 =p 1X +p 1Y

[0055] An empirical ratio is set for the dataset, assuming that the probability ratio of adjacent pixels to non-adjacent pixels turning into blob regions is 1:9, that is:

[0056] p 1X :p 1Y =9:1

[0057] have:

[0058] p 1X =0.9*p 10

[0059] p 1Y =0.1*p 10

[0060] At this point, the transition probabilities of all types of leaf surface pixels have been determined;

[0061] (3) Calculation and adjustment of leaf spot spread: By multiplying each pixel by the probability transfer matrix, the healthy pixel Hp is divided into adjacent pixels and non-adjacent pixels, and it is considered that it has a probability of being transferred to spot pixels. The transfer probability is calculated by classification. For spot pixels of different degrees, the relevant probability values ​​are used for prediction.

[0062] (4) Image adjustment: If a healthy pixel is surrounded by lesion pixels on three adjacent sides, it is adjusted to be a lesion pixel of the same level. If a spot pixel is surrounded by spot pixels of a higher level on three adjacent sides, it is adjusted to be a spot pixel of a higher level. After the above adjustment, the spot area becomes a connected area, which is more in line with common sense.

[0063] Another objective of this invention is to provide a plant leaf spot diffusion prediction system based on Markov models and long short-term memory networks, the system specifically comprising:

[0064] The image acquisition module periodically acquires images of the same plant leaf surface with spots;

[0065] The preprocessing module performs preprocessing on the acquired images, such as background removal and spot segmentation, and extracts the proportion of spots at each level on the leaf surface. By repeating the above preprocessing operations on leaf surface images acquired at different times, the temporal series features of each level of spots are obtained.

[0066] The LSTM prediction model training module uses extracted time series features to train the LSTM network, enabling it to predict the subsequent diffusion ratio of different types of blob models.

[0067] The speckle diffusion prediction module predicts speckle diffusion based on discrete Markov chains.

[0068] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0069] First, this invention can predict and simulate the spread of leaf lesions to a certain extent, with characteristics such as the proportion of lesion areas and the proportion of severely lesion areas showing a high degree of consistency with real data. Furthermore, the model has generalizability and can be easily extended to the study of lesion spread in other plant diseases.

[0070] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0071] This invention proposes a method for predicting the spread of leaf spots on plants. Theoretically, it can predict and simulate the subsequent spread of leaf spots or lesions on any plant, and analyze and predict the spread rate of different plant diseases. The expected benefits are reflected in the prediction of diseases in agricultural and forestry crops. Based on the occurrence of diseases, it can predict the subsequent spread rate of diseases, allowing for early prevention and control measures to reduce disease losses.

[0072] The commercial value of this invention lies in its ability to assess the effectiveness of pesticides. If a pesticide can prevent pests and diseases, its efficacy can be compared to the predicted effect. If the pesticide is effective, the rate of lesion spread should be significantly slower than predicted, thus proving the pesticide's effectiveness. Commercial application of this invention provides valuable reference for pesticide effectiveness testing.

[0073] (2) The technical solution of this invention fills a technical gap in the industry both domestically and internationally:

[0074] Based on the search and analysis, no relevant agricultural and forestry technologies have been found in China. These technologies focus on the segmentation of leaf spots and disease identification. There are no related patents that study the prediction of spot or lesion spread. This indicates that there are currently no domestic patents that use LSTM networks and Markov models to predict the spread of leaf spots. This technology is still in its infancy in China.

[0075] Secondly, the technical solution of this invention solves several problems in the prediction of leaf spot diffusion in existing technologies and achieves significant technological progress. The following is a detailed description:

[0076] Technical problems to be solved:

[0077] 1) Insufficient prediction accuracy: Traditional plant disease prediction methods are often based on empirical models or simple statistical analysis. These methods have limited prediction accuracy when dealing with complex and nonlinear leaf spot diffusion processes. This invention improves the ability to capture complex diffusion patterns by combining Markov models and Long Short-Term Memory (LSTM) networks, thereby enhancing prediction accuracy.

[0078] 2) Low data processing efficiency: Traditional prediction methods are often inefficient when processing large amounts of image data, failing to meet the needs of real-time monitoring and rapid response. The technical solution of this invention significantly improves data processing efficiency through efficient image processing and feature extraction processes, as well as utilizing the rapid learning capability of the LSTM model.

[0079] 3) Lack of dynamic prediction capability: Existing technologies often only provide static disease analysis and are difficult to effectively predict the dynamic spread of leaf spots. This invention achieves dynamic prediction of leaf spot spread by simulating pixel state changes using Markov chains.

[0080] Technological advancements achieved:

[0081] 1) Improved prediction accuracy: By combining LSTM and Markov models, this invention can more accurately capture the complex patterns and trends of leaf spot spread, thus providing more reliable prediction results. This is crucial for developing effective disease control strategies.

[0082] 2) Enhanced real-time performance: The technical solution of this invention can efficiently process and analyze large amounts of image data, providing real-time prediction of leaf spot spread. This helps to detect and respond to plant diseases in a timely manner, reducing losses.

[0083] 3) Dynamic Prediction and Monitoring: By simulating changes in leaf pixel states using Markov chains, this invention enables dynamic prediction and monitoring of the leaf spot diffusion process. This helps to more accurately understand the development of diseases and provides decision-makers with more valuable information.

[0084] 4) The method has strong universality: The technical solution of this invention is not only applicable to the prediction of the spread of spots on grape leaves, but can also be extended to the prediction and monitoring of other plant diseases. This demonstrates the broad application prospects of this invention in the field of plant protection.

[0085] In summary, the technical solution of the present invention has achieved significant technological progress by solving the key problems of the prior art, and provides a more accurate, efficient and dynamic method for predicting the spread of leaf spots on plants. Attached Figure Description

[0086] Figure 1 This is a flowchart of a plant leaf spot diffusion prediction method based on Markov model and long short-term memory network provided in an embodiment of the present invention.

[0087] Figure 2 This is a specific memory unit structure diagram of the LSTM model provided in the embodiments of the present invention;

[0088] Figure 3 This is a block diagram of a plant leaf spot diffusion prediction system based on Markov models and long short-term memory networks provided in an embodiment of the present invention.

[0089] Figure 4 This is a schematic diagram of leaf segmentation provided in an embodiment of the present invention.

[0090] Figure 5 This is a schematic diagram of grape leaf spot diffusion prediction provided in an embodiment of the present invention. Detailed Implementation

[0091] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0092] like Figure 1 As shown, this embodiment of the invention provides a method for predicting the spread of leaf spots in plants based on Markov models and long short-term memory networks. The method specifically includes:

[0093] S1: Image acquisition and preprocessing: Continuously acquire a series of images of the grape leaf spot diffusion process, extract leaf spot features related to each image, and construct a time series from the features extracted from the series of images.

[0094] S2: LSTM (Long Short-Term Memory) prediction model training uses features extracted from leaf spot patterns in sequential images to train the LSTM model, enabling it to predict subsequent spot feature regions;

[0095] S3: Spot diffusion prediction based on discrete Markov chain. The test image is converted into a grayscale image according to the leaf surface features. The pixel change of each leaf surface grayscale image is regarded as a discrete Markov process. The leaf surface pixel features are predicted by the LSTM network and the probability transition matrix of the next step is calculated. The next leaf surface grayscale image is calculated by the discrete Markov model. Finally, the diffusion prediction image is obtained by image adjustment.

[0096] In step S1, images of the same plant leaf with spots are periodically collected, for example, every 2-3 days, until a large area of ​​the leaf withers or falls off. The collected leaf images are processed, with each image processing step including background removal, lesion segmentation, image grayscale conversion, disease severity grading, and feature extraction. For a series of images with a disease cycle, each type of feature can be extracted once from each image to form a time series of features, which is used as training data for a time series model.

[0097] 1. Background removal was performed using 360AI artificial intelligence tools.

[0098] 2. Leaf spot segmentation is performed using a lesion segmentation algorithm (a mature algorithm).

[0099] 3. Severity classification of speckle color based on K-means clustering algorithm

[0100] For the segmented spot areas, the leaf spots gradually change color as they spread. Taking grape leaves as an example, the early spots change from green to yellow, then to dark yellow, and finally to brown. To represent this gradual change in the diffusion model, the spots are divided into three levels according to severity: mild (spotlevel 1), moderate (spotlevel 2), and severe (spotlevel 3).

[0101] For the classification of pixels in the spotted area, considering practical considerations and to effectively and accurately determine the cluster centers for the spotted level, we first selected a leaf image with light, moderate, and severe spotted areas as determined by manual analysis. After extracting the lesions through algorithmic segmentation, we then used an unsupervised K-means clustering algorithm to cluster the lesion areas, setting the cluster centers to 3. The algorithm is as follows:

[0102] Using the gray values ​​of the spotted regions as samples x, the sample set is divided into 3 clusters based on the distance between samples. The goal is to maximize the distance between points within a cluster and minimize the distance between clusters. Given a sample set D, the k-means algorithm minimizes the squared error of the resulting cluster partition C. Let the number of clusters be 3, and μ... iLet be the cluster center of cluster i. Then the mean square error E of k-means clustering can be expressed as the formula:

[0103]

[0104] The K-Means clustering algorithm is essentially the model optimization process of the EM algorithm (Expectation-Maximization algorithm, EM). The specific steps are as follows:

[0105] (1) Randomly select k samples as the mean vector of the initial cluster class;

[0106] (2) Divide each sample dataset into the clusters that are closest to it;

[0107] (3) Update the mean vector of the cluster class according to the cluster to which each sample belongs;

[0108] (4) Repeat steps (2) and (3). When the set number of iterations is reached or the mean vector of the clusters no longer changes, the model construction is complete, and the clustering algorithm results are output.

[0109] After classifying the pixel levels of each spot using the K-means algorithm, a labeling template was set for each image to reflect the degree of lesion in each pixel. The template was 800*800 pixels in size, with each location corresponding to a pixel in the original image. Background blank areas in the labeling template were not labeled, and pixels on healthy leaves were labeled as 0. Mildly lesioned areas were labeled '1', moderately lesioned areas as '2', and severely lesioned areas as '3'.

[0110] Meanwhile, it was observed that the spot area increases during the spot diffusion process. Therefore, the healthy pixels adjacent to the spot area are larger than the pixels in the healthy leaf area. Thus, the pixels that combine the spots and healthy pixels are specially marked and marked with 11 on the template as edge pixels.

[0111] To clearly and intuitively observe the changes between healthy pixels and different levels of spots on the leaf surface, the graded leaf images are converted into grayscale maps, using different grayscale levels to represent various types of pixels on the leaf surface. For example, white is used to represent the background with a grayscale value of "255", a light color is used to represent the healthy color of the leaf surface with a grayscale value of "200", and darker grayscale values ​​are used to represent different levels of spot color, until the darkest areas of spots are represented by black with a grayscale value of "0". This grayscale map can more intuitively display the leaf surface characteristics.

[0112] 4. Leaf spot feature extraction

[0113] Leaf spot pixel expansion is a process in which healthy pixels decrease while spot pixels increase, and the increase in spot pixels gradually increases according to severity level. Therefore, ideally, healthy pixels should decrease while spot pixels increase in a time-series of images. However, in practice, due to the difficulty in effectively controlling the time, angle, lighting, and lens distance of each shot, the images vary in size and angle, making it difficult to characterize leaf spot features using absolute pixel count.

[0114] To more accurately represent the temporal changes of healthy pixels and speckled pixels at different levels, percentages are used. Hp% represents the proportion of healthy pixel areas to the entire leaf surface, Sp1% represents the proportion of slightly speckled pixels to the leaf surface area, Sp2% represents the proportion of moderately speckled pixels, and Sp3% represents the proportion of severely speckled pixels. The calculation formulas are as follows:

[0115]

[0116]

[0117]

[0118]

[0119] Among them, the above features satisfy the following constraints:

[0120] Hp%+Sp1%+Sp2%+Sp3%=1

[0121] Right now:

[0122] Hp%=1-Sp1%-Sp2%-Sp3%

[0123] In other words, by predicting the area of ​​spots at different degrees, the percentage of pixels on the healthy leaf surface can be calculated using the formula. Therefore, subsequent predictions only need to target the area of ​​the spots to reduce computation.

[0124] S2, which simulates the process of spot diffusion, essentially predicts the distribution and proportion of healthy pixels and spotted pixels on the leaf surface in the next state. Generally, in the next state, the healthy area always decreases, while the spotted area on the leaf surface only increases, and the severity also increases unidirectionally. To predict the feature values ​​of the leaf spots in the next state, an LSTM model is used. This model needs to be trained before it can be used for prediction.

[0125] Long Short-Term Memory (LSTM) networks are a special type of Recurrent Neural Network (RNN). Proposed by Hochreiter and Schmidhuber (1997), they have become widely popular and have undergone numerous improvements and modifications. LSTMs are widely used to solve various problems with excellent results. Specifically, LSTMs were designed primarily to avoid the long-term dependency problem mentioned earlier. The essence of LSTMs is their ability to remember information over a very long period, and this is achieved quite easily.

[0126] All recurrent neural network (RNN) structures are replicated from modules with identical structures. Long Short-Term Memory (LSTM) neural networks are a special type of RNN that excels at processing time-dependent data, particularly for predicting and classifying sequential data. Compared to traditional RNNs, it has a unique internal structure containing one or more memory blocks. Each memory block consists of a memory unit and three gate controllers (input gate, forget gate, and output gate, which control the inflow, retention, and outflow of information), solving the gradient vanishing or exploding problems encountered by traditional RNNs when processing long sequence data.

[0127] In LSTM, the input gate controls the entry of new information, the forget gate determines whether information is retained or discarded, and the output gate acts as a conduit controlling the flow of information from the cell state to the output state, ensuring that the output information is the filtered and important part. This structure of LSTM enables it to learn long-term dependencies and effectively process and predict data with complex temporal dynamics. For a detailed description of the internal memory cell structure of the LSTM model, see [link to LSTM model documentation]. Figure 2 As shown. The computation process of LSTM can be described as follows: First, the input gate i at time t is calculated. t The forget gate is f t The output gate is o t and memory cell state c t The values ​​are then used to calculate the hidden layer memory cell state ht and output y. tWhere σ represents the activation function used for gate controller calculation, g represents the nonlinear change of the input, tanh function is used to calculate the hidden layer memory unit state, W is the weighting value of different gates, and b is the constant bias term. Through this process, the LSTM neural network can dynamically control the output and input, thereby learning the long-term and short-term dependencies in sequence data. By training the LSTM model using extracted series of feature data, the trained model is used to predict subsequent blob features, thus obtaining the predicted value of the next state.

[0128] The specific calculation process of the LSTM model is as follows:

[0129] i t =σ(W xi x t +W hi h t-1 +W ci C t-1 +b i )

[0130] f t =σ(W xf x t +W hf ht-1+W cf c t-1 +b f )

[0131] c t =f t c t-1 +i t g(W xc x t +W hc h t-1 +b c )

[0132] o t =σ(W xo x t +W ho h t -1+W co c t-1 +b o )

[0133] h t =o t tanh(c t )

[0134] y t =W hy h t +b y

[0135] In this invention, three long short-term memory networks are used to predict Sp1%, Sp2% and Sp3% respectively, while calculating the value of Hp%.

[0136] In step S3, for the spot diffusion task, the analysis and prediction are performed in a discrete state manner. Each captured image is regarded as a 'discrete value'. The feature value of the leaf image is the area ratio of spots of different degrees. Considering that the leaf surface is composed of individual pixels, as long as the change of each pixel is determined, the overall condition of the leaf surface at the next moment can be predicted.

[0137] Based on the above concept, the prediction method involves first using the trained LSTM model to predict the leaf surface features for the next state, then comparing the predicted value with the current value to calculate the transition probability of each type of pixel on the leaf surface. Next, a Markov model is used to calculate the probability of each pixel. Finally, adjustments are made based on the calculation results to obtain the predicted grayscale image. The following details each part of the model:

[0138] 1. Discrete Markov Chain

[0139] In this task, the change of each leaf pixel is treated as a discrete-state Markov process, and therefore described using the following discrete Markov model. Assume a leaf pixel is in state X at a certain moment. i Then its sequence over the entire period can be regarded as the following set of states: X = {X1, X2, ..., X} t ,X t+1 …}, whose conditional probability distribution p(X) t+1 |X t This is called the state transition probability. Suppose a random variable changes from state 0 to state k, and at time t-1 it is in state j. Then the probability that it is in state i at time t is p. ij =(X t =i|X t-1 If j), then the transition probability matrix P can be set as:

[0140]

[0141] TT

[0142]

[0143] Discrete Markov state changes can be written in the following matrix form:

[0144] X t+1 =PX t

[0145] 2. Setting the leaf pixel transition probability matrix and solving for the transition probability

[0146] In the leaf spot diffusion model, each pixel is considered as a Markov process. Assume that the leaf pixels in the grayscale model image have the following states:

[0147] Status 0: Healthy status;

[0148] Condition 1: Mild spots;

[0149] Status 2: Moderate spots;

[0150] Status 3: Severe spots.

[0151] Use p ij Let represent the probability that a pixel transitions from state j to state i. Then the probability transition matrix is ​​as follows:

[0152]

[0153] In the probability transition matrix represented by the formula, for example: p 10 p represents the probability of transitioning from a healthy state to a mildly spotted state. 1X p represents the probability that adjacent pixels will become mild speckles. 21 p represents the probability that mild spots will progress to moderate spots. 32 This represents the probability of mild spots transforming into moderate spots. Based on actual observations, the following assumptions are made regarding the probability of leaf surface pixels:

[0154] (1) The model assumes that state transitions only occur from healthy to speckled, and the degree of transition only increases from mild to severe, meaning the transition process is unidirectional and irreversible:

[0155] (2) The severity of the disease cannot be skipped in one step of the transfer, that is, the severity increases by one level in one transfer;

[0156] (3) Considering that the lesion of adjacent healthy pixels is greater than that of healthy pixels that are not adjacent to lesions, the adjacent state is proposed as a separate state.

[0157] Based on the above settings, and considering that the sum of each row of the probability transition matrix in a Markov process is 1, the probability transition matrix of the leaf pixel can be simplified as follows:

[0158]

[0159] Since the sum of each row of the probability transition matrix is ​​1, then:

[0160] p 22 =1-p 32

[0161] p 11 =1-p 21

[0162] p 00=1-p 10

[0163] According to the formula, we only need to find p. 10 ,p 21 ,p 32 Then the probability transition matrix P can be determined, and these three probability values ​​can be obtained by statistically analyzing the areas of each region from time t-1 to time t. That is, the transition probabilities of each state of a pixel can be calculated using the following method:

[0164] set up The areas of healthy, mildly spotted, moderately spotted, and severely spotted leaf surfaces at time t-1 are respectively. To predict the corresponding region surface at time t using the LSTM network, we have:

[0165]

[0166]

[0167]

[0168] Here, p10 represents the probability of a healthy pixel turning into a mild blemish. It consists of two parts: the probability of a neighboring pixel turning into a blemish pixel, denoted as p. 1X Second, the probability of a non-adjacent healthy pixel being converted into a speckle pixel, denoted as p. 1Y Then we have:

[0169] p 01 =p 1X +p 1Y

[0170] Of the two probability values ​​mentioned above, p can generally be considered to be obtained through observation, i.e., experience. 1X >p 1Y In other words, adjacent pixels are more likely to transform into blob pixels. Based on manual analysis of the experimental data in this round, this paper sets an empirical ratio for the dataset, assuming that the probability ratio of adjacent pixels to non-adjacent pixels transforming into blob regions is 1:9, that is:

[0171] p 1X :p 1Y =9:1

[0172] have:

[0173] p 1X =0.9*p 10

[0174] p 1Y =0.1*p 10

[0175] At this point, the pixel transfer probabilities for all types of leaf surfaces have been determined.

[0176] 3. Calculation and adjustment of leaf spot spread

[0177] The pixel transition probability is obtained and can be calculated by multiplying each pixel by the probability transition matrix: for healthy pixels Hp, they are divided into adjacent pixels and non-adjacent pixels, and it is considered that they have a probability of transitioning into speckle pixels. The transition probability is calculated for each category. For speckle pixels of different degrees, the relevant probability values ​​are also used for prediction.

[0178] 4. Image Adjustment: Because the diffusion image calculated by probability may have 'hole pixels', that is, a slightly spotted pixel surrounded by medium or heavily spotted pixels, which is obviously not normal, the following adjustments are made:

[0179] (1) If a healthy pixel is surrounded by diseased pixels on three adjacent sides, then adjust it to a diseased pixel of the same level;

[0180] (2) If a spot pixel is surrounded by three adjacent spots of a higher level, then adjust the pixel to a spot pixel of a higher level.

[0181] After the above adjustments, the spotted area becomes a connected area, which is more in line with common sense.

[0182] like Figure 3 As shown, the present invention provides a plant leaf spot diffusion prediction system based on Markov models and long short-term memory networks, which specifically includes:

[0183] The image acquisition module periodically acquires images of the same plant leaf surface with spots;

[0184] The preprocessing module performs preprocessing on the acquired images, such as background removal and spot segmentation, and extracts the proportion of spots at each level on the leaf surface. By repeating the above preprocessing operations on leaf surface images acquired at different times, the temporal series features of each level of spots are obtained.

[0185] The LSTM prediction model training module uses extracted time series features to train the LSTM network, enabling it to predict the subsequent diffusion ratio of different types of blob models.

[0186] The speckle diffusion prediction module predicts speckle diffusion based on discrete Markov chains.

[0187] The specific application field of this invention is in the field of agricultural and forestry pest and disease research and control. It can be specifically applied to the study of the spread of lesions or spots on various crops, to study the spread speed and mode of diseases on the leaf surface, and to predict future spread trends.

[0188] The relevant products can be designed as methods for identifying pesticide effectiveness and can be used to develop subsequent systems to compare and analyze the inhibitory effect of pesticides on the spread of lesions after application, so as to quantitatively identify the effectiveness and feasibility of pesticides.

[0189] This invention has now been applied to a grape leaf spot diffusion prediction model. Currently, some grape leaf images have been collected, and after processing according to the process of this invention, including background removal, leaf spot level classification, grayscale image construction, feature extraction, LSTM network training, and spot diffusion prediction based on Markov chains, preliminary results have been achieved.

[0190] Leaf segmentation diagram Figure 4 The first image is the original image. The second to fifth images are the healthy pixel area, the Spotlevel1 area with mild spots, the Spotlevel2 area with moderate spots, and the Spotlevel3 area with severe spots, respectively.

[0191] Illustration of grape leaf spot spread prediction Figure 5 The first image is the original image, and the following images are schematic diagrams of the spread of the spots.

[0192] After testing, the LSTM network model in this invention achieved a training accuracy of over 90%. The subsequent diffusion model showed relative errors of 10%, 12%, and 14% respectively compared to the actual values ​​for the predicted diffusion of mild, moderate, and severe speckled regions.

[0193] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0194] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for predicting the spread of leaf spot in plants based on Markov models and long short-term memory networks, characterized in that, The method specifically includes: S1: Image acquisition and preprocessing: Continuously acquire a series of images of the grape leaf spot diffusion process, extract leaf spot features related to each image, and construct a time series from the features extracted from the series of images. S2: LSTM prediction model training: The LSTM model is trained using features extracted from leaf spot patterns in sequential images, enabling it to predict subsequent spot feature regions. S3: Spot diffusion prediction based on discrete Markov chain. The test image is converted into a gray level image according to the leaf surface features. The pixel change of each leaf gray level image is regarded as a discrete Markov process. The leaf pixel features are predicted by the LSTM network and the probability transition matrix of the next step is calculated. The next leaf gray level image is calculated by the discrete Markov model. Finally, the diffusion prediction image is obtained through image adjustment. S3 involves the following prediction method: first, the trained LSTM model is used to predict the leaf surface features of the next state in one step; the predicted value is compared with the current value to calculate the transition probability of each type of pixel on the leaf surface; then, a Markov model is used to calculate the probability of each pixel; finally, adjustments are made based on the calculation results to obtain the predicted grayscale image. (1) Discrete Markov chain: Assume that a leaf pixel is in a state at a certain time. Then its sequence over the entire period can be regarded as the following set of states: Its conditional probability distribution This is called the state transition probability. Suppose a random variable changes from state 0 to state k, and at time t-1 it is in state j. Then the probability that it is in state i at time t is... Then the transition probability matrix can be set as follows: S.T , Discrete Markov state changes can be written in the following matrix form: (2) Setting the leaf pixel transition probability matrix and solving the transition probability: Each pixel in the leaf spot diffusion model is regarded as a Markov process. Assume that the leaf pixels in the grayscale model image have the following states: State 0: healthy state, State 1: mild spot, State 2: moderate spot, State 3: severe spot. Let represent the probability that a pixel transitions from state j to state i. Then the probability transition matrix is ​​as follows: The probability transition matrix of the leaf pixels can be simplified to: Since the sum of each row of the probability transition matrix is ​​1, then: According to the formula, we only need to find Then the probability transition matrix P can be determined. These three probability values ​​can be obtained by statistically analyzing the area of ​​each region from time t-1 to time t. That is, the transition probability of each state of the pixel can be calculated using the following method: set up , , , The areas of healthy, mildly spotted, moderately spotted, and severely spotted leaf surfaces at time t-1 are respectively. , , , To predict the corresponding region surface at time t using the LSTM network, we have: in, This represents the probability that a healthy pixel will become a mild blemish. It consists of two parts: the probability that a neighboring pixel will become a blemish pixel, denoted as [missing information]. Second, the probability of a non-adjacent healthy pixel being converted into a speckle pixel, denoted as... Then we have: An empirical ratio is set for the dataset, assuming that the probability ratio of adjacent pixels to non-adjacent pixels turning into blob regions is 1:9, that is: have: At this point, the transition probabilities of all types of leaf surface pixels have been determined; (3) Calculation and adjustment of leaf spot spread: By multiplying each pixel by the probability transfer matrix, the healthy pixel Hp is divided into adjacent pixels and non-adjacent pixels, and it is considered that it has a probability of being transferred to spot pixels. The transfer probability is calculated by classification. For spot pixels of different degrees, the relevant probability values ​​are used for prediction. (4) Image adjustment: If a healthy pixel is surrounded by lesion pixels on three adjacent sides, it is adjusted to be a lesion pixel of the same level. If a spot pixel is surrounded by spot pixels of a higher level on three adjacent sides, it is adjusted to be a spot pixel of a higher level. After the above adjustments, the spot area becomes a connected area, which is more in line with common sense.

2. The method for predicting the spread of plant leaf spots based on Markov models and long short-term memory networks as described in claim 1, characterized in that, S1 involves periodically collecting images of the same plant leaf with spots, every 2-3 days, until the leaf surface withers or falls off over a large area.

3. The method for predicting the spread of plant leaf spots based on Markov models and long short-term memory networks as described in claim 1, characterized in that, S1 processes the collected leaf images. Each image processing step includes background removal, lesion segmentation, image grayscale conversion, disease severity grading, and feature extraction. For a series of images with a disease cycle, each type of feature can be extracted once from each image to form a time series of features, which is used as training data for a time series model. (1) Background removal was performed using 360 AI tools; (2) Leaf spot segmentation was performed using a lesion segmentation algorithm; (3) Classify the severity of spot color based on k-means clustering algorithm; (4) Convert the graded leaf images into grayscale images; (5) Hp% represents the proportion of healthy pixel areas to the entire leaf surface, Sp1% represents the proportion of slightly spotted pixels to the leaf surface area, Sp2% represents the proportion of moderately spotted pixels, and Sp3% represents the proportion of severely spotted pixels. The calculation formula is as follows: Among them, the above features satisfy the following constraints: Right now: 。 4. The method for predicting the spread of plant leaf spots based on Markov models and long short-term memory networks as described in claim 1, characterized in that, In step S2, an LSTM model is selected to predict the leaf spot feature values ​​of the next state. This model needs to be trained before it can be used for prediction. Three long short-term memory networks are used.

5. A system based on the plant leaf spot diffusion prediction method according to any one of claims 1-4, characterized in that, The system specifically includes: - Image acquisition and preprocessing module, used to continuously acquire a series of images of the spread of grape leaf spots, extract leaf spot features related to each image, and construct a time series from the features extracted from the series of images; - The LSTM prediction model training module uses features extracted from leaf spot patterns in sequential images to train the LSTM model, enabling it to predict subsequent spot feature regions. - The spot diffusion prediction module, based on the Discrete Markov chain, converts the test image into a grayscale image according to the leaf surface features. The pixel changes of each leaf surface grayscale image are regarded as a Discrete Markov process. The leaf surface pixel features are predicted by the LSTM network and the probability transition matrix of the next step is calculated. The next leaf surface grayscale image is calculated using the Discrete Markov model. Finally, the diffusion prediction image is obtained through image adjustment.

6. The system according to claim 5, characterized in that, The image acquisition and preprocessing module periodically acquires images of the same plant leaf with spots, every 2-3 days, until the leaves wither or fall off over a large area. The module then performs background removal, lesion segmentation, image grayscale conversion, disease severity classification, and feature extraction on the collected leaf images.

7. The system according to claim 5, characterized in that, The LSTM prediction model training module uses an LSTM model to predict the leaf spot feature values ​​of the next state. This module contains three long short-term memory networks and trains the model with training data to improve the accuracy of prediction.

8. The system according to claim 5, characterized in that, The spot diffusion prediction module first uses a trained LSTM model to predict the leaf surface features of the next state, then calculates the transition probability of various pixels on the leaf surface, then uses a Markov model to calculate the probability of each pixel, and finally adjusts the calculation results to obtain the predicted grayscale map. The image is then adjusted to make the spot regions connected to more realistically simulate the diffusion of leaf spots.

Images