Plant leaf spot diffusion prediction method and system

Abstract

The invention belongs to the field of plant disease prevention and control, and discloses a plant leaf spot diffusion prediction method and system based on a Markov model and a long-short memory time network, and the method comprises the steps: continuously collecting a grape leaf spot diffusion process image series, extracting the related leaf spot features of each image, and carrying out the prediction of the leaf spot diffusion; constructing features extracted from the series of images into a time sequence; training an LSTM (Long Short Term Memory) prediction model, so that the LSTM prediction model can predict a subsequent spot feature region; the method comprises the following steps of: converting a test picture into gray scale maps according to leaf surface features, regarding the change of pixels of each leaf surface gray scale map as a discrete Markov process, calculating a probability transfer matrix of the next step through leaf surface pixel features predicted by an LSTM network, calculating a next leaf surface gray scale map by adopting a discrete Markov model, and calculating a probability transfer matrix of the next step; and a final diffusion prediction map is obtained through image adjustment. The method predicts and simulates the diffusion process of leaf scabs to a certain extent, and the model has generalization and is easy to popularize to spot diffusion research of other plant disease species.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the technical field of plant disease prevention and control, and in particular relates to a method and system for predicting plant leaf spot diffusion based on a Markov model and a long short-term memory network. Background Art

[0002] The appearance of spots on plant leaves is mostly caused by diseases and insect pests, and the spread of spots is accompanied by the withering of leaves. Therefore, predicting the speed and trend of the spread of leaf spots is helpful to study the spread of plant diseases and has a certain auxiliary role in the research on disease prevention and control of agricultural and forestry crops.

[0003] The present invention is based on the further expansion and development of the leaf spot segmentation technology, and is a further development based on the leaf spot segmentation technology for future application in the spread of plant diseases and insect pests.

[0004] Related patents: A Transformer-based grape leaf spot image segmentation method; a method and system for classifying flue-cured tobacco mature spots and disease spots based on texture and shape features; a grape disease spot detection method based on convolutional networks;

[0005] The above patents mainly focus on lesion segmentation, mainly study the lesion segmentation of different crop images, and further identify the disease types, but do not pay attention to the spread of lesions, and lack prediction research. The present invention mainly focuses on the spread of crop lesions, and studies the spread speed and mode of the spots. Therefore, the present invention has obvious innovations and has made great improvements and innovations on the basis of existing patents.

[0006] Simulating the spot diffusion of leaf images essentially studies the process of converting healthy pixels on the image into spot pixels. By observing the collected series of images, some laws of leaf spot diffusion are found:

[0007] (1) The degree of leaf spot lesions varies. They are light in color at first and then gradually become darker. For example, on grape leaves, the spots first appear light yellow, then turn yellow, and finally turn brown until they wither.

[0008] (2) The spot diffusion rule mainly spreads by expanding the existing spots, that is, the healthy parts adjacent to the spot area are more likely to be transformed into the spot area, and there are also healthy areas that are directly transformed into the spot area.

[0009] (3) Since it is necessary to photograph multiple leaf surfaces over several consecutive days, various uncertain factors may affect the shooting angle, lighting, and lens distance. As a result, the size of the same leaf surface photographed on different days may vary, which brings challenges to subsequent analysis.

[0010] In view of the above analysis, the technical problems that need to be solved urgently in the prior art are:

[0011] To simulate the spread of spots in leaf images, we need to deal with the different degrees of leaf spot lesions, the fact that healthy pixels adjacent to the spot area are more susceptible to infection, and the different sizes of the same leaf at different times due to shooting angles and distances. Summary of the invention

[0012] In view of the problems existing in the prior art, the present invention provides a method and system for predicting plant leaf spot diffusion based on a Markov model and a long short-term memory network.

[0013] The present invention is implemented as follows: a method for predicting plant leaf spot diffusion based on a Markov model and a long short-term memory network, characterized in that the method for predicting plant leaf spot diffusion based on a Markov model and a long short-term memory network specifically includes:

[0014] S1: Image acquisition and preprocessing: continuously collect a series of images of grape leaf spot diffusion process, extract the leaf spot features related to each image, and construct the features extracted from the series of images into a time series;

[0015] S2: LSTM prediction model training, using the features extracted from leaf spots in sequence images to train the LSTM model so that it can predict the subsequent spot feature areas;

[0016] S3: Spot diffusion prediction based on discrete Markov chain, convert the test image into a grayscale map according to the leaf surface features, regard the pixel change of each leaf grayscale map as a discrete Markov process, predict the leaf pixel features through the LSTM network and calculate the next step of the probability transfer matrix, use the discrete Markov model to calculate the next leaf grayscale map, and then adjust the image to get the final diffusion prediction map.

[0017] Furthermore, in S1, images of the same plant leaf surface with spots are collected periodically, for example, once every 2-3 days, until a large area of ​​the leaf surface withers or falls off.

[0018] Further, the S1 processes the collected leaf images. The main steps of each image processing include background removal, lesion segmentation, image graying, disease degree classification and feature extraction. For a series of images with a lesion cycle, each type of feature can be extracted once from each image to form a time series of features, which is used as training data for the time series model:

[0019] (1) Background removal using 360AI tools;

[0020] (2) Leaf spot segmentation is performed using the disease spot segmentation algorithm (mature algorithm)

[0021] (3) Classification of spot color severity based on k-means clustering algorithm;

[0022] (4) converting the classified leaf surface image into a grayscale image;

[0023] (5) Hp% is the percentage of healthy pixel area to the entire leaf surface, Sp1% is the percentage of mildly spotted pixels to the leaf surface area, Sp2% is the percentage of moderately spotted pixels, and Sp3% is the percentage of severely spotted pixels. The calculation formula is as follows:

[0024]

[0025]

[0026]

[0027]

[0028] Among them, the above characteristics meet the constraints:

[0029] Hp%+Sp1%+Sp2%+Sp3%=1

[0030] Right now:

[0031] Hp%=1-Sp1%-Sp2%-Sp3%

[0032] Furthermore, in S2, an LSTM model is selected to predict the leaf spot feature value of the next state. The model needs to be trained and can be used for prediction only after training. The present invention uses three long short-term memory networks.

[0033] Furthermore, the prediction method of S3 is to use the trained LSTM model to predict the leaf surface features of the next state one step at a time, compare the predicted value with the current value, calculate the transfer probability of various pixels on the leaf surface, and then use the Markov model to calculate the probability of each pixel. Finally, the calculation result is adjusted to obtain the predicted grayscale level map:

[0034] (1) Discrete Markov chain: Assume that a leaf pixel is in a state X at a certain moment. i , then its sequence in the full cycle can be regarded as the following state set: X = {X1, X2…X t ,X t+1 …}, its conditional distribution probability p(X t+1 |X t ) is called the state transition probability. Assuming that the state of the random variable changes from state 0 to state k and is in state j at time t-1, then the probability that it is in state i at time t is p ij =(X t =i|Xt-1 =j), then the transition probability matrix can be set as:

[0035]

[0036] ST

[0037]

[0038] The discrete Markov state change can be written in the following matrix form:

[0039] X t+1 =PX t

[0040] (2) Setting the leaf pixel transfer probability matrix and solving the transfer probability: Consider each pixel in the leaf spot diffusion model as a Markov process. Assume that the leaf pixels in the grayscale model have the following states: state 0: healthy state, state 1: mild spot, state 2: moderate spot, state 3: severe spot, and use p ij represents the probability of a pixel transferring from state j to state i, then the probability transfer matrix is ​​as follows:

[0041]

[0042] The leaf pixel probability transfer matrix can be simplified as:

[0043]

[0044] According to the probability transfer matrix, the sum of each row is 1, then:

[0045] p 22 =1-p 32

[0046] p 11 =1-p 21

[0047] p 00 =1-p 10

[0048] According to the formula, we only need to find p 10 ,p 21 ,p 32 , then the probability transfer matrix P can be determined, and the three probability values ​​can be counted by the area of ​​each region from time t-1 to time t, that is, the transition probability of each pixel state can be calculated by the following method:

[0049] set up are the leaf surface healthy area, light spot area, moderate spot area and heavy spot area at time t-1, For the LSTM network to predict the corresponding area surface at time t, we have:

[0050]

[0051]

[0052]

[0053] Among them, p 10 It represents the probability of a healthy pixel being converted into a mild diseased spot. It consists of two parts. One is the probability of an adjacent pixel being converted into a spot pixel, denoted as p 1X , the second is the probability of non-adjacent healthy pixels being converted into spot pixels, denoted as p 1Y , then:

[0054] p 01 =p 1X +p 1Y

[0055] An empirical ratio is set for the data set, and the probability ratio of adjacent pixels and non-adjacent pixels turning into spot areas is 1:9, that is:

[0056] p 1X :p 1Y =9:1

[0057] have:

[0058] p 1X =0.9*p 10

[0059] p 1Y =0.1*p 10

[0060] At this point, the transition probabilities of all types of leaf pixels have been determined;

[0061] (3) Calculation and adjustment of leaf surface diffused disease spots: Each pixel is multiplied by the probability transfer matrix for calculation. For healthy pixels Hp, they are divided into adjacent pixels and non-adjacent pixels. It is considered that they have the probability of transferring to spot pixels. The transfer probability is calculated by classification. For spot pixels of different degrees, the relevant probability values ​​are also used for prediction.

[0062] (4) Image adjustment: If a healthy pixel is surrounded by diseased pixels on three adjacent sides, it will be adjusted to a diseased pixel of the same level. If a spot pixel is surrounded by spot pixels of a higher level on three adjacent sides, it will be adjusted to a spot pixel of a higher level. After the above adjustment, the spot area becomes a connected area, which is more in line with common sense.

[0063] Another object of the present invention is to provide a plant leaf spot diffusion prediction system based on a Markov model and a long short-term memory network, the system specifically comprising:

[0064] An image acquisition module periodically acquires images of the same plant leaf surface with spots;

[0065] The preprocessing module performs background removal, spot segmentation and other preprocessing on the collected images, extracts the proportion of spots of each level on the leaf surface, and obtains the time series characteristics of spots of each level by repeating the above preprocessing operations on the leaf surface images collected at different times;

[0066] The LSTM prediction model training module uses the extracted time series features to train the LSTM network so that it can predict the subsequent diffusion ratio of different categories of spot models;

[0067] The speckle diffusion prediction module performs speckle diffusion prediction based on discrete Markov chains.

[0068] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0069] First, the present invention can predict and simulate the diffusion process of leaf lesions to a certain extent, among which the characteristics such as the proportion of lesion area and the proportion of severe lesion area are highly consistent with the real data. At the same time, the model is generalizable and can be easily extended to the study of spot diffusion of other plant diseases.

[0070] (1) The expected benefits and commercial value of the technical solution of the present invention after transformation are:

[0071] The present invention proposes a method for predicting the spread of plant leaf spots. In theory, it can predict and simulate the subsequent spread of leaf spots or disease spots of any plant, and can analyze and predict the spread rate of different plant diseases. The expected benefits are reflected in the prediction of diseases of agricultural and forestry crops. According to the occurrence of diseases, the speed of subsequent disease spread can be predicted, and preventive measures can be taken in advance to reduce losses caused by diseases.

[0072] The commercial value of this invention is that it can identify the effectiveness of pesticides. If a certain pesticide can prevent pests and diseases, the effect of the application can be compared with the predicted effect. If the application effect is good, the spread rate of the lesions should be less than the predicted effect, which can prove the effectiveness of the pesticide. If it is commercially available, the effectiveness test of pesticides will have a certain reference value.

[0073] (2) The technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0074] According to the search and analysis, the domestic related agricultural and forestry technologies currently retrieved are mainly focused on the segmentation of plant leaf spot and disease identification. There is no relevant patent for the research on the prediction of spot or disease spread. This shows that domestic patents have not yet used LSTM network and Markov model to predict the spread of leaf spot, and this technology is still blank in China.

[0075] Second, the technical solution of the present invention solves many problems of the prior art in predicting the spread of plant leaf spots and achieves significant technical progress. The following is a detailed description:

[0076] Technical issues solved:

[0077] 1) Insufficient prediction accuracy: Traditional plant disease prediction methods are often based on empirical models or simple statistical analysis. These methods have limited prediction accuracy when dealing with complex, nonlinear leaf spot diffusion processes. The present invention improves the ability to capture complex diffusion patterns by combining the Markov model and the long short-term memory network (LSTM), thereby improving prediction accuracy.

[0078] 2) Low data processing efficiency: Traditional prediction methods are often inefficient when processing large amounts of image data and cannot meet the needs of real-time monitoring and rapid response. The technical solution of the present invention significantly improves data processing efficiency through efficient image processing and feature extraction processes, as well as the rapid learning ability of the LSTM model.

[0079] 3) Lack of dynamic prediction capability: The existing technology can only provide static disease analysis, and it is difficult to effectively predict the dynamic diffusion process of leaf spots. The present invention simulates the change of pixel state through Markov chain to achieve dynamic prediction of leaf spot diffusion.

[0080] Technological progress achieved:

[0081] 1) Improve prediction accuracy: By combining LSTM and Markov models, the present invention can more accurately capture the complex patterns and trends of leaf spot diffusion, thereby providing more reliable prediction results. This is crucial for developing effective disease prevention and control strategies.

[0082] 2) Enhanced real-time performance: The technical solution of the present invention can efficiently process and analyze a large amount of image data and provide real-time prediction of leaf spot spread, which helps to detect and respond to plant diseases in a timely manner and reduce losses.

[0083] 3) Dynamic prediction and monitoring: By simulating the changes in leaf pixel states using a Markov chain, the present invention achieves dynamic prediction and monitoring of the leaf spot diffusion process. This helps to more accurately understand the development of the disease and provide more valuable information for decision makers.

[0084] 4) The method has strong universality: The technical solution of the present invention is not only applicable to the prediction of the spread of grape leaf spots, but can also be extended to the prediction and monitoring of other plant diseases. This shows the wide application prospects of the present invention in the field of plant protection.

[0085] In summary, the technical solution of the present invention has achieved significant technological progress by solving the key problems of the prior art and provided a more accurate, efficient and dynamic method for predicting the spread of plant leaf spots. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 It is a flow chart of a method for predicting plant leaf spot diffusion based on a Markov model and a long short-term memory network provided by an embodiment of the present invention;

[0087] Figure 2 It is a specific internal memory unit structure diagram of the LSTM model provided in an embodiment of the present invention;

[0088] Figure 3 It is a module diagram of a plant leaf spot diffusion prediction system based on a Markov model and a long short-term memory network provided in an embodiment of the present invention.

[0089] Figure 4 It is a schematic diagram of leaf surface segmentation provided by an embodiment of the present invention.

[0090] Figure 5 It is a schematic diagram of grape leaf spot diffusion prediction provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0091] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0092] like Figure 1 As shown, an embodiment of the present invention provides a method for predicting plant leaf spot diffusion based on a Markov model and a long short-term memory network, the method specifically comprising:

[0093] S1: Image acquisition and preprocessing: continuously collect a series of images of grape leaf spot diffusion process, extract the leaf spot features related to each image, and construct the features extracted from the series of images into a time series;

[0094] S2: LSTM (Long Short-Term Memory) prediction model training, using the features extracted from the leaf spots in the sequence images to train the LSTM model so that it can predict the subsequent spot feature areas;

[0095] S3: Spot diffusion prediction based on discrete Markov chain, convert the test image into a grayscale map according to the leaf surface features, regard the pixel change of each leaf grayscale map as a discrete Markov process, predict the leaf pixel features through the LSTM network and calculate the next step of the probability transfer matrix, use the discrete Markov model to calculate the next leaf grayscale map, and then adjust the image to get the final diffusion prediction map.

[0096] In S1, images of the same plant leaf surface with spots are collected periodically, for example, once every 2-3 days, until a large area of ​​the leaf surface withers or falls off. The collected leaf images are processed, and the main steps of each image processing include background removal, spot segmentation, image graying, disease degree classification and feature extraction. For a series of images with a lesion period, each type of feature can be extracted once from each image to form a time series of features, which is used as training data for the time series model.

[0097] 1. Background removal uses 360AI artificial intelligence tools.

[0098] 2. Leaf spot segmentation uses the disease spot segmentation algorithm (mature algorithm)

[0099] 3. Spot color severity classification based on K-means clustering algorithm

[0100] For the segmented spots, the spots on the leaf surface gradually change color as they spread. For example, in the case of grape leaves, the early spots change from green to yellow, then to dark yellow, and finally to brown. In order to reflect this gradual change process in the diffusion model, the spots are divided into three levels according to their severity: mild, represented by 'spotlevel1', moderate, represented by 'spotlevel2', and severe, represented by 'spotlevel3'.

[0101] For the classification of pixels in the spot area, considering the actual situation, in order to effectively and accurately obtain the cluster center of the spot level, we first selected a leaf image that was manually judged to have mild, moderate and severe spots. After extracting the spots through algorithm segmentation, the unsupervised K-means clustering algorithm was used to cluster the spot area, and the cluster center was set to 3. The algorithm is as follows:

[0102] The gray value of the spot area is taken as the sample x, and the sample set is divided into 3 clusters according to the distance between the samples. The points in the cluster are connected as closely as possible, and the distance between clusters is as large as possible. Given a sample set D, the k-means algorithm minimizes the square error of the cluster partition C obtained by clustering. Assume that the number of clusters is 3, μ iis the cluster center of cluster i, then the mean square error E of k-means clustering can be expressed as:

[0103]

[0104] The steps of the K-Means clustering algorithm are essentially the model optimization process of the EM algorithm (Expectation-Maximization algorithm, EM). The specific steps are as follows:

[0105] (1) Randomly select k samples as the mean vector of the initial cluster;

[0106] (2) Divide each sample data set into the cluster with the closest distance;

[0107] (3) Update the mean vector of the cluster according to the cluster to which each sample belongs;

[0108] (4) Repeat steps (2) and (3). When the set number of iterations is reached or the mean vector of the cluster no longer changes, the model is built and the clustering algorithm results are output.

[0109] After the K-means algorithm is used to grade the pixel levels of each spot, a marking template is set for each image to reflect the degree of lesions of each pixel. The size is the same as the image size, 800*800, and each position corresponds to a pixel point in the original image. The background blank area in the marking template is not marked, and the healthy leaf pixel position is marked as 0. The template for the area with mild spots is marked as '1', the template for the area with moderate spots is marked as '2', and the template for the area with severe spots is marked as '3'.

[0110] At the same time, it is observed that the spot diffusion process makes the spot area larger, so the lesions of healthy pixels adjacent to the spot area are larger than the pixels in the healthy leaf area. Therefore, the combined pixels around the lesions and healthy pixels are specially marked and marked with 11 on the template as edge pixels.

[0111] In order to clearly and intuitively observe the changes between the healthy pixels and the levels of spots on the leaf surface, the graded leaf surface image is also converted into a grayscale map, that is, different grayscale levels are used to represent various pixels on the leaf surface. For example, white is used to represent the background, and the grayscale value is: "255", light colors are used to represent the healthy color of the leaf surface, and the grayscale value is "200", and darker grayscale values ​​are used to represent the color of spots of different levels, until the heavy spot area is represented by black: the grayscale value is "0", so that the grayscale map can more intuitively display the characteristics of the leaf surface.

[0112] 4. Leaf spot feature extraction

[0113] Leaf spot pixel expansion is a process in which healthy pixels decrease while spot pixels increase, and the spot pixels gradually increase according to the severity level. Therefore, ideally, healthy pixels on the time series image should decrease while spot pixels increase. However, from the actual effect, it is difficult to effectively control the time, angle, lighting and lens distance of each shot, and the images taken are of different sizes and angles, making it difficult to characterize the leaf spot characteristics with the absolute number of pixels.

[0114] In order to more reasonably characterize the temporal variation relationship of healthy pixels and spotted pixels at each level, it is expressed as a percentage. Hp% is the proportion of healthy pixel area to the entire leaf surface, Sp1% is the proportion of mild spotted pixels in the leaf surface area, Sp2% is the proportion of moderate spotted pixels, and Sp3% is the proportion of severe spotted pixels. The calculation formula is as follows:

[0115]

[0116]

[0117]

[0118]

[0119] Among them, the above characteristics meet the constraints:

[0120] Hp%+Sp1%+Sp2%+Sp3%=1

[0121] Right now:

[0122] Hp%=1-Sp1%-Sp2%-Sp3%

[0123] In other words, we only need to predict the area of ​​each spot to calculate the percentage of healthy leaf pixels using the formula. Therefore, we only need to predict part of the spot area in the future to reduce the amount of calculation.

[0124] S2 simulates the process of spot diffusion, which is essentially to predict the distribution and proportion of healthy pixels and spot pixels on the leaf surface in the next state. According to general rules, in the next state, the healthy area will always decrease, the spot area on the leaf surface will only increase, and the severity will also increase in one direction. In order to predict the feature value of the leaf spot in the next state, the LSTM model is used for prediction. The model needs to be trained before it can be used for prediction.

[0125] Long Short Term Memory networks, commonly called "LSTMs" - are a special type of Recurrent Neural Network (RNN). Proposed by Hochreiter & Schmidhuber (1997), it was widely popular and has been improved and adjusted by many people since then. LSTMs are widely used to solve various problems and have achieved excellent results. Specifically, the design of LSTMs is mainly to avoid the long-term dependency problem mentioned above. The essence of LSTMs is to be able to remember information for a long period of time, and it can do it very easily.

[0126] All recurrent neural network structures are made up of replicated (neural network) modules with exactly the same structure. Long Short-term Memory Networks (LSTM) is a special type of recurrent neural network (RNN) that excels in processing time-dependent data, especially for the prediction and classification of sequence data. Compared with traditional RNN, it has a unique internal structure, containing one or more memory blocks, each of which consists of a memory unit and three gate controllers (input gate, forget gate and output gate, which control the inflow, retention and outflow of information), which solves the gradient vanishing or exploding problem encountered by traditional RNN when processing long sequence data.

[0127] In LSTM, the input gate controls the entry of new information, the forget gate determines whether the information is retained or discarded, and the output gate acts like a conduit to control the flow of information from the cell state to the output state, ensuring that the output information is the important part that has been screened. This structure of LSTM enables it to learn long-term dependencies and effectively process and predict data with complex time dynamics. The specific internal memory unit structure of the LSTM model can be found in Figure 2 The calculation process of LSTM can be described as follows: First, calculate the input gate i at time t t , the forget gate is f t , the output gate is o t and the memory cell state c t The values ​​of , and then calculate the hidden layer memory unit state ht and output y based on these values t. Where σ represents the activation function used for gate controller calculation, function g represents the nonlinear change of input, tanh function is used to calculate the state of hidden layer memory unit, W is the weight value of different gates, and b is the constant term bias. Through this process, the LSTM neural network can dynamically control the output and input, thereby learning the long-term and short-term dependencies in the sequence data. The LSTM model is trained by using the extracted series feature data, and the trained model is used to predict the subsequent spot features to obtain the predicted value of the next state.

[0128] The specific calculation process of the LSTM model is as follows:

[0129] i t =σ(W xi x t +W hi h t-1 +W ci C t-1 +b i )

[0130] f t =σ(W xf x t +W hf ht-1+W cf c t-1 +b f )

[0131] c t =f t c t-1 +i t g(W xc x t +W hc h t-1 +b c )

[0132] o t =σ(W xo x t +W ho h t -1+W co c t-1 +b o )

[0133] h t =o t tanh(c t )

[0134] y t =W hy h t +b y

[0135] In the present invention, three long short-term memory networks are used to predict Sp1%, Sp2% and Sp3% respectively, and at the same time, the value of Hp% is calculated.

[0136] The S3, for the spot diffusion task, adopts a discrete state method for analysis and prediction, and regards a captured image as a 'discrete value'. The characteristic value of the leaf surface image is the proportion of spot areas of different degrees. Considering that the leaf surface is composed of independent pixels, as long as the change of each pixel is determined, the overall situation of the leaf surface at the next moment can be predicted.

[0137] Based on the above concept, the prediction method is to use the trained LSTM model to predict the leaf surface features of the next state, compare the predicted value with the current value, calculate the transition probability of various pixels on the leaf surface, and then use the Markov model to calculate the probability of each pixel. Finally, adjust the calculation results to obtain the predicted grayscale map. The following is a detailed introduction to each part of the model:

[0138] 1. Discrete Markov Chain

[0139] In this task, the change of each leaf pixel is set as a discrete state Markov process, so the following discrete Markov model is used to describe it. Assume that a leaf pixel is in a state X at a certain moment. i , then its sequence in the full cycle can be regarded as the following state set: X = {X1, X2…X t ,X t+1 …}, its conditional distribution probability p(X t+1 |X t ) is called the state transition probability. Assuming that the state of the random variable changes from state 0 to state k and is in state j at time t-1, then the probability that it is in state i at time t is p ij =(X t =i|X t-1 =j), then the transition probability matrix P can be set as:

[0140]

[0141] TT

[0142]

[0143] The discrete Markov state change can be written in the following matrix form:

[0144] X t+1 =PX t

[0145] 2. Leaf pixel transfer probability matrix setting and transfer probability solution

[0146] Consider each pixel in the leaf spot diffusion model as a Markov process. Assume that the leaf pixels in the grayscale model have the following states:

[0147] State 0: healthy state;

[0148] State 1: Mild spots;

[0149] State 2: moderate spots;

[0150] State 3: Severe spotting.

[0151] Use p ij represents the probability of a pixel transferring from state j to state i, then the probability transfer matrix is ​​as follows:

[0152]

[0153] In the probability transfer matrix represented by the formula, for example: p 10 represents the probability of transitioning from a healthy state to a mildly spotted state; p 1X is the probability that the adjacent pixel turns into a light spot; p 21 is the probability of mild spots turning into moderate spots; p 32 is the probability of mild spots turning into moderate spots. Based on the actual observations, the following assumptions are made about the leaf pixel probability:

[0154] (1) In the model, it is assumed that the state transition will only shift from healthy to spotted, and the degree will also shift from light to heavy, that is, the transition process is one-way and irreversible:

[0155] (2) The severity level cannot be skipped during a one-step transfer, i.e., the severity level increases by one level each time the transfer is made;

[0156] (3) Considering that the adjacent healthy pixels are more pathological than the healthy pixels not adjacent to the lesions, the adjacent state is proposed as a separate state.

[0157] According to the above settings, and combined with the fact that the sum of each row of the probability transfer matrix in the Markov process is 1, the leaf pixel probability transfer matrix can be simplified to:

[0158]

[0159] According to the probability transfer matrix, the sum of each row is 1, then:

[0160] p 22 =1-p 32

[0161] p 11 =1-p 21

[0162] p 00=1-p 10

[0163] According to the formula, we only need to find p 10 ,p 21 ,p 32 , then the probability transfer matrix P can be determined, and the three probability values ​​can be counted by the area of ​​each region from time t-1 to time t. That is, the transition probability of each pixel state can be calculated by the following method:

[0164] set up are the leaf surface healthy area, light spot area, moderate spot area and heavy spot area at time t-1, For the LSTM network to predict the corresponding area surface at time t, we have:

[0165]

[0166]

[0167]

[0168] Among them, p10 represents the probability of a healthy pixel being converted into a mild diseased spot, which consists of two parts. One is the probability of an adjacent pixel being converted into a spot pixel, denoted by p 1X , the second is the probability of non-adjacent healthy pixels being converted into spot pixels, denoted as p 1Y , then:

[0169] p 01 =p 1X +p 1Y

[0170] Of the two probability values ​​above, it can be generally believed through observation or experience that p 1X >p 1Y , that is, the adjacent pixels are more likely to be transformed into spot pixels. According to the manual analysis of this round of experimental data, in this paper, an empirical ratio is set for the data set, and it is believed that the probability ratio of adjacent pixels and non-adjacent pixels to be transformed into spot areas is 1:9, that is:

[0171] p 1X :p 1Y =9:1

[0172] have:

[0173] p 1X =0.9*p 10

[0174] p 1Y =0.1*p 10

[0175] At this point, the transition probabilities of all types of leaf pixels have been determined.

[0176] 3. Calculation and adjustment of leaf surface spread spots

[0177] The pixel transfer probability is obtained and can be calculated by multiplying each pixel by the probability transfer matrix: for healthy pixels Hp, they are divided into adjacent pixels and non-adjacent pixels, and it is considered that they have the probability of transferring to spot pixels. The transfer probability is calculated by classification, and for spot pixels of different degrees, the relevant probability values ​​are also used for prediction.

[0178] 4. Image adjustment: Because the diffusion image calculated by probability will have "hole pixels", that is, for example, a light spot pixel is surrounded by medium or heavy spot pixels, which is obviously not in line with common sense, so the following adjustments are made:

[0179] (1) If a healthy pixel is surrounded by diseased pixels on three adjacent sides, it will be adjusted to a diseased pixel of the same level;

[0180] (2) If a spot pixel is surrounded by spot pixels of a higher level on three adjacent sides, the pixel is adjusted to a spot pixel of a higher level;

[0181] After the above adjustments, the spotted area becomes a connected area, which is more in line with common sense.

[0182] like Figure 3 As shown, an embodiment of the present invention provides a plant leaf spot diffusion prediction system based on a Markov model and a long short-term memory network, which specifically includes:

[0183] An image acquisition module periodically acquires images of the same plant leaf surface with spots;

[0184] The preprocessing module performs background removal, spot segmentation and other preprocessing on the collected images, extracts the proportion of spots of each level on the leaf surface, and obtains the time series characteristics of spots of each level by repeating the above preprocessing operations on the leaf surface images collected at different times;

[0185] The LSTM prediction model training module uses the extracted time series features to train the LSTM network so that it can predict the subsequent diffusion ratio of different categories of spot models;

[0186] The speckle diffusion prediction module performs speckle diffusion prediction based on discrete Markov chains.

[0187] The specific application field of the present invention lies in the field of agricultural and forestry pest research and prevention and control. It can be specifically applied to the study of the spread of disease spots or spots on various crops, the study of the spread speed and method of diseases on leaf surfaces, and the prediction of future spread trends.

[0188] Related products can be designed as identification methods for the effectiveness of pesticides, and subsequent systems can be derived to conduct comparative analysis on the inhibition of lesion spread after pesticide application, so as to quantitatively identify the effectiveness and feasibility of pesticides.

[0189] The present invention has now been tried in the grape leaf spot diffusion prediction model. At present, some grape leaf surface images have been collected. After following the processing flow of the present invention, background removal, leaf surface spot level classification, grayscale level map construction, feature extraction, LSTM network training, and Markov chain-based spot diffusion prediction were performed, and preliminary results were achieved.

[0190] Leaf segmentation diagram Figure 4 The first picture is the original picture, the second to the fifth pictures are the healthy pixel part, the light spot part of Spotlevel1, the moderate spot part of Spotlevel2, and the heavy spot part of Spotlevel3 respectively.

[0191] Schematic diagram of grape leaf spot diffusion prediction Figure 5 , the first one is the original picture, and the following one is the schematic diagram of spot diffusion.

[0192] After testing, the LSTM network model training in the present invention has a training accuracy rate of more than 90%, and the subsequent diffusion model has relative errors of 10%, 12% and 14% respectively in comparing the predicted diffusion and actual values ​​for mild, moderate and severe spot areas.

[0193] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0194] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A method for predicting plant leaf spot diffusion based on Markov model and long short-term memory network, characterized in that: The method specifically includes: S1: Image acquisition and preprocessing: continuously collect a series of images of grape leaf spot diffusion process, extract the leaf spot features related to each image, and construct the features extracted from the series of images into a time series; S2: LSTM prediction model training, using the features extracted from leaf spots in sequence images to train the LSTM model so that it can predict the subsequent spot feature areas; S3: Spot diffusion prediction based on discrete Markov chain, convert the test image into a grayscale map according to the leaf surface features, regard the pixel change of each leaf grayscale map as a discrete Markov process, predict the leaf pixel features through the LSTM network and calculate the next step of the probability transfer matrix, use the discrete Markov model to calculate the next leaf grayscale map, and then adjust the image to get the final diffusion prediction map.

2. The method for predicting plant leaf spot diffusion based on Markov model and long short-term memory network as claimed in claim 1, characterized in that: In the step S1, images of the same plant leaf surface with spots are collected periodically, for example, once every 2-3 days, until a large area of ​​the leaf surface withers or falls off.

3. The method for predicting plant leaf spot diffusion based on Markov model and long short-term memory network as claimed in claim 1, characterized in that: S1 processes the collected leaf images. The main steps of each image processing include background removal, spot segmentation, image graying, disease degree classification and feature extraction. For a series of images with a disease cycle, each type of feature can be extracted once from each image to form a time series of features, which is used as training data for the time series model: (1) Background removal using 360AI tools; (2) Leaf spot segmentation is performed using the disease spot segmentation algorithm (mature algorithm) (3) Classification of spot color severity based on k-means clustering algorithm; (4) converting the classified leaf surface image into a grayscale image; (5) Hp% is the percentage of healthy pixel area to the entire leaf surface, Sp1% is the percentage of mildly spotted pixels to the leaf surface area, Sp2% is the percentage of moderately spotted pixels, and Sp3% is the percentage of severely spotted pixels. The calculation formula is as follows: Among them, the above characteristics meet the constraints: Hp%+Sp1%+Sp2%+Sp3%=1 Right now: Hp%=1-Sp1%-Sp2%-Sp3%.

4. The method for predicting plant leaf spot diffusion based on Markov model and long short-term memory network as claimed in claim 1, characterized in that: The S2 uses an LSTM model to predict the leaf spot feature value of the next state. The model needs to be trained and can be used for prediction after training. The present invention uses three long short-term memory networks.

5. The method for predicting plant leaf spot diffusion based on Markov model and long short-term memory network as claimed in claim 1, characterized in that: The prediction method of S3 is to use the trained LSTM model to predict the leaf surface features of the next state each time, compare the predicted value with the current value, calculate the transfer probability of various pixels on the leaf surface, and then use the Markov model to calculate the probability of each pixel. Finally, adjust the calculation result to obtain the predicted grayscale level map: (1) Discrete Markov chain: Assume that a leaf pixel is in a state X at a certain moment. i , then its sequence in the full cycle can be regarded as the following state set: X = {X1, X2…X t ,X t+1 …}, its conditional distribution probability p(X t+1 |X t ) is called the state transition probability. Assuming that the state of the random variable changes from state 0 to state k and is in state j at time t-1, then the probability that it is in state i at time t is p ij =(X t =i|X t-1 =j), then the transition probability matrix can be set as: UT The discrete Markov state change can be written in the following matrix form: X t+1 =PX t (2) Setting the leaf pixel transfer probability matrix and solving the transfer probability: Consider each pixel in the leaf spot diffusion model as a Markov process. Assume that the leaf pixels in the grayscale model have the following states: state 0: healthy state, state 1: mild spot, state 2: moderate spot, state 3: severe spot, and use p ij represents the probability of a pixel transferring from state j to state i, then the probability transfer matrix is ​​as follows: The leaf pixel probability transfer matrix can be simplified as: According to the probability transfer matrix, the sum of each row is 1, then: p 22 =1-p 32 p 11 =1-p 21 p 00 =1-p 10 According to the formula, we only need to find p 10 ,p 21 ,p 32 , then the probability transfer matrix P can be determined, and the three probability values ​​can be counted by the area of ​​each region from time t-1 to time t, that is, the transition probability of each pixel state can be calculated by the following method: set up are the leaf surface healthy area, light spot area, moderate spot area and heavy spot area at time t-1, For the LSTM network to predict the corresponding area surface at time t, we have: Among them, p 10 It represents the probability of a healthy pixel being converted into a mild diseased spot. It consists of two parts. One is the probability of an adjacent pixel being converted into a spot pixel, denoted as p 1X , the second is the probability of non-adjacent healthy pixels being converted into spot pixels, denoted as p 1Y , then: p 01 =p 1X +p 1Y An empirical ratio is set for the data set, and the probability ratio of adjacent pixels and non-adjacent pixels turning into spot areas is 1:9, that is: p 1X :p 1Y =9:1 have: p 1X =0.9*p 10 p 1Y =0.1*p 10 At this point, the transition probabilities of all types of leaf pixels have been determined; (3) Calculation and adjustment of leaf surface diffused disease spots: Each pixel is multiplied by the probability transfer matrix for calculation. For healthy pixels Hp, they are divided into adjacent pixels and non-adjacent pixels. It is considered that they have the probability of transferring to spot pixels. The transfer probability is calculated by classification. For spot pixels of different degrees, the relevant probability values ​​are also used for prediction. (4) Image adjustment: If a healthy pixel is surrounded by diseased pixels on three adjacent sides, it will be adjusted to a diseased pixel of the same level. If a spot pixel is surrounded by spot pixels of a higher level on three adjacent sides, it will be adjusted to a spot pixel of a higher level. After the above adjustment, the spot area becomes a connected area, which is more in line with common sense.

6. A plant leaf spot diffusion prediction system based on the Markov model and long short-term memory network as described in claims 1-5, characterized in that: The system specifically includes: - Image acquisition and preprocessing module, used to continuously collect a series of images of grape leaf spot diffusion process, extract leaf spot features related to each image, and construct the features extracted from the series of images into a time series; -LSTM prediction model training module, which uses the features extracted from leaf spots in sequence images to train the LSTM model so that it can predict the subsequent spot feature areas; -The spot diffusion prediction module converts the test image into a grayscale map based on the leaf surface features based on the discrete Markov chain. The pixel changes of each leaf surface grayscale map are regarded as a discrete Markov process. The leaf surface pixel features are predicted by the LSTM network and the probability transfer matrix of the next step is calculated. The discrete Markov model is used to calculate the next leaf surface grayscale map, and then the final diffusion prediction map is obtained through image adjustment.

7. The system according to claim 6, characterized in that The image acquisition and preprocessing module periodically collects images of the same plant leaf with spots, for example, once every 2-3 days, until a large area of ​​the leaves wither or fall off, and performs background removal, spot segmentation, image graying, disease severity classification and feature extraction on the collected leaf images.

8. The system according to claim 6, characterized in that The LSTM prediction model training module uses the LSTM model to predict the leaf spot feature value of the next state. The module contains three long and short memory networks and trains the model through training data to improve the accuracy of the prediction.

9. The system according to claim 6, characterized in that The spot diffusion prediction module first uses the trained LSTM model to make a one-step prediction of the leaf surface features in the next state, then calculates the transfer probability of each type of pixel on the leaf surface, and then uses the Markov model to calculate the probability of each pixel. Finally, it adjusts the calculation results to obtain the predicted grayscale map, and adjusts the image to make the spot area become a connected area, so as to more realistically simulate the diffusion of leaf spots.

Images