Potato late blight prediction method based on generative pre-training model

Through a generative pre-training model based on GPT2, combined with data preprocessing and time series alignment technology, the encoding and output layer is optimized, and the problem of difficult to utilize multi-dimensional time series data in the existing technology is solved, and the efficiency of late potato disease is achieved. Accurate prediction.

CN119046677AActive Publication Date: 2024-11-29NANJING AGRICULTURAL UNIVERSITY

Patent Information

Application Number
CN202410963656.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-29
Estimated Expiration
2044-07-18

AI Technical Summary

Technical Problem

The existing method for predicting late potato disease in the late potato is based on traditional statistical models and expert systems, and it is difficult to make full use of multi-dimensional time series data. The prediction effect is not ideal and cannot adapt to the complex agricultural environment and emerging disease patterns.

Method used

A generative pre-training model based on GPT2 is adopted, and through data preprocessing, time series alignment, coding layer and output layer optimization, combined with channel independence and blocking technology, two-stage fine-tuning is carried out to achieve accurate prediction of late potato disease.

Benefits of technology

It improves the accuracy and efficiency of potato late blight prediction, can effectively analyze complex time series data, adapt to multi-time scale meteorological and disease data, and provide more efficient disease warning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046677A_ABST
    Figure CN119046677A_ABST
Patent Text Reader

Abstract

The invention discloses a potato late blight prediction method based on a generative pre-training model, and the method comprises the steps: carrying out the data preprocessing of the pre-obtained potato late blight data, constructing a generative pre-training model for time series prediction based on GPT2, and introducing instance normalization on the basis of the layer normalization of the generative pre-training model; channel independence and a blocking technology are adopted to adapt to the processing capability of the pre-training model; a time sequence alignment stage is introduced, and a coding layer and an output layer are optimized; an autoregression training method which is the same as the original pre-training method of the model is adopted in the time sequence alignment stage; two-stage fine tuning is adopted in the fine tuning stage of the generative pre-training model, linear detection is carried out firstly, then comprehensive fine tuning is carried out, a loss function is determined, and prediction of potato late blight is achieved. The invention provides a time sequence alignment method, so that the generative pre-training model can fully understand and utilize multi-dimensional time sequence data, and occurrence of potato late blight is effectively predicted.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the fields of agricultural information technology and artificial intelligence, and specifically relates to a potato late blight prediction method based on a generative pre-training model. Background Art

[0002] In the field of potato late blight prediction, existing identical or similar products mainly include prediction methods based on traditional statistical models and expert systems. Methods based on traditional statistical models usually use historical disease data and meteorological conditions to make predictions through statistical means such as correlation analysis and regression models, such as linear regression and multivariate regression models. Expert systems use predefined rules and knowledge bases to make disease predictions, relying on the experience and knowledge of agricultural experts to determine the possibility of disease occurrence based on input meteorological and crop growth data.

[0003] Traditional statistical models and expert systems can usually only process limited feature data and cannot fully utilize multi-dimensional time series data. Especially when faced with complex meteorological and crop growth data, the prediction effects of these methods are often not ideal and cannot accurately reflect the dynamic changes in disease occurrence. Expert systems rely on predefined rules and knowledge bases, and traditional statistical models rely on fixed regression relationships. These methods lack prediction accuracy and reliability under different environmental conditions and emerging disease patterns. The models are difficult to adapt to different agricultural ecosystems and lack the flexibility to respond to new situations. Although expert systems can provide valuable predictions in some cases, their rigid rules and insufficient model flexibility make it difficult to adapt to different environments and new disease patterns. The agricultural environment is complex and changeable, and the occurrence of diseases is affected by many factors. Existing models are difficult to provide timely and accurate disease warnings in a rapidly changing environment.

[0004] At present, there is no potato late blight prediction method developed using generative pre-training models. Generative pre-training models have unique advantages in processing complex time series data and multi-dimensional features, and are expected to overcome the limitations of existing technologies and provide more efficient and accurate disease prediction solutions. Therefore, the development of a potato late blight prediction method based on a generative pre-training model has important application value and prospects. Summary of the invention

[0005] Purpose of the invention: The present invention provides a potato late blight prediction method based on a generative pre-training model, which can effectively predict the occurrence of potato late blight.

[0006] Technical solution: The potato late blight prediction method based on a generative pre-training model described in the present invention comprises the following steps:

[0007] (1) Preprocess the potato late blight data obtained in advance, mainly including data cleaning and meteorological feature construction;

[0008] (2) Based on GPT2, a generative pre-training model for time series prediction is built. Instance normalization is introduced based on the layer normalization of the generative pre-training model. Channel independence and block technology are used to adapt to the processing capacity of the pre-training model.

[0009] (3) Introducing a time series alignment phase and optimizing its encoding layer and output layer; in the time series alignment phase, the same autoregressive training method as the original pre-training of the model is used to align the pre-trained generative model with the segmented time series data;

[0010] (4) In the fine-tuning stage of the generative pre-training model, a two-stage fine-tuning is adopted, firstly linear detection and then comprehensive fine-tuning to determine the loss function and realize the prediction of potato late blight.

[0011] Furthermore, the data cleaning in step (1) includes processing missing values, deleting or converting unnecessary columns, and converting data types.

[0012] Furthermore, the meteorological characteristics construction implementation process in step (1) is as follows:

[0013] Date digitization, converting the date into the number of days since the beginning of the year;

[0014] The temperature fluctuation index measures the range of temperature fluctuation by calculating the difference between the highest and lowest temperatures of each day;

[0015] Precipitation: measured precipitation data are used;

[0016] Humidity, using the percentage value of relative humidity as the feature.

[0017] Furthermore, in step (2), based on the layer normalization of the generative pre-trained model, the instance normalization implementation process is introduced as follows:

[0018] For a given input time series sample Generate normalized time series samples based on instance normalization The sample has zero mean and unit standard deviation; in the model output structure, reversible instance normalization is used to improve prediction accuracy; the reversible instance normalization involves batch-specific instance normalization and subsequent denormalization, both of which share the same trainable affine transformation.

[0019] Furthermore, the implementation process of using channel independence and block technology to adapt the processing capability of the pre-trained model in step (2) is as follows:

[0020] The channel independence method converts the multivariate time series data of each meteorological variable, which was originally stored in the multi-channel dimension C, into multiple univariate time series data. That is, this conversion changes the dimension of the data from Convert to Each individual meteorological variable is now processed independently and merged into the batch size dimension;

[0021] The block technology is used to aggregate continuous time steps into a single block-based Token; the time dimension of the input sample is changed from T in Reduce to T p , where T p Indicates the number of blocks; at the same time, the feature dimension increases from 1 to P, where P represents the length of the block. Given a normalized time series sample Generate a series of blocks

[0022] Furthermore, the optimization implementation process of the coding layer in step (3) is as follows:

[0023] A one-dimensional convolutional layer is used as a new Token encoding layer to embed tokens

[0024] e token =Conv token (p)

[0025] For the position encoding layer, a trainable lookup table E is used pos To map the patch position, the generated position encoding

[0026] e pos =E pos (i)

[0027] in, An index representing the patch location; a hierarchical temporal encoding method with two-level aggregation is used to capture information at different time scales;

[0028] First-level aggregation, high-dimensional mapping of temporal attributes: For each temporal attribute, a trainable lookup table is configured. Each lookup table maps the corresponding temporal attribute to a high-dimensional space. Subsequently, these high-dimensional representations are summed to form a comprehensive temporal embedding that captures the characteristics of each time scale.

[0029] Second-level aggregation, representation of multiple timestamps within a patch: Considering that a patch contains multiple timestamps, each timestamp represents a different time window, a pooling method is used to integrate multiple timestamps within a patch;

[0030] Through the above two-level aggregation strategy, the final time embedding e is constructed. temp :

[0031] e temp =Pooling(Σ(E a (t a ))

[0032] Finally, the token embedding, position embedding, and time embedding are summed The final embedding z is obtained and then input into the pre-trained GPT block.

[0033] Furthermore, the optimization process of the output layer in step (3) is as follows:

[0034] The final embedding z is flattened and the processed embedding Where T p represents the length of the time series, and D is the dimension of the embedding; the resulting embedding z then passes through a linear output layer W tsa Transform it to map it back to the patched time series data format;

[0035] The flattened data is passed through a predefined linear output layer Processing is done to map the data from the embedding space to the target prediction space; a sigmoid activation function is applied to the output layer to ensure that the output value is in the range [0,1] and represents the probability of the potato late blight state; finally, a denormalization step with reversible instance normalization is used to process the rearranged data and adjust the scale and distribution of the data to make it closer to the original data distribution during training.

[0036] Furthermore, it is characterized in that the linear detection implementation process in step (4) is as follows:

[0037] All layers in the pre-trained model are frozen, and only the final linear output layer is updated. Specifically, first, all weights from the input layer to the penultimate layer are frozen so that the parameters of these layers remain unchanged during training. Then, only the weights and bias of the linear output layer are updated, and training is performed using a dataset specific to the potato late blight prediction task. The goal is to minimize the classification error and make the model output as close to the true label as possible.

[0038] Furthermore, it is characterized in that the overall fine-tuning stage in step (4) is implemented as follows:

[0039] Unfreeze all layers of the model, including the previously frozen input layer to the penultimate layer, so that the parameters of these layers can be updated during training; use a lower learning rate for training so that the model does not destroy the learned effective feature representations when slightly adjusting parameters; every parameter of the model from the input layer to the output layer participates in the training and is optimized using the dataset of the new task. The training goal is still to minimize the classification error.

[0040] Furthermore, the loss function in step (4) is a cross entropy loss function, and by minimizing the cross entropy loss, the model reduces the prediction error.

[0041] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects:

[0042] This paper introduces the generative pre-training model (GPT) for the first time in potato late blight prediction, using its powerful data processing and language understanding capabilities to effectively analyze complex time series data and improve the accuracy and efficiency of prediction;

[0043] The present invention proposes a time series alignment method specifically for meteorological, crop growth and disease data, which enables the generative pre-trained model to fully understand and utilize multi-dimensional time series data, thereby improving the model's predictive ability;

[0044] The present invention optimizes the architecture of the generative pre-training model based on the characteristics of agricultural data, especially in terms of meteorological and disease data input at multiple time scales; through a specific fine-tuning strategy, the model can maintain high-precision prediction even under limited data conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a schematic diagram of the structure of the generative pre-training model proposed in the present invention;

[0046] Figure 2 This is a schematic diagram of the secondary aggregation coding proposed by the present invention;

[0047] Figure 3 The flowchart for two-stage fine-tuning. DETAILED DESCRIPTION

[0048] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0049] The present invention proposes a potato late blight prediction method based on a generative pre-trained model. First, meteorological and disease data are used for fine data preprocessing, a targeted feature sequence is constructed, a time series alignment operation is performed, and the preprocessed sequence data is used as a question-answering knowledge base. Subsequently, these sequence data are input into the pre-trained generative pre-trained model for further model fine-tuning and performance evaluation. Through this process, a final model that can effectively predict the occurrence of potato late blight is formed. The prediction framework combines the fine-tuning strategies of time series prediction and generative pre-trained models, optimizes the information extraction and processing process, and ensures the accuracy and practicality of disease prediction. Specifically, the following steps are included:

[0050] Step 1: Collect disease data on potato late blight, including meteorological information, crop growth data, and disease occurrence records. Preprocess the data and divide it into training set, validation set, and test set to provide basic data support for model training and testing.

[0051] Step 1.1: Data cleaning.

[0052] In the preprocessing stage, the main focus is to ensure the completeness, accuracy and consistency of the data, and to provide a clean and usable data set for subsequent analysis and model building. This includes handling missing values, deleting or converting unnecessary columns, and appropriately converting data types to enhance the expressiveness and convenience of the data.

[0053] Data scale integration can effectively improve the efficiency of data processing and analysis, merging the year, month M, and day D into a date and time object Date, which not only facilitates subsequent Y data processing and analysis, but also helps to use time series analysis methods or other time-based analysis techniques.

[0054] Through the above steps, the preprocessing stage not only completes the missing data, but also transforms and optimizes the data structure through appropriate mathematical methods, enhancing the data expression ability and the convenience of analysis. These preprocessing measures provide a solid foundation for subsequent model construction and data analysis, ensuring the quality of the data and the reliability of the analysis results.

[0055] Step 1.2: Construct meteorological features.

[0056] In the meteorological data feature construction stage, the goal is to use existing data to construct new features and convert data formats to reveal hidden patterns and trends in the data, thereby providing strong support for model training.

[0057] The digitization of dates can effectively improve the practical value of date information. By converting dates into the number of days since the beginning of the year, this continuous numerical representation enables the model to identify long-term trends or seasonal changes, thereby improving the utilization efficiency of time information.

[0058] When constructing the features of meteorological data, the present invention considers the processing methods of several key meteorological indicators to optimize the performance of the potato late blight prediction model:

[0059] 1) Temperature Fluctuation Index: This index measures the temperature fluctuation range by calculating the difference between the highest and lowest temperatures of each day. Known as the daily temperature difference, this quantitative index not only intuitively reflects the daily temperature fluctuation range, but is also crucial for understanding how temperature affects the development of late blight.

[0060] 2) Precipitation: The model directly uses measured precipitation data (unit: mm). The detailed information on precipitation is suitable for scenarios that require accurate analysis of the impact of precipitation, helping the model to accurately predict the potential risk of late blight under different precipitation conditions.

[0061] 3) Humidity: Different from the commonly used qualitative humidity level, the present invention chooses to use the percentage value of relative humidity as a feature. This method enables the model to process and understand humidity changes more accurately, thereby improving the accuracy and reliability of the prediction.

[0062] Through these feature engineering methods, not only can the model be provided with richer and more explanatory data, but it can also help the model better identify and learn complex patterns and relationships in the data, so as to improve the model's prediction accuracy and generalization ability.

[0063] Step 2: Build a generative pre-trained model for time series prediction based on GPT2. Introduce instance normalization based on the layer normalization of the generative pre-trained model. Adopt channel independence and block technology to adapt to the processing capacity of the pre-trained model.

[0064] In the field of time series, self-supervised learning has become an important method in representation learning. Although Transformer is widely regarded as the main candidate model for end-to-end time series analysis, models based on CNN (convolutional neural network) or RNN (recurrent neural network) are generally still the preferred architectures in time series self-supervised learning. However, Transformer is able to model long-distance dependencies and capture complex patterns well, which fits the complex sequential relationships of time series data.

[0065] Existing large-scale generative pre-trained models are usually pre-trained on a broad general language corpus, which limits their application in specific fields because they cannot directly learn expertise beyond the scope of general language usage. To address this problem, the present invention introduces a time series alignment stage, which aims to adjust the large-scale generative pre-trained model so that it can better match the characteristics of time series data. Considering that the structure of the generative pre-trained model is a causal-based generative pre-trained model, the present invention adopts the same autoregressive training method as its original pre-training in this alignment stage to maintain the consistency of the method and maximize the model's adaptability and predictive ability to time series data.

[0066] like Figure 1As shown, it is a schematic diagram of the generative pre-trained model structure for time series prediction. In the potato late blight prediction task, the model needs to effectively process a large amount of time series data and capture the correlation between weather and climate factors and the disease. The present invention regards it as one of the methods of self-supervised learning, evaluates the table representation learning ability of the model, and processes the time series into a mode that can be trained using a generative pre-trained model. The generative pre-trained model structure for time series prediction includes two stages: (a) the time series alignment stage, using the autoregressive method to align the pre-trained generative pre-trained model with the segmented time series data. (b) The model prediction fine-tuning stage, first only unfreezes the output layer for linear detection, and then performs comprehensive fine-tuning, that is, unfreezes all layers and PEFT components in the model.

[0067] 2.1: Normalization processing.

[0068] Normalization is crucial to adjust the pre-trained model to achieve stable performance when adapting to different modalities. Based on the layer normalization of the generative pre-trained model, this paper introduces instance normalization to improve the consistency and reliability when processing diverse time series datasets.

[0069] For a given input time series sample The present invention applies instance normalization (IN) to generate normalized time series samples This sample has zero mean and unit standard deviation.

[0070] x normed =IN(x in )

[0071] In addition, in the model output structure, the present invention adopts reversible instance normalization (RevIN) to improve prediction accuracy. RevIN involves batch-specific instance normalization and subsequent denormalization, both of which share the same trainable affine transformation. The additional denormalization step solves the distribution shift problem between training and test data.

[0072] In the time series tokenization step, the present invention applies the normalization of RevIN:

[0073] p=patching(CI(RevIN norm (x in )))

[0074] When predicting potato late blight, this normalization process ensures that data collected from different meteorological stations or different time periods can be analyzed and model trained under unified standards, improving the model's ability to process time series data, thereby enhancing the accuracy of the prediction and the generalization ability of the model.

[0075] 2.2: Channel independence and blocking technology.

[0076] In the pre-trained generative pre-trained model, the size of the context window is usually small, which is suitable for conventional natural language processing tasks. However, for long-term time series prediction, this window size is not enough. In the potato late blight prediction experiment of the present invention, the prediction length far exceeds the limitation of the generative pre-trained model. In order to meet this challenge, when processing multivariate meteorological data, the present invention adopts channel independence and block technology to adapt to the processing capacity of the pre-trained model.

[0077] The channel independence method converts the multivariate time series data of each meteorological variable, such as temperature, humidity, precipitation, etc., which is originally stored in the multi-channel dimension C, into multiple univariate time series data. Specifically, this conversion changes the dimension of the data from Convert to Each individual meteorological variable is now processed independently and merged into the batch size dimension to simplify the subsequent data processing process.

[0078] Subsequently, a block-based technique is used to aggregate consecutive time steps into a single block-based token. This step effectively converts the time dimension of the input sample from T to in Reduce to T p , where T p Indicates the number of blocks. At the same time, the feature dimension increases from 1 to P, where P represents the length of the block. Given a normalized time series sample The present invention first applies channel independence (CI) and then performs block division to generate a series of blocks

[0079] Potato late blight prediction requires analysis of long-term meteorological and disease development data to predict the occurrence and development trend of the disease. This method not only effectively solves the problem of context window size limitation, but also significantly reduces the time and space complexity of the model processing time series data, enabling the present invention to more effectively predict the occurrence and development of potato late blight.

[0080] Step 3: Optimize the encoding layer and output layer of the time series alignment phase of the generative pre-trained model.

[0081] In order to preserve the temporal characteristics of potato late blight meteorological data, the present invention retains most of the structures in the time series alignment stage and optimizes the structures of the encoding layer and the output layer.

[0082] 3.1: Coding layer optimization.

[0083] For the potato late blight prediction task, one of the main challenges faced by the present invention in the process of adapting the generative pre-training model to process time series data is that the original Token encoding layer is designed for text, which becomes inapplicable due to modality mismatch. In order to solve the difficulty of processing multi-scale time information, the present invention designs a new time encoding layer.

[0084] In standard natural language processing practice, token encoding usually uses a trainable lookup table to map scalar tokens to high-dimensional space. However, the time series data of the present invention is in vector form, so this method is no longer applicable. Therefore, the present invention abandons the token encoding layer in the original model and instead uses a one-dimensional convolutional layer as the new token encoding layer. This design makes token embedding

[0085] e token =Conv token (p)

[0086] Compared with linear layers, convolutional layers are chosen due to their superiority in preserving local semantic information within time series data.

[0087] For the position encoding layer, the present invention uses a trainable lookup table E pos To map the patch position, the generated position encoding

[0088] e pos =E pos (i)

[0089] in, An index representing the location of the patch.

[0090] In order to capture information of different time scales when processing time series data with rich time attributes, this paper adopts a hierarchical time encoding method. The method is divided into two levels of aggregation to adapt to the multi-scale time attributes in time series data and effectively integrate this information to improve the performance of the prediction model.

[0091] like Figure 2 As shown, the first level aggregation, high-dimensional mapping of time attributes: For each time attribute (1 day, 3 days, 5 days, 7 days), the present invention configures a trainable lookup table, such as E 1day ,E 3day ,E 5day ,E 7day Each lookup table maps the corresponding temporal attribute to a high-dimensional space, through which the model can understand the subtle changes in temporal data in a more detailed way. Subsequently, these high-dimensional representations are summed to form a comprehensive temporal embedding that captures the characteristics of each time scale.

[0092] Second-level aggregation, representation of multiple timestamps in a patch: Considering that a patch contains multiple timestamps, each of which represents a different time window (1 day, 3 days, 5 days, 7 days), the present invention adopts a pooling method to integrate multiple timestamps in a patch. Through the above two-level aggregation strategy, the present invention constructs the final time embedding temp :

[0093] e temp =Pooling(∑(E a (t a ))

[0094] Finally, the token embedding, position embedding, and time embedding are summed The final embedding is obtained and then fed into the pre-trained GPT block.

[0095] 3.2: Output layer optimization.

[0096] In the output layer, the present invention converts the high-dimensional final embedding z in the model into a future data format that can be used for actual prediction. First, the embedding z is flattened to convert the multi-dimensional feature vector into one dimension so that it can be processed by the subsequent linear layer. The processed embedding Where T p represents the length of the time series (in patches), and D is the dimension of the embedding. This embedding contains a high-level representation of the time series data after being processed by the deep learning model, providing the necessary information for the next step of prediction. The resulting embedding z is then passed through a linear output layer W tsa Transform to map it back to the patched time series data format. The role of this output layer is to transform the embedding space into a specific prediction result based on the representation learned by the model, with each output value corresponding to one or more time points in the original time series.

[0097] The flattened data is passed through a predefined linear output layer Processing, this step is mainly to map the data from the embedding space to the target prediction space. In order to meet the needs of the binary classification task, the present invention applies a sigmoid activation function in the output layer to ensure that the output value is in the range of [0,1] and represents the probability of the potato late blight state. Finally, the denormalization step of reversible instance normalization (RevIN) is used to process the rearranged data, adjust the scale and distribution of the data, and make it closer to the original data distribution during training, thereby enhancing the prediction accuracy of the model in practical applications.

[0098] Step 4: Fine-tune the model for the time prediction task to achieve the prediction of potato late blight.

[0099] In the potato late blight prediction task, after aligning the generative pre-trained model with the time series data, the model will inherit the training weights included in these encoding layers to the prediction fine-tuning stage. When fine-tuning the backbone model for the prediction task, there are two main training strategies to choose from: comprehensive fine-tuning and linear probing. In addition, the two-stage fine-tuning: linear probing first, then comprehensive fine-tuning, is also suitable for the generative pre-trained model fine-tuning task.

[0100] 4.1: Loss function selection.

[0101] When developing a prediction model for potato late blight, choosing a suitable loss function is key because it directly affects the model training effect and the final prediction accuracy. In the research of the present invention, the present invention uses cross entropy loss to measure the difference between the predicted value and the actual value. Cross entropy is a commonly used loss function, especially suitable for binary classification tasks. By minimizing the cross entropy loss, the model reduces the prediction error. In practical applications, a lower cross entropy loss value indicates that the model can more accurately predict the dynamic changes of the disease, thereby helping farmers to take effective management and prevention measures in a timely manner.

[0102] 4.2: Two-stage fine-tuning

[0103] Two-stage fine-tuning combines the advantages of linear detection and comprehensive fine-tuning, and is a model fine-tuning strategy that takes into account both efficiency and accuracy. Figure 3 As shown, linear exploration is first performed to quickly adapt to the new task, and then comprehensive fine-tuning is performed on this basis to optimize the overall performance of the model.

[0104] In the linear detection stage, the present invention first freezes all layers in the pre-trained model and only updates the last linear output layer. This step specifically includes the following operations: first, freeze all weights from the input layer to the penultimate layer so that the parameters of these layers remain unchanged during the training process; then only update the weights and bias of the linear output layer, and train using a dataset specific to the potato late blight prediction task. The goal is to minimize the classification error and make the model output as close to the true label as possible.

[0105] A higher learning rate is used in the linear detection phase to quickly evaluate the adaptability of the pre-trained model to the new task. The training time in this phase is shorter, which helps save computing resources. Through linear detection, a preliminary version of the model is obtained, which has a certain degree of task adaptability, and the feature representation ability is also preliminarily verified.

[0106] After completing the linear detection, the comprehensive fine-tuning phase begins. At this point, all layers of the model are unfrozen so that they can be fully optimized on the new task data. The specific steps are as follows: Unfreeze all layers, including the previously frozen input layer to the penultimate layer, so that the parameters of these layers can be updated during the training process; Use a lower learning rate for training, which helps the model avoid destroying the learned effective feature representations when fine-tuning the parameters. The appropriate learning rate selection requires finding a balance between retaining pre-training knowledge and adapting to the needs of the new task. In the comprehensive fine-tuning phase, every parameter of the model from the input layer to the output layer participates in the training and is optimized using the dataset of the new task. The training goal is still to minimize the classification error, but more emphasis is placed on improving the overall model performance. The comprehensive fine-tuning phase ensures that the model can fully utilize the knowledge learned during the pre-training process and achieve the best performance through optimization for the new task. Although the training time in this phase is relatively long, it can significantly improve the model's predictive ability and generalization performance.

[0107] In the potato late blight prediction task, in order to verify the effectiveness of the present invention, the performance of multiple models in different time windows was evaluated for the four evaluation indicators of accuracy, precision, recall, and F1 value. The present invention adopts a generative pre-trained model series model as the main model (Potato-GPT), and regards other models as baseline models (CARAH, SVM, ARIMA, LSTM, and Informer). Through the four time windows of 1 day, 3 days, 5 days, and 7 days, the performance of each model in predicting late blight was compared, and their accuracy and prediction ability in the short-term and medium-term time ranges were comprehensively evaluated. Through this evaluation method, the ability of each model to predict potato late blight in different time ranges can be fully understood. Table 1 shows the performance differences of potato late blight prediction models at different time window scales.

[0108] Table 1 Performance evaluation of potato late blight prediction models at different time window scales

[0109]

[0110] The Potato-GPT model has a higher accuracy than most other models in all time windows; in particular, the accuracy reaches 0.8746 in the 5-day forecast window. This shows that the Potato-GPT model is able to maintain high accuracy over different time spans. The CARAH model has a consistent accuracy of about 0.8324 in all time windows, which is stable but slightly lower than the best performance of the Potato-GPT model. The SVM and ARIMA models have low accuracies in all time windows, 0.6975 and 0.6833, respectively, indicating that they are not effective when processing time series data. The LSTM and Informer models perform well in short time windows (1 day), but the accuracy fluctuates as the time window increases, especially the LSTM has a significant decrease in accuracy in the 5-day and 7-day windows.

[0111] The Potato-GPT model has excellent accuracy in all time windows, especially in the 5-day forecast window, where the accuracy reaches 0.8915, which is significantly higher than other models. This means that the Potato-GPT model has higher accuracy in predicting positive samples. The CARAH model is consistent with its accuracy in terms of accuracy, which remains between 0.8324 and 0.8483, and performs stably but not as well as the Potato-GPT model. The SVM and ARIMA models have low accuracy of 0.7012 and 0.6850, respectively, in all time windows, further illustrating their limitations in dealing with complex time series forecasting tasks. The LSTM model has a high accuracy in the 1-day window, but it decreases in the 3-day and 7-day windows. The Informer model has a certain advantage in the long-term window (7 days) with an accuracy of 0.8314.

[0112] The Potato-GPT model has excellent recall in all time windows, especially in the 5-day forecast window, where the recall reaches 0.8564. This shows that the Potato-GPT model can effectively capture more positive samples. The CARAH model has a consistent recall of about 0.8324 in different time windows, which is stable but slightly lower than the best performance of the Potato-GPT model. The SVM and ARIMA models have low recall in all time windows, 0.6921 and 0.6805 respectively, indicating that they are not effective in capturing positive samples. The LSTM model has a high recall in the 1-day window, but fluctuates in other time windows.

[0113] The Potato-GPT model has excellent F1 scores in all time windows, especially in the 5-day forecast window, where the F1 score reaches 0.8472. This shows that the Potato-GPT model has obvious advantages in balancing precision and recall. The F1 scores of the CARAH model in different time windows are consistent, ranging from about 0.7885 to 0.8324, which is stable but not as good as the Potato-GPT model. The F1 scores of the SVM and ARIMA models are 0.6842 and 0.6671, respectively, which are low in all time windows, further illustrating their limitations in dealing with complex time series forecasting tasks. The F1 scores of the LSTM and Informer models perform well in short time windows (1 day), but fluctuate greatly in long time windows, especially the F1 scores of the LSTM in the 5-day and 7-day windows drop significantly.

[0114] From the detailed analysis of Table 1, it can be seen that the Potato-GPT model outperforms other models in terms of accuracy, precision, recall, and F1 score, especially in long-term prediction tasks. This proves the effectiveness and applicability of the Potato-GPT model in the potato late blight prediction task. The Potato-GPT model can not only maintain high performance in a short-term prediction window, but also stably provide excellent prediction results in a long-term prediction window, showing its strong ability in processing time series data.

[0115] The present invention solves the problem of using meteorological and disease data in generative pre-training models through a time series alignment method, making it adaptable to the training requirements of the model; by optimizing the generative pre-training model architecture and adjusting the meteorological and disease data inputs at multiple time scales, the adaptability of the model in potato late blight prediction is solved, providing an effective solution for potato late blight prediction.

Claims

1. A potato late blight prediction method based on a generative pre-training model, It is characterized in that The following steps are involved: (1) Preprocess the potato late blight data obtained in advance, mainly including data cleaning and meteorological feature construction; (2) Based on GPT2, a generative pre-training model for time series prediction is constructed. Instance normalization is introduced based on the layer normalization of the generative pre-training model. Channel independence and block technology are used to adapt to the processing capacity of the pre-training model. (3) Introducing the time series alignment stage and optimizing its encoding layer and output layer; in the time series alignment stage, the same autoregressive training method as the original pre-training of the model is used to align the pre-trained generative model with the segmented time series data; (4) In the fine-tuning stage of the generative pre-training model, a two-stage fine-tuning is adopted, firstly linear detection and then comprehensive fine-tuning to determine the loss function and realize the prediction of potato late blight.

2. A potato late blight prediction method based on a generative pre-training model according to claim 1, It is characterized in that The data cleaning described in step (1) includes processing missing values, deleting or converting unnecessary columns, and converting data types.

3. A potato late blight prediction method based on a generative pre-training model according to claim 1, It is characterized in that The meteorological characteristics construction implementation process in step (1) is as follows: Date digitization, converting the date into the number of days since the beginning of the year; The temperature fluctuation index measures the temperature fluctuation range by calculating the difference between the highest and lowest temperatures of each day; Precipitation: measured precipitation data are used; Humidity, using the percentage value of relative humidity as the feature.

4. The potato late blight prediction method based on a generative pre-training model according to claim 1, It is characterized in that In step (2), based on the layer normalization of the generative pre-trained model, the instance normalization implementation process is introduced as follows: For a given input time series sample Generate normalized time series samples based on instance normalization The sample has zero mean and unit standard deviation; in the model output structure, reversible instance normalization is used to improve prediction accuracy; the reversible instance normalization involves batch-specific instance normalization and subsequent denormalization, both of which share the same trainable affine transformation.

5. The potato late blight prediction method based on a generative pre-training model according to claim 1, It is characterized in that The implementation process of using channel independence and block technology to adapt the processing capacity of the pre-trained model in step (2) is as follows: The channel independence method converts the multivariate time series data of each meteorological variable, which was originally stored in the multi-channel dimension C, into multiple univariate time series data. That is, this conversion changes the dimension of the data from Convert to Each individual meteorological variable is now processed independently and merged into the batch size dimension; The block technology is used to aggregate continuous time steps into a single block-based Token; the time dimension of the input sample is changed from T in Reduce to T p , where T p Indicates the number of blocks; at the same time, the feature dimension increases from 1 to P, where P represents the length of the block. Given a normalized time series sample Generate a series of blocks 6. The potato late blight prediction method based on a generative pre-training model according to claim 1, It is characterized in that The optimization process of the coding layer in step (3) is as follows: A one-dimensional convolutional layer is used as a new Token encoding layer to embed tokens e token =Conv token (p) For the position encoding layer, a trainable lookup table E is used pos To map the patch position, the generated position encoding yes pos =E pos (i) in, An index representing the patch location; a hierarchical temporal encoding method with two-level aggregation is used to capture information at different time scales; First-level aggregation, high-dimensional mapping of temporal attributes: For each temporal attribute, a trainable lookup table is configured. Each lookup table maps the corresponding temporal attribute to a high-dimensional space. Subsequently, these high-dimensional representations are summed to form a comprehensive temporal embedding that captures the characteristics of each time scale. Second-level aggregation, representation of multiple timestamps within a patch: Considering that a patch contains multiple timestamps, each timestamp represents a different time window, a pooling method is used to integrate multiple timestamps within a patch; Through the above two-level aggregation strategy, the final time embedding e is constructed. temp : e temp =Pooling(∑(E a (t a )) Finally, the token embedding, position embedding, and time embedding are summed The final embedding z is obtained and then input into the pre-trained GPT block.

7. The potato late blight prediction method based on a generative pre-training model according to claim 1, It is characterized in that The optimization process of the output layer in step (3) is as follows: The final embedding z is flattened and the processed embedding Where T p represents the length of the time series, and D is the dimension of the embedding; the resulting embedding z then passes through a linear output layer W tsa Transform it to map it back to the patched time series data format; The flattened data is passed through a predefined linear output layer Processing is done to map the data from the embedding space to the target prediction space; a sigmoid activation function is applied to the output layer to ensure that the output value is in the range [0,1] and represents the probability of the potato late blight state; finally, a denormalization step with reversible instance normalization is used to process the rearranged data and adjust the scale and distribution of the data to make it closer to the original data distribution during training.

8. The potato late blight prediction method based on a generative pre-training model according to claim 1, It is characterized in that The linear detection implementation process of step (4) is as follows: Freeze all layers in the pre-trained model and only update the last linear output layer. Specifically, first freeze all weights from the input layer to the penultimate layer so that the parameters of these layers remain unchanged during training; then update only the weights and bias of the linear output layer and train using a dataset specific to the potato late blight prediction task; The goal is to minimize the classification error so that the model output is as close to the true label as possible.

9. The potato late blight prediction method based on a generative pre-training model according to claim 1, It is characterized in that The implementation process of the comprehensive fine-tuning stage described in step (4) is as follows: Unfreeze all layers of the model, including the previously frozen input layer to the penultimate layer, so that the parameters of these layers can be updated during training; use a lower learning rate for training so that the model avoids destroying the learned effective feature representation when fine-tuning parameters; Every parameter of the model from the input layer to the output layer is involved in the training and optimized using the data set of the new task. The training goal is still to minimize the classification error.

10. The potato late blight prediction method based on a generative pre-training model according to claim 1, It is characterized in that The loss function in step (4) is a cross entropy loss function, which reduces the prediction error of the model by minimizing the cross entropy loss.

Citation Information

Patent Citations

  • Semi-supervised few-sample time sequence anomaly detection and classification system and method based on fine tuning guided by pre-training model

    CN116186606A

  • Residual life prediction method based on convolutional neural network and long short-term memory network

    CN117010263A

  • Knowledge grap pre-training method based on structural context infor

    US20240177047A1

Cited By

  • Prediction method and device of time series data and electronic equipment

    CN119692561A