Universal time Transform model zero sample prediction training method

Abstract

The invention belongs to the technical field of universal time Transform models, and particularly relates to a universal time Transform model zero sample prediction training method which comprises the following specific steps: S1, data preprocessing and enhancement: firstly collecting and integrating time series data of multiple fields, then performing feature engineering on the time series data, and then enhancing the data; s2, improving a model structure and a training strategy: firstly, adding an adaptive weight adjustment mechanism in an attention module of a universal time Transform model; through the multi-field data fusion and data enhancement technology, types and features of training data are enriched, so that the model can learn wider data modes and laws; therefore, the model can be better generalized when facing zero sample prediction tasks in different fields, the prediction accuracy and stability are improved, and the overfitting problem caused by single data is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of universal temporal Transformer models, and in particular to a zero-shot prediction training method for universal temporal Transformer models. Background Art

[0002] With the continuous development of artificial intelligence technology, time series data prediction has a wide range of application needs in many fields, including finance, healthcare, and meteorology. The Universal Time Transformer model has shown great potential in processing time series data. Zero-shot prediction training enables the model to make effective predictions even when it has never seen specific data patterns, further expanding the model's application scenarios. However, the following issues arise during the implementation of zero-shot prediction training for the Universal Time Transformer model:

[0003] 1. Data diversity and generalization issues: In practical applications, time series data from different fields exhibit distinct characteristics and patterns. For example, in the financial sector, stock price time series data is influenced by multiple factors, such as the market macroeconomic environment and company performance, resulting in complex fluctuations and a high degree of randomness. Meanwhile, temperature time series data from the meteorological sector exhibits significant seasonality and cyclicality. If training data comes only from a single field or type, the model will struggle to generalize and accurately capture data characteristics when performing zero-shot predictions on data from other fields, leading to large prediction errors.

[0004] 2. Model adaptability to unknown patterns: During training, the Generalized Temporal Transformer model learns the patterns and regularities in the training data. When encountering new data patterns not previously seen in the training data, the model may be unable to effectively handle them. For example, in the medical field, the time series data for the onset of a new disease may exhibit unique patterns of transmission and development. If the model has not been exposed to similar patterns, zero-shot predictions of the disease's future development trends may result in errors, failing to provide accurate evidence for medical decision-making.

[0005] 3. Lack of effective evaluation metrics: Existing evaluation metrics are mostly designed for prediction tasks under supervised learning and lack specificity for evaluating zero-shot prediction training. For example, the traditional mean squared error (MSE) metric can effectively measure the error between the model's predicted value and the true value in supervised learning. However, in the zero-shot prediction scenario, due to the lack of training data directly related to the prediction task, the MSE metric cannot accurately reflect the model's predictive ability for unknown data and cannot fully evaluate the model's performance after zero-shot prediction training.

[0006] 4. Training efficiency and computing resource issues: The General Temporal Transformer model has a complex structure and a large number of parameters. Zero-shot prediction training requires processing a large amount of time series data. This results in a lengthy training process and a significant demand for computing resources. For example, when processing large-scale meteorological time series data for zero-shot training of long-term climate forecasts, ordinary computing equipment may not be able to meet the training requirements, and the training time is excessively long, seriously affecting the efficiency of model development and application.

[0007] Based on the above, a zero-shot prediction training method for a universal temporal Transformer model is invented. Summary of the Invention

[0008] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:

[0009] The zero-shot prediction training method for the universal temporal Transformer model includes the following specific steps:

[0010] S1, data preprocessing and enhancement: first collect and integrate time series data from multiple fields, then perform feature engineering on the time series data, and then enhance the data;

[0011] S2, improving the model structure and training strategy: First, we add an adaptive weight adjustment mechanism to the attention module of the universal temporal Transformer model. Then, we use meta-learning methods to learn common knowledge and patterns under different tasks and data distributions during model training. Finally, we introduce a generative adversarial network to build a generator and a discriminator.

[0012] S3, design dedicated evaluation indicators: first introduce domain knowledge evaluation indicators, then conduct cross-domain generalization ability assessment, and finally conduct real-time evaluation and feedback;

[0013] S4, optimizes training efficiency and computing resources: first compress and lightweight the model, then cache and reuse the model, and finally conduct distributed training and cloud computing.

[0014] As a preferred solution of the zero-shot prediction training method of the universal temporal Transformer model of the present invention, the specific steps of S1 are as follows:

[0015] S11, multi-domain data fusion: First, collect time series data from multiple fields, then integrate the data from different fields. During the integration process, the data will be standardized to have the same scale and distribution to facilitate model learning;

[0016] S12, feature engineering: perform feature engineering on the fused data, including feature extraction, feature selection, and feature combination;

[0017] S13, data enhancement: perform time translation, scaling, noise addition, cropping and splicing, and flip enhancement on the data.

[0018] As a preferred solution of the zero-shot prediction training method of the universal temporal Transformer model of the present invention, the specific steps of S13 are as follows:

[0019] S131, Time Shift: For a given time series data, it is moved forward or backward on the time axis by a certain time step to simulate different time starting points, allowing the model to learn the time-invariant characteristics of the data and enhance its adaptability to data patterns at different time stages;

[0020] S132, scaling: scaling up or down the values of time series data to adapt the model to the changing patterns of data at different scales, improve the ability to capture data features, and avoid over-reliance on a specific value range.

[0021] S133, Noise Addition: Add noise to time series data to simulate interference factors that may exist in actual data. By adding noise, the diversity of the data is increased, allowing the model to learn how to extract effective information from the noise during training, thereby improving the robustness of the model.

[0022] S134, cropping and splicing: cropping subsequences from the original time series data, and then randomly splicing the subsequences into a new time series, so that the model can learn the local features of the data and the combination relationship between different parts, and enhance the model's ability to understand complex data structures;

[0023] S135, Flip: Flip time series data upside down or in reverse order to help the model learn the symmetric characteristics and reverse change patterns of the data, and improve the model's comprehensive understanding of the data pattern.

[0024] As a preferred solution of the zero-shot prediction training method of the universal temporal Transformer model of the present invention, the specific steps of S2 are as follows:

[0025] S21, introduces an improved attention mechanism: In the attention module of the universal temporal Transformer model, an adaptive weight adjustment mechanism is added to dynamically adjust the attention weights of different time steps and feature dimensions according to the characteristics of the input data, allowing the model to better focus on key information;

[0026] S22, Meta-learning Assisted Training: Meta-learning is used to learn common knowledge and patterns across different tasks and data distributions during model training. This allows the model to grasp the common characteristics and changing patterns of the data by training on multiple different types of zero-shot prediction tasks, improving its adaptability to unknown patterns.

[0027] S23, Adversarial Training: Introduce a generative adversarial network to construct a generator and a discriminator. The generator generates fake data similar to real time series data, and the discriminator is responsible for distinguishing the generated fake data from real data. During the training process, the generator and the discriminator perform adversarial training to continuously improve the quality of the data generated by the generator and the discrimination ability of the discriminator.

[0028] As a preferred solution of the zero-shot prediction training method of the universal temporal Transformer model of the present invention, the specific steps of S3 are as follows:

[0029] S31, Introducing domain knowledge evaluation indicators: Design evaluation indicators based on professional knowledge of the application domain;

[0030] S32, Cross-domain Generalization Ability Evaluation: Design cross-domain evaluation indicators to measure the generalization ability of the model between data from different domains;

[0031] S33, Real-time Evaluation and Feedback: Establish a real-time evaluation system to collect prediction results and real data in real time during the actual prediction process of the model, and use the designed evaluation indicators for evaluation. Based on the evaluation results, problems in the model during the prediction process can be promptly identified and the information can be fed back to the model for real-time adjustment and optimization;

[0032] S34, Quantitative assessment of uncertainty: Use Bayesian methods and Monte Carlo dropout technology to quantitatively assess the uncertainty of model prediction results.

[0033] As a preferred solution of the zero-sample prediction training method of the universal temporal Transformer model of the present invention, the specific steps of S31 are:

[0034] S311, Determine the forecast target: Clarify the specific forecast target and clearly define the forecast time range and accuracy requirements to provide direction for the subsequent design of evaluation indicators;

[0035] S312, Mining Field Professional Indicators: Sort out existing professional evaluation indicators or concepts in the target field;

[0036] S313, Constructing composite evaluation indicators: Design composite evaluation indicators based on prediction objectives and professional indicators;

[0037] S314, determine the calculation method: for the design evaluation indicators, clarify the specific calculation method and formula;

[0038] S315, Set evaluation criteria: Set the qualification or excellence criteria for evaluation indicators based on the actual situation in the field and the requirements of the prediction task;

[0039] S316, Conduct experimental verification: Use historical data and different models to conduct experiments, calculate the designed evaluation index values, analyze whether the index can accurately reflect the predictive performance of the model in its field, and compare the performance of different models under the same evaluation index. Observe the consistency between the index value and the actual prediction effect to test the effectiveness of the index;

[0040] S317, Optimize and adjust indicators: Optimize the evaluation indicators based on the experimental verification results.

[0041] As a preferred solution of the zero-shot prediction training method of the universal temporal Transformer model of the present invention, the specific steps of S32 are as follows:

[0042] S321, Screening and Evaluation Fields: Comprehensively consider the characteristics, differences, and practical application value of time series data in different fields, and select multiple representative fields;

[0043] S322, Data Collection and Organization: First, for each selected domain, sufficient time series data is collected, including training data and test data. The training data is used to train the model in a specific domain, and the test data is used to evaluate the performance of the model in cross-domain scenarios. The collected data is then uniformly preprocessed to ensure data quality and consistency, facilitating subsequent model training and evaluation.

[0044] S323, accuracy-based indicator design:

[0045] S3231, Inter-domain prediction accuracy improvement: First calculate the prediction accuracy of the model in the target domain, then compare it with the prediction accuracy in the original domain to determine the accuracy improvement;

[0046] S3232, average cross-domain prediction accuracy: averages the prediction accuracy of the model in multiple target domains to obtain the average cross-domain prediction accuracy;

[0047] S324, similarity-based indicator design:

[0048] S3241, Prediction Result Feature Similarity: First, use the feature extraction algorithm to extract the feature vectors of the prediction results in different fields. Then, calculate the cosine similarity, Euclidean distance and other metrics between the feature vectors to evaluate the similarity of the model's prediction results in different fields.

[0049] S3242, Probability Distribution Similarity: Using statistical methods to compare the probability distribution differences of model prediction results in different fields;

[0050] S325, model training: Use the general temporal Transformer model to train on the selected original domain data and adjust the model parameters;

[0051] S326, cross-domain prediction: Apply the trained model to the test data of each target domain, perform zero-shot prediction, and obtain prediction results;

[0052] S327, indicator calculation: Based on the designed cross-domain generalization ability evaluation indicators, the prediction results of the model in each target field are calculated to obtain the specific values of each indicator.

[0053] As a preferred solution of the zero-shot prediction training method of the universal temporal Transformer model of the present invention, the specific steps of S34 are as follows:

[0054] S341, clarify the evaluation object and goal: First, clarify the specific content that needs to be quantitatively evaluated in the output results of the general time Transformer model zero-shot prediction. Then, set the goal of uncertainty quantitative evaluation based on the actual application scenario and needs;

[0055] S342, select uncertainty quantification method:

[0056] S3421, Bayesian-based approach: First, a reasonable prior distribution is set for the model parameters based on domain knowledge and historical data. Then, the Bayesian formula is used to calculate the posterior distribution of the model parameters in combination with the observed data. The prediction results are then sampled based on the posterior distribution to obtain multiple prediction samples. The distribution of these samples is then analyzed to quantify the uncertainty of the prediction results.

[0057] S3422, Monte Carlo dropout method: First, during the training and prediction process of the Universal Temporal Transformer model, the dropout mechanism is enabled. Some neurons are randomly dropped during each forward propagation to simulate different model structures. Then, multiple forward propagation predictions are performed on the same input data, with different neurons dropped during each prediction. Multiple prediction results are obtained. The uncertainty of the prediction results is quantified by calculating the statistics of these multiple predictions.

[0058] S343, Data Preparation: Prepare test data for uncertainty quantitative assessment, ensuring that the data characteristics are similar to and representative of data in actual application scenarios;

[0059] S344, executing the quantification method: calculating the prediction results of the model according to the selected uncertainty quantification method;

[0060] S345, Calculate uncertainty index: Calculate specific uncertainty index based on the output of the quantification method;

[0061] S346, Visualization: Present the uncertainty assessment results in an intuitive visual way;

[0062] S347, combined with prediction results analysis: Combine the uncertainty quantification results with the model's predicted values for analysis, so that in practical applications, the degree of trust in and use of the prediction results can be adjusted according to the size of the uncertainty.

[0063] As a preferred solution of the zero-shot prediction training method of the universal temporal Transformer model of the present invention, the specific steps of S4 are as follows:

[0064] S41, Model Compression and Lightweighting: Using model compression technology to reduce the number of parameters and computational complexity of the general temporal Transformer model;

[0065] S42, Model Caching and Reuse: Establish a model caching mechanism to cache trained model parameters and intermediate results. When a new zero-shot prediction training task is performed, the cache is first checked to see if there is a model or intermediate result similar to the current task. If so, the model or result can be reused to avoid repeated training and calculation.

[0066] S43, Distributed Training and Cloud Computing: Using a distributed training framework, training tasks are distributed to multiple computing nodes for parallel execution, speeding up training. At the same time, using a cloud computing platform, computing resources are dynamically allocated according to the needs of training tasks, avoiding resource waste and improving training efficiency.

[0067] Compared with existing technologies:

[0068] 1. Addressing data diversity and generalization issues: Through multi-domain data fusion and data augmentation techniques, the types and characteristics of training data are enriched, enabling the model to learn a wider range of data patterns and regularities. This helps the model better generalize when facing zero-shot prediction tasks in different fields, improves prediction accuracy and stability, and reduces overfitting caused by single data.

[0069] 2. Addressing the model's adaptability to unknown patterns: By introducing an improved attention mechanism and meta-learning-assisted training, the model can more flexibly capture key information in the data and learn common knowledge and patterns from multiple different types of zero-shot prediction tasks. This enhances the model's adaptability to unknown data patterns, enabling it to more quickly understand and process new time series patterns, thereby improving the reliability and effectiveness of predictions.

[0070] 3. Addressing the lack of effective evaluation metrics: By designing evaluation metrics that combine domain knowledge and cross-domain generalization capabilities, we can more comprehensively and accurately measure the performance of models after zero-shot prediction training. These metrics not only consider the accuracy of prediction results, but also combine domain-specific expertise and the model's generalization capabilities across different domains. This provides a more targeted and practical basis for model evaluation and optimization, helping to promote the development of models in practical applications.

[0071] 4. Regarding training efficiency and computing resource issues: By adopting model compression and lightweight technologies as well as distributed training and cloud computing, model training efficiency can be significantly improved and the demand for computing resources can be reduced. Model compression reduces the number of parameters and computational complexity while maintaining performance, accelerating training and inference. Distributed training and cloud computing fully utilize the resources of multiple computing nodes and cloud platforms, enabling parallel execution of training tasks and dynamic allocation of resources, avoiding resource waste, enabling models to complete training in a shorter time and be put into practical application more quickly. DETAILED DESCRIPTION

[0072] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below.

[0073] The present invention provides a zero-shot prediction training method for a universal temporal Transformer model, comprising the following specific steps:

[0074] S1, data preprocessing and enhancement: first collect and integrate time series data from multiple fields, then perform feature engineering on the time series data, and then enhance the data;

[0075] The specific steps of S1 are as follows:

[0076] S11, multi-domain data fusion: First, collect time series data from multiple fields, then integrate the data from different fields. During the integration process, the data will be standardized to have the same scale and distribution to facilitate model learning;

[0077] S12, feature engineering: perform feature engineering on the fused data, including feature extraction, feature selection, and feature combination;

[0078] S13, data enhancement: time shift, scaling, noise addition, cropping and splicing, and flip enhancement of data;

[0079] The specific steps of S13 are as follows:

[0080] S131, Time Shift: For a given time series data, it is moved forward or backward on the time axis by a certain time step to simulate different time starting points, allowing the model to learn the time-invariant characteristics of the data and enhance its adaptability to data patterns at different time stages;

[0081] S132, scaling: scaling up or down the values of time series data to adapt the model to the changing patterns of data at different scales, improve the ability to capture data features, and avoid over-reliance on a specific value range.

[0082] S133, Noise Addition: Add noise to time series data to simulate interference factors that may exist in actual data. By adding noise, the diversity of the data is increased, allowing the model to learn how to extract effective information from the noise during training, thereby improving the robustness of the model.

[0083] S134, cropping and splicing: cropping subsequences from the original time series data, and then randomly splicing the subsequences into a new time series, so that the model can learn the local features of the data and the combination relationship between different parts, and enhance the model's ability to understand complex data structures;

[0084] S135, Flip: Flip time series data upside down or in reverse order to help the model learn the symmetric characteristics and reverse change patterns of the data, improving the model's comprehensive understanding of the data pattern;

[0085] S2, improving the model structure and training strategy: First, we add an adaptive weight adjustment mechanism to the attention module of the universal temporal Transformer model. Then, we use meta-learning methods to learn common knowledge and patterns under different tasks and data distributions during model training. Finally, we introduce a generative adversarial network to build a generator and a discriminator.

[0086] The specific steps of S2 are as follows:

[0087] S21, introduces an improved attention mechanism: In the attention module of the universal temporal Transformer model, an adaptive weight adjustment mechanism is added to dynamically adjust the attention weights of different time steps and feature dimensions according to the characteristics of the input data, allowing the model to better focus on key information;

[0088] S22, Meta-learning Assisted Training: Meta-learning is used to learn common knowledge and patterns across different tasks and data distributions during model training. This allows the model to grasp the common characteristics and changing patterns of the data by training on multiple different types of zero-shot prediction tasks, improving its adaptability to unknown patterns.

[0089] S23, Adversarial Training: Introducing a generative adversarial network, we construct a generator and a discriminator. The generator generates fake data that is similar to real time series data, while the discriminator distinguishes between the fake data and real data. During the training process, the generator and the discriminator conduct adversarial training to continuously improve the quality of the data generated by the generator and the discriminator's discriminative ability.

[0090] S3, design dedicated evaluation indicators: first introduce domain knowledge evaluation indicators, then conduct cross-domain generalization ability assessment, and finally conduct real-time evaluation and feedback;

[0091] The specific steps of S3 are as follows:

[0092] S31, Introducing domain knowledge evaluation indicators: Design evaluation indicators based on professional knowledge of the application domain;

[0093] The specific steps of S31 are:

[0094] S311, Determine the forecast target: Clarify the specific forecast target and clearly define the forecast time range and accuracy requirements to provide direction for the subsequent design of evaluation indicators;

[0095] S312, Mining Field Professional Indicators: Sort out existing professional evaluation indicators or concepts in the target field;

[0096] S313, Constructing composite evaluation indicators: Design composite evaluation indicators based on prediction objectives and professional indicators;

[0097] S314, determine the calculation method: for the design evaluation indicators, clarify the specific calculation method and formula;

[0098] S315, Set evaluation criteria: Set the qualification or excellence criteria for evaluation indicators based on the actual situation in the field and the requirements of the prediction task;

[0099] S316, Conduct experimental verification: Use historical data and different models to conduct experiments, calculate the designed evaluation index values, analyze whether the index can accurately reflect the predictive performance of the model in its field, and compare the performance of different models under the same evaluation index. Observe the consistency between the index value and the actual prediction effect to test the effectiveness of the index;

[0100] S317, Optimize and adjust indicators: Optimize the evaluation indicators based on the experimental verification results;

[0101] S32, Cross-domain Generalization Ability Evaluation: Design cross-domain evaluation indicators to measure the generalization ability of the model between data from different domains;

[0102] The specific steps of S32 are as follows:

[0103] S321, Screening and Evaluation Fields: Comprehensively consider the characteristics, differences, and practical application value of time series data in different fields, and select multiple representative fields;

[0104] S322, Data Collection and Organization: First, for each selected domain, sufficient time series data is collected, including training data and test data. The training data is used to train the model in a specific domain, and the test data is used to evaluate the performance of the model in cross-domain scenarios. The collected data is then uniformly preprocessed to ensure data quality and consistency, facilitating subsequent model training and evaluation.

[0105] S323, accuracy-based indicator design:

[0106] S3231, Inter-domain prediction accuracy improvement: First calculate the prediction accuracy of the model in the target domain, then compare it with the prediction accuracy in the original domain to determine the accuracy improvement;

[0107] S3232, average cross-domain prediction accuracy: averages the prediction accuracy of the model in multiple target domains to obtain the average cross-domain prediction accuracy;

[0108] S324, similarity-based indicator design:

[0109] S3241, Prediction Result Feature Similarity: First, use the feature extraction algorithm to extract the feature vectors of the prediction results in different fields. Then, calculate the cosine similarity, Euclidean distance and other metrics between the feature vectors to evaluate the similarity of the model's prediction results in different fields.

[0110] S3242, Probability Distribution Similarity: Using statistical methods to compare the probability distribution differences of model prediction results in different fields;

[0111] S325, model training: Use the general temporal Transformer model to train on the selected original domain data and adjust the model parameters;

[0112] S326, cross-domain prediction: Apply the trained model to the test data of each target domain, perform zero-shot prediction, and obtain prediction results;

[0113] S327, indicator calculation: Based on the designed cross-domain generalization ability evaluation indicators, the prediction results of the model in each target domain are calculated to obtain the specific value of each indicator;

[0114] S33, Real-time Evaluation and Feedback: Establish a real-time evaluation system to collect prediction results and real data in real time during the actual prediction process of the model, and use the designed evaluation indicators for evaluation. Based on the evaluation results, problems in the model during the prediction process can be promptly identified and the information can be fed back to the model for real-time adjustment and optimization;

[0115] S34, Uncertainty Quantification Assessment: Use Bayesian methods and Monte Carlo dropout techniques to quantitatively assess the uncertainty of model prediction results;

[0116] The specific steps of S34 are as follows:

[0117] S341, clarify the evaluation object and goal: First, clarify the specific content that needs to be quantitatively evaluated in the output results of the general time Transformer model zero-shot prediction. Then, set the goal of uncertainty quantitative evaluation based on the actual application scenario and needs;

[0118] S342, select uncertainty quantification method:

[0119] S3421, Bayesian-based approach: First, a reasonable prior distribution is set for the model parameters based on domain knowledge and historical data. Then, the Bayesian formula is used to calculate the posterior distribution of the model parameters in combination with the observed data. The prediction results are then sampled based on the posterior distribution to obtain multiple prediction samples. The distribution of these samples is then analyzed to quantify the uncertainty of the prediction results.

[0120] S3422, Monte Carlo dropout method: First, during the training and prediction process of the Universal Temporal Transformer model, the dropout mechanism is enabled. Some neurons are randomly dropped during each forward propagation to simulate different model structures. Then, multiple forward propagation predictions are performed on the same input data, with different neurons dropped during each prediction. Multiple prediction results are obtained. The uncertainty of the prediction results is quantified by calculating the statistics of these multiple predictions.

[0121] S343, Data Preparation: Prepare test data for uncertainty quantitative assessment, ensuring that the data characteristics are similar to and representative of data in actual application scenarios;

[0122] S344, executing the quantification method: calculating the prediction results of the model according to the selected uncertainty quantification method;

[0123] S345, Calculate uncertainty index: Calculate specific uncertainty index based on the output of the quantification method;

[0124] S346, Visualization: Present the uncertainty assessment results in an intuitive visual way;

[0125] S347, combined with prediction results analysis: Combine the uncertainty quantification results with the model's predicted values for analysis, so that in practical applications, the degree of trust in and use of the prediction results can be adjusted according to the uncertainty level;

[0126] By setting up a quantitative uncertainty assessment, users can have a more comprehensive understanding of the reliability of the model's prediction results and avoid making wrong decisions due to blindly trusting the model's predicted values. In fields such as medical diagnosis and financial investment, which have extremely high requirements for prediction accuracy and reliability, by quantifying uncertainty, decision makers can formulate more reasonable decision-making strategies based on the degree of uncertainty. For example, in medicine, for disease prediction results with high uncertainty, doctors can take further inspection measures. In financial investment, investors can adjust their investment portfolios based on the uncertainty of the prediction to reduce risks. In addition, this evaluation step can also provide direction for model optimization, helping researchers to identify the causes of high uncertainty in model predictions and improve the model in a targeted manner.

[0127] S4, optimizes training efficiency and computing resources: first, compress and lightweight the model, then cache and reuse the model, and finally conduct distributed training and cloud computing;

[0128] The specific steps of S4 are as follows:

[0129] S41, Model Compression and Lightweighting: Using model compression technology to reduce the number of parameters and computational complexity of the general temporal Transformer model;

[0130] S42, Model Caching and Reuse: Establish a model caching mechanism to cache trained model parameters and intermediate results. When a new zero-shot prediction training task is performed, the cache is first checked to see if there is a model or intermediate result similar to the current task. If so, the model or result can be reused to avoid repeated training and calculation.

[0131] S43, Distributed Training and Cloud Computing: Using a distributed training framework, training tasks are distributed to multiple computing nodes for parallel execution, speeding up training. At the same time, using a cloud computing platform, computing resources are dynamically allocated according to the needs of training tasks, avoiding resource waste and improving training efficiency.

[0132] Although the present invention has been described above with reference to embodiments, various modifications may be made thereto and equivalent components may be substituted without departing from the scope of the present invention. In particular, as long as there are no structural conflicts, the various features of the embodiments disclosed herein may be combined with each other in any manner, and the omission of an exhaustive description of such combinations in this specification is solely for the sake of space and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A zero-shot prediction training method for a universal temporal Transformer model, characterized by: The specific steps are as follows: S1, data preprocessing and enhancement: first collect and integrate time series data from multiple fields, then perform feature engineering on the time series data, and then enhance the data; S2, improving the model structure and training strategy: First, we add an adaptive weight adjustment mechanism to the attention module of the universal temporal Transformer model. Then, we use meta-learning methods to learn common knowledge and patterns under different tasks and data distributions during model training. Finally, we introduce a generative adversarial network to build a generator and a discriminator. S3, design dedicated evaluation indicators: first introduce domain knowledge evaluation indicators, then conduct cross-domain generalization ability assessment, and finally conduct real-time evaluation and feedback; S4, optimizes training efficiency and computing resources: first compress and lightweight the model, then cache and reuse the model, and finally conduct distributed training and cloud computing.

2. The zero-shot prediction training method for the universal temporal Transformer model according to claim 1, characterized in that: The specific steps of S1 are as follows: S11, multi-domain data fusion: First, collect time series data from multiple fields, then integrate the data from different fields. During the integration process, the data will be standardized to have the same scale and distribution to facilitate model learning; S12, Feature Engineering: Perform feature engineering on the fused data. Including feature extraction, feature selection and feature combination; S13, data enhancement: perform time translation, scaling, noise addition, cropping and splicing, and flip enhancement on the data.

3. The zero-shot prediction training method for the universal temporal Transformer model according to claim 2, characterized in that: The specific steps of S13 are as follows: S131, Time Shift: For a given time series data, it is moved forward or backward on the time axis by a certain time step to simulate different time starting points, allowing the model to learn the time-invariant characteristics of the data and enhance its adaptability to data patterns at different time stages; S132, scaling: scaling up or down the values of time series data to adapt the model to the changing patterns of data at different scales, improve the ability to capture data features, and avoid over-reliance on a specific value range. S133, Noise Addition: Add noise to time series data to simulate interference factors that may exist in actual data. By adding noise, the diversity of the data is increased, allowing the model to learn how to extract effective information from the noise during training, thereby improving the robustness of the model. S134, cropping and splicing: cropping subsequences from the original time series data, and then randomly splicing the subsequences into a new time series, so that the model can learn the local features of the data and the combination relationship between different parts, and enhance the model's ability to understand complex data structures; S135, Flip: Flip time series data upside down or in reverse order to help the model learn the symmetric characteristics and reverse change patterns of the data, and improve the model's comprehensive understanding of the data pattern.

4. The zero-shot prediction training method for a universal temporal Transformer model according to claim 1, characterized in that: The specific steps of S2 are as follows: S21, introduces an improved attention mechanism: In the attention module of the universal temporal Transformer model, an adaptive weight adjustment mechanism is added to dynamically adjust the attention weights of different time steps and feature dimensions according to the characteristics of the input data, allowing the model to better focus on key information; S22, Meta-learning Assisted Training: Meta-learning is used to learn common knowledge and patterns across different tasks and data distributions during model training. This allows the model to grasp the common characteristics and changing patterns of the data by training on multiple different types of zero-shot prediction tasks, improving its adaptability to unknown patterns. S23, Adversarial Training: Introduce a generative adversarial network to construct a generator and a discriminator. The generator generates fake data similar to real time series data, and the discriminator is responsible for distinguishing the generated fake data from real data. During the training process, the generator and the discriminator perform adversarial training to continuously improve the quality of the data generated by the generator and the discrimination ability of the discriminator.

5. The zero-shot prediction training method for a universal temporal Transformer model according to claim 1, characterized in that: The specific steps of S3 are as follows: S31, Introducing domain knowledge evaluation indicators: Design evaluation indicators based on professional knowledge of the application domain; S32, Cross-domain Generalization Ability Evaluation: Design cross-domain evaluation indicators to measure the generalization ability of the model between data from different domains; S33, Real-time Evaluation and Feedback: Establish a real-time evaluation system to collect prediction results and real data in real time during the actual prediction process of the model, and use the designed evaluation indicators for evaluation. Based on the evaluation results, problems in the model during the prediction process can be promptly identified and the information can be fed back to the model for real-time adjustment and optimization; S34, Quantitative assessment of uncertainty: Use Bayesian methods and Monte Carlo dropout technology to quantitatively assess the uncertainty of model prediction results.

6. The zero-shot prediction training method for a universal temporal Transformer model according to claim 5, characterized in that: The specific steps of S31 are: S311, Determine the forecast target: Clarify the specific forecast target and clearly define the forecast time range and accuracy requirements to provide direction for the subsequent design of evaluation indicators; S312, Mining Field Professional Indicators: Sort out existing professional evaluation indicators or concepts in the target field; S313, Constructing composite evaluation indicators: Design composite evaluation indicators based on prediction objectives and professional indicators; S314, determine the calculation method: for the design evaluation indicators, clarify the specific calculation method and formula; S315, Set evaluation criteria: Set the qualification or excellence criteria for evaluation indicators based on the actual situation in the field and the requirements of the prediction task; S316, Conduct experimental verification: Use historical data and different models to conduct experiments, calculate the designed evaluation index values, analyze whether the index can accurately reflect the predictive performance of the model in its field, and compare the performance of different models under the same evaluation index. Observe the consistency between the index value and the actual prediction effect to test the effectiveness of the index; S317, Optimize and adjust indicators: Optimize the evaluation indicators based on the experimental verification results.

7. The zero-shot prediction training method for a universal temporal Transformer model according to claim 5, characterized in that: The specific steps of S32 are as follows: S321, Screening and Evaluation Fields: Comprehensively consider the characteristics, differences, and practical application value of time series data in different fields, and select multiple representative fields; S322, Data Collection and Organization: First, for each selected domain, sufficient time series data is collected, including training data and test data. The training data is used to train the model in a specific domain, and the test data is used to evaluate the performance of the model in cross-domain scenarios. The collected data is then uniformly preprocessed to ensure data quality and consistency, facilitating subsequent model training and evaluation. S323, accuracy-based indicator design: S3231, Inter-domain prediction accuracy improvement: First calculate the prediction accuracy of the model in the target domain, then compare it with the prediction accuracy in the original domain to determine the accuracy improvement; S3232, average cross-domain prediction accuracy: averages the prediction accuracy of the model in multiple target domains to obtain the average cross-domain prediction accuracy; S324, similarity-based indicator design: S3241, Prediction Result Feature Similarity: First, use the feature extraction algorithm to extract the feature vectors of the prediction results in different fields. Then, calculate the cosine similarity, Euclidean distance and other metrics between the feature vectors to evaluate the similarity of the model's prediction results in different fields. S3242, Probability Distribution Similarity: Using statistical methods to compare the probability distribution differences of model prediction results in different fields; S325, model training: Use the general temporal Transformer model to train on the selected original domain data and adjust the model parameters; S326, cross-domain prediction: Apply the trained model to the test data of each target domain, perform zero-shot prediction, and obtain prediction results; S327, indicator calculation: Based on the designed cross-domain generalization ability evaluation indicators, the prediction results of the model in each target field are calculated to obtain the specific values of each indicator.

8. The zero-shot prediction training method for a universal temporal Transformer model according to claim 5, characterized in that: The specific steps of S34 are as follows: S341, clarify the evaluation object and goal: First, clarify the specific content that needs to be quantitatively evaluated in the output results of the general time Transformer model zero-shot prediction. Then, set the goal of uncertainty quantitative evaluation based on the actual application scenario and needs; S342, select uncertainty quantification method: S3421, Bayesian-based approach: First, a reasonable prior distribution is set for the model parameters based on domain knowledge and historical data. Then, the Bayesian formula is used to calculate the posterior distribution of the model parameters in combination with the observed data. The prediction results are then sampled based on the posterior distribution to obtain multiple prediction samples. The distribution of these samples is then analyzed to quantify the uncertainty of the prediction results. S3422, Monte Carlo dropout method: First, during the training and prediction process of the Universal Temporal Transformer model, the dropout mechanism is enabled. Some neurons are randomly dropped during each forward propagation to simulate different model structures. Then, multiple forward propagation predictions are performed on the same input data, with different neurons dropped during each prediction. Multiple prediction results are obtained. The uncertainty of the prediction results is quantified by calculating the statistics of these multiple predictions. S343, Data Preparation: Prepare test data for uncertainty quantitative assessment, ensuring that the data characteristics are similar to and representative of data in actual application scenarios; S344, executing the quantification method: calculating the prediction results of the model according to the selected uncertainty quantification method; S345, Calculate uncertainty index: Calculate specific uncertainty index based on the output of the quantification method; S346, Visualization: Present the uncertainty assessment results in an intuitive visual way; S347, combined with prediction results analysis: Combine the uncertainty quantification results with the model's predicted values for analysis, so that in practical applications, the degree of trust in and use of the prediction results can be adjusted according to the size of the uncertainty.

9. The zero-shot prediction training method for a universal temporal Transformer model according to claim 1, characterized in that: The specific steps of S4 are as follows: S41, Model Compression and Lightweighting: Using model compression technology to reduce the number of parameters and computational complexity of the general temporal Transformer model; S42, Model Caching and Reuse: Establish a model caching mechanism to cache trained model parameters and intermediate results. When a new zero-shot prediction training task is performed, the cache is first checked to see if there is a model or intermediate result similar to the current task. If so, the model or result can be reused to avoid repeated training and calculation. S43, Distributed Training and Cloud Computing: Using a distributed training framework, training tasks are distributed to multiple computing nodes for parallel execution, speeding up training. At the same time, using a cloud computing platform, computing resources are dynamically allocated according to the needs of training tasks, avoiding resource waste and improving training efficiency.