Model training method and device, time sequence prediction method and device, equipment and storage medium

Abstract

The invention relates to a model training method and device, a time sequence prediction method and device, equipment and a storage medium. Performing aggregation processing on a first number of continuous first data in the first time series data to obtain a first aggregation value sequence composed of first aggregation values capable of reflecting the dispersion degree of the first number of continuous first data; converting the first aggregate value sequence into a first word sequence based on a mapping relation between aggregate values and words; constructing at least one training sample based on the first word sequence; the training sample comprises a section of second word sequence and a section of third word sequence in the first word sequence, and the third word sequence is located behind the second word sequence and is adjacent to the second word sequence; and training the language model by using the at least one training sample. Therefore, the language model not only can be used for processing different time sequence data, but also can solve the problem of self-similarity solution of the time sequence data in the microscopic level.

Description

Technical Field

[0001] The present disclosure relates to the technical field of time series data processing, and in particular to a model training method, a time series prediction method, an apparatus, a device and a storage medium. Background Art

[0002] Time series data, also known as time series data, refers to a collection of data recorded in chronological order. Time series data is usually collected, recorded or observed over a period of time at a certain time interval or timestamp. Time series data is widely used in various fields, such as financial market analysis, weather forecasting, traffic flow monitoring, production process monitoring, etc. With the rapid development of artificial intelligence technology, large language models (LLM) play an important role in various application scenarios. Using large model technology to solve problems such as time series data prediction has become an important research topic. Summary of the invention

[0003] A technical problem to be solved by the present disclosure is how to use large model technology to solve the problem of time series data.

[0004] According to the first aspect of the present disclosure, a model training method is provided, comprising: aggregating a first number of continuous first data in first time series data to obtain a first aggregate value sequence composed of first aggregate values ​​that can reflect the degree of discreteness of the first number of continuous first data; based on the mapping relationship between aggregate values ​​and words, converting the first aggregate value sequence into a first word sequence; constructing at least one training sample based on the first word sequence; the training sample includes a second word sequence and a third word sequence in the first word sequence, the third word sequence is located after the second word sequence and adjacent to the two-word sequence, and the second word sequence and the third word sequence each include at least one word; training a language model using the at least one training sample; wherein the language model is adjusted with the goal of reducing the difference between the output obtained by the language model processing the second word sequence and the third word sequence.

[0005] Optionally, aggregation processing is performed on a first number of continuous first data in the first time series data, including: calculating the coefficient of fluctuation of the first number of continuous first data in the first time series data; or calculating the variance of the first number of continuous first data in the first time series data; or calculating the standard deviation of the first number of continuous first data in the first time series data.

[0006] Optionally, the first quantity is two.

[0007] Optionally, the calculation formula of the aggregation process is expressed as:

[0008]

[0009] Among them, x1 and x2 represent any two adjacent first data in the first time series data.

[0010] Optionally, based on the mapping relationship between aggregation values ​​and words, converting the first aggregation value sequence into a first word sequence includes: based on the mapping relationship between aggregation values ​​and rare words, converting the first aggregation value sequence into a first rare word sequence.

[0011] Optionally, the length of the second word sequence is greater than the length of the third word sequence.

[0012] According to a second aspect of the present disclosure, a time series prediction method is provided, comprising: aggregating a first number of continuous second data in second time series data to obtain a second aggregate value sequence composed of second aggregate values ​​that can reflect the degree of discreteness of the first number of continuous second data; based on a mapping relationship between aggregate values ​​and words, converting the second aggregate value sequence into a fourth word sequence; constructing at least one prediction sample based on the fourth word sequence; the prediction sample is the last word in the fourth word sequence, or the prediction sample is a continuous word sequence including the last word in the fourth word sequence; processing the prediction sample using a language model to obtain a fifth word sequence output by the language model, the fifth word sequence including at least one word; based on the mapping relationship between aggregate values ​​and words, converting the fifth word sequence into a third aggregate value sequence.

[0013] Optionally, the method further includes: analyzing the fluctuation or dispersion of the second data within a future period of time based on the third aggregate value sequence.

[0014] Optionally, the method further includes: based on the second time series data, performing an operation opposite to the aggregation process on the third aggregate value in the third aggregate value sequence to obtain second data within a future period of time.

[0015] Optionally, aggregation processing is performed on a first number of consecutive second data in the second time series data, including: calculating the coefficient of fluctuation of the first number of consecutive second data in the second time series data; or calculating the variance of the first number of consecutive second data in the second time series data; or calculating the standard deviation of the first number of consecutive second data in the second time series data.

[0016] Optionally, the first quantity is two.

[0017] Optionally, the calculation formula of the aggregation process is expressed as:

[0018]

[0019] Among them, x1 and x2 represent any two adjacent first data in the first time series data.

[0020] Optionally, based on the mapping relationship between aggregate values ​​and words, converting the second aggregate value sequence into a fourth word sequence includes: based on the mapping relationship between aggregate values ​​and rare words, converting the second aggregate value sequence into a fourth rare word sequence.

[0021] Optionally, the language model is a language model trained using the method described in the first aspect above.

[0022] According to a third aspect of the present disclosure, a model training device is provided, comprising: an aggregation module for performing aggregation processing on a first number of continuous first data in first time series data to obtain a first aggregate value sequence composed of first aggregate values ​​that can reflect the discrete degree of the first number of continuous first data; a conversion module for converting the first aggregate value sequence into a first word sequence based on a mapping relationship between aggregate values ​​and words; a construction module for constructing at least one training sample based on the first word sequence; the training sample includes a second word sequence and a third word sequence in the first word sequence, the third word sequence is located after the second word sequence and adjacent to the two-word sequence, and the second word sequence and the third word sequence each include at least one word; a training module for training a language model using the at least one training sample; wherein the language model is adjusted with the goal of reducing the difference between the output obtained by the language model processing the second word sequence and the third word sequence.

[0023] Optionally, the aggregation module calculates the coefficient of fluctuation of a first number of consecutive first data in the first time series data; or the aggregation module calculates the variance of a first number of consecutive first data in the first time series data; or the aggregation module calculates the standard deviation of a first number of consecutive first data in the first time series data.

[0024] Optionally, the first quantity is two.

[0025] Optionally, the calculation formula of the aggregation process is expressed as:

[0026]

[0027] Among them, x1 and x2 represent any two adjacent first data in the first time series data.

[0028] Optionally, the conversion module converts the first aggregate value sequence into a first rare word sequence based on a mapping relationship between aggregate values ​​and rare words.

[0029] Optionally, the length of the second word sequence is greater than the length of the third word sequence.

[0030] According to a fourth aspect of the present disclosure, a time series prediction device is provided, comprising: an aggregation module, configured to aggregate a first number of continuous second data in second time series data to obtain a second aggregate value sequence composed of second aggregate values ​​that can reflect the degree of discreteness of the first number of continuous second data; a first conversion module, configured to convert the second aggregate value sequence into a fourth word sequence based on a mapping relationship between aggregate values ​​and words; a construction module, configured to construct at least one prediction sample based on the fourth word sequence; the prediction sample is the last word in the fourth word sequence, or the prediction sample is a continuous word sequence including the last word in the fourth word sequence; a prediction module, configured to process the prediction sample using a language model to obtain a fifth word sequence output by the language model, wherein the fifth word sequence includes at least one word; and a second conversion module, configured to convert the fifth word sequence into a third aggregate value sequence based on a mapping relationship between aggregate values ​​and words.

[0031] Optionally, the device further includes: an analysis module, configured to analyze, based on the third aggregate value sequence, a fluctuation or a discrete degree of the second data within a future period of time.

[0032] Optionally, the device further includes: a calculation module, configured to perform an operation opposite to the aggregation process on the third aggregate value in the third aggregate value sequence based on the second time series data, so as to obtain second data within a future period of time.

[0033] Optionally, the aggregation module calculates the coefficient of fluctuation of a first number of consecutive second data in the second time series data; or the aggregation module calculates the variance of a first number of consecutive second data in the second time series data; or the aggregation module calculates the standard deviation of a first number of consecutive second data in the second time series data.

[0034] Optionally, the first quantity is two.

[0035] Optionally, the calculation formula of the aggregation process is expressed as:

[0036]

[0037] Among them, x1 and x2 represent any two adjacent first data in the first time series data.

[0038] Optionally, the first conversion module converts the second aggregate value sequence into a fourth rare word sequence based on a mapping relationship between aggregate values ​​and rare words.

[0039] Optionally, the language model is a language model trained using the method described in the first aspect above.

[0040] According to a fifth aspect of the present disclosure, a computing device is provided, comprising: a processor; and a memory on which executable code is stored, and when the executable code is executed by the processor, the processor executes the method described in the first aspect or the second aspect above.

[0041] According to a sixth aspect of the present disclosure, a computer program product is provided, comprising an executable code. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method described in the first aspect or the second aspect above.

[0042] According to a seventh aspect of the present disclosure, a non-temporary machine-readable storage medium is provided, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor executes the method described in the first aspect or the second aspect above.

[0043] The present disclosure aggregates a first number of continuous first data in first time series data to obtain a first aggregate value sequence composed of first aggregate values ​​that can reflect the discreteness of the first number of continuous first data, converts the first aggregate value sequence into a first word sequence based on a mapping relationship between aggregate values ​​and words, constructs at least one training sample based on the first word sequence, and trains a language model. Therefore, the language model can not only be used to process different time series data, but also solve the self-similarity problem of time series data at the micro level. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.

[0045] Figure 1 It is a schematic flowchart showing a model training method according to an embodiment of the present disclosure.

[0046] Figure 2 is a schematic flow chart showing a time series prediction method according to an embodiment of the present disclosure.

[0047] Figure 3 It is a schematic diagram showing the structure of a model training device according to an embodiment of the present disclosure.

[0048] Figure 4 It is a schematic diagram showing the structure of a timing prediction device according to an embodiment of the present disclosure.

[0049] Figure 5 FIG. 4 is a schematic diagram showing the structure of a computing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0051] First, some time series analysis techniques are illustrated.

[0052] 1) Autoregressive model (AR): It is one of the most basic models in time series analysis, which predicts future values ​​based on past observations. The autoregressive model assumes that the current value is a linear combination of past values ​​plus a random error term.

[0053] 2) Moving average model (MA): The moving average model predicts future values ​​by calculating the weighted average of past error terms. The moving average model focuses on smoothing random error terms.

[0054] 3) Auto-Regressive Moving Average Model (ARMA): The ARMA model combines the characteristics of the AR and MA models and is a linear combination of current values, past values, and past error terms. The auto-regressive moving average model is suitable for the prediction of stationary time series.

[0055] 4) Autoregressive Integrated Moving Average Model (ARIMA): ARIMA is the abbreviation of ARIMA (p, d, q). Among them, p is the order of the autoregressive term, d is the order of the difference, and q is the order of the moving average term. When the time series is not stationary, it can be made stationary through the difference operation, and then the ARMA model can be applied for prediction.

[0056] 5) Seasonal ARIMA model (Seasonal Autoregressive Integrated Moving Average Model, SARIMA): The SARIMA model adds seasonal component processing to the ARIMA model. The SARIMA model is suitable for time series data with obvious seasonal characteristics.

[0057] 6) Simple exponential smoothing: Applicable to stationary time series without trend and seasonality. It predicts future values ​​by giving higher weight to the most recent observations.

[0058] 7) Holt exponential smoothing: Applicable to time series with linear trends. In addition to smoothing the observed values, the trend component is also taken into account.

[0059] 8) Holt-Winters exponential smoothing: Applicable to time series with trend and seasonality. It smoothes the trend component and seasonal component separately and makes predictions based on the observed values.

[0060] 9) Seasonal Decomposition: Decompose the time series into trend component, seasonal component and residual component. The decomposed components can be further analyzed or predicted in combination with other models.

[0061] 10) Dynamic Linear Models (DLMs): At each time point, the model corresponds to a linear model, but the regression coefficients change over time. It is suitable for scenarios where it is necessary to capture dynamic changing relationships in time series.

[0062] 11) Neural network models: such as recurrent neural network (RNN), long short-term memory network (LSTM) and Transformer. These models capture nonlinear relationships and long-term dependencies in time series through complex network structures and learning capabilities. In particular, LSTM and Transformer perform well in processing long time series data.

[0063] 12) Other specific models: such as the GARCH model (used to model financial time series with time-varying volatility), the TBATS model (a forecasting model based on exponential smoothing that can handle multiple periodicities), etc.

[0064] The above time series analysis techniques often have some unique challenges when processing time series data, such as periodicity, trend, seasonality, etc. The reason for such challenges is that almost all existing time series analysis models assume that time series data has periodicity or trend patterns, and this assumption has become a limitation for the further development of time series analysis models.

[0065] In view of this, the present disclosure proposes that the assumption can be changed from "time series data has periodic laws or trend laws" to "time series data has laws." Then, based on the new assumption, the laws of time series data are divided into two categories: macro laws and micro laws. For a certain time series data, macro laws and micro laws can exist at the same time, that is, the data at any point in time can be in both macro laws and micro laws. For example, financial markets, weather forecasts, traffic flow monitoring, and production process monitoring basically have macro laws, but also micro laws. However, previous research methods only focused on macro laws and ignored micro laws. Therefore, the goal of the present disclosure is to solve the prediction of micro laws in time series data.

[0066] Before introducing the solution disclosed in the present invention, the theoretical basis of microscopic laws: fractal theory is first introduced.

[0067] Fractals refer to systems or collections that, under certain conditions, show similarities between the whole and the parts in terms of morphology, structure, information, function, time, energy, etc. through the principle of self-similarity and iterative generation. It has the morphological characteristics of filling space in a non-integer dimensional form, and can be regarded as rough or fragmentary geometric shapes. These shapes can be divided into several parts, and each part is (at least approximately) a reduced shape of the whole, that is, it has a self-similar property. The present disclosure assumes that the microscopic laws of time series data comply with the self-similarity of the fractal principle.

[0068] When exploring macroscopic laws, most algorithms expect to find the parameters of the laws, so as to obtain the results through formula calculations. Microscopic laws can be subdivided into causes and processes based on self-similarity. The causes are ever-changing and unpredictable, but the processes are self-similar. Therefore, the present invention does not dwell on what events or factors cause the microscopic changes in the time series, but focuses on finding out what the laws based on self-similarity are. The premise of self-similarity exploration is to assume that the causes of microscopic laws are not predictable, but the processes are predictable.

[0069] In the exploration of the self-similarity of micro-rules, the present disclosure associates self-similarity with the ability of a language model (such as a large language model (LLM)) learned from a training set, and believes that self-similarity is equivalent to finding rules from a training set, that is, the self-similarity problem within the training set is transformed into the problem of predicting the maximum probability of the Next Token (next word), and the result of the maximum probability of the Next Token is the result of the maximum self-similarity of the training set.

[0070] Therefore, language models can be used to solve the self-similarity problem of time series data at the micro level.

[0071] The following is an illustrative example of how to use language models to solve the self-similarity problem of time series data at the micro level from the two aspects of model training and model prediction.

[0072] Figure 1 It is a schematic flowchart showing a model training method according to an embodiment of the present disclosure. Figure 1 The method shown can be implemented in software via a computer program, and can also be executed via a specifically configured computing device.

[0073] See also Figure 1 In step S110, a first number of continuous first data in the first time series data are aggregated to obtain a first aggregate value sequence consisting of first aggregate values ​​that can reflect the degree of discreteness of the first number of continuous first data.

[0074] The first time series data refers to a set of data recorded in chronological order. The first time series data can be obtained by collecting, recording or observing at a certain time interval or timestamp within a period of time. The first time series data can be, but is not limited to, time series data generated in various fields such as financial market analysis, weather forecasting, traffic flow monitoring, and production process monitoring.

[0075] In some embodiments, one or more first time series data may be obtained, and the one or more first time series data may be time series data in the same application scenario (such as a stock trading price prediction scenario). Correction S110 may be performed for each first time series data to obtain a first aggregate value sequence corresponding to the first time series data.

[0076] The first time series data may include multiple data records arranged in chronological order, and each data record may include one or more fields. The first data may refer to the field value in the data record that is related to the time series prediction target. Taking the time series prediction target as an example of predicting the stock transaction price, the stock transaction information within a period of time (such as 10 days) can be extracted according to the transaction sequence, and the first time series data consisting of multiple data records including multiple fields such as transaction sequence number, transaction time, transaction price, transaction volume, etc. can be obtained. At this time, the first data in the first time series data may refer to the transaction price.

[0077] Performing aggregating processing on a first number of continuous first data in the first time series data may refer to performing aggregating processing on any first number of continuous first data in the first time series data.

[0078] The aggregation processing performed on the first number of continuous first data refers to an aggregation operation in which the aggregation result (i.e., the first aggregate value) can reflect the degree of discreteness of the first number of continuous first data, and may include but is not limited to aggregation operations such as calculating the coefficient of fluctuation, calculating the variance, and calculating the standard deviation. That is, the first aggregate value may be but is not limited to the coefficient of fluctuation, the variance, or the standard deviation of the first number of continuous first data. Exemplarily, the coefficient of fluctuation of the first number of continuous first data in the first time series data may be calculated to obtain a first aggregate value sequence; or the variance of the first number of continuous first data in the first time series data may be calculated to obtain a first aggregate value sequence; or the standard deviation of the first number of continuous first data in the first time series data may be calculated to obtain a first aggregate value sequence.

[0079] Each first aggregate value in the first aggregate value sequence corresponds to a first number of continuous first data in the first time series data. The first aggregate values ​​in the first aggregate value sequence are also arranged in chronological order. Moreover, the first number of continuous first data corresponding to adjacent first aggregate values ​​differ only in the first data at the starting position and the end position, and the first data in the middle part are the same. Exemplarily, a certain first data (such as the first) in the first time series data can be used as the starting position, and the first number can be used as the window size to slide the window, and the first number of continuous first data obtained by each sliding can be aggregated to obtain the first aggregate value sequence. For example, assuming that the first time series data has 100 first data and the first number is 2, then the first aggregate value sequence consisting of 99 first aggregate values ​​can be obtained in the manner of "aggregating the first first data with the second first data", "aggregating the second first data with the third first data", and "aggregating the third first data with the fourth first data".

[0080] Generally speaking, time series data with different numerical ranges are usually incomparable. For example, time series data reflecting wheat prices are not comparable to time series data reflecting gold prices. However, the first aggregate value sequence obtained by aggregating the first time series data can reflect the change in the discreteness (or fluctuation) of a period of time series data, and the change law of the discreteness (or fluctuation law) of time series data with different numerical ranges is similar, and the value range of the aggregate value is bounded and comparable. Therefore, by constructing training samples based on the first aggregate value sequence obtained by aggregation processing, the language model subsequently trained based on the training samples can be used to process different time series data, thereby improving the versatility of the trained language model.

[0081] In addition, if the first time series data is aggregated according to time periods, the microscopic regularities in the time series data will be destroyed. The present disclosure can retain microscopic self-similarity by abandoning the time period aggregation method and instead using number aggregation. Furthermore, in order to better retain the microscopic similarity in the first time series data, the first number can take a smaller value. For example, preferably, the first number can be 2, so that the microscopic self-similarity can be retained as much as possible.

[0082] Taking the first number as 2 and the aggregation method of calculating the coefficient of variation (CV) as an example, the calculation formula of the aggregation process in step S110 can be expressed as:

[0083]

[0084] Among them, x1 and x2 represent any two adjacent first data in the first time series data. It represents the absolute value of the coefficient of fluctuation of x1 and x2 calculated according to the standard coefficient of fluctuation formula. It is usually a small value. In order to improve the accuracy of data processing, you can set an amplification factor to increase the Amplify to a larger value. The amplification factor is a value greater than 1. The specific value can be flexibly set according to the actual situation, such as determining the value of the amplification factor according to the specific accuracy requirements. Taking the stock price prediction scenario mentioned above as an example, considering that the fluctuation of the price of two adjacent transactions is small, the amplification factor can take a larger value, such as 100000.

[0085] In step S120, based on the mapping relationship between the aggregate values ​​and the words, the first aggregate value sequence is converted into a first word sequence.

[0086] A word can refer to a single word, or a phrase consisting of multiple words. Preferably, a word can refer to a single word, and in particular, can refer to the uncommon words described below.

[0087] The mapping relationship between the aggregate value and the word can be predetermined. For example, a word list can be pre-built, and then a word can be assigned to each first aggregate value in the first aggregate value sequence to correspond the first aggregate value to the word. The same aggregate value corresponds to the same word. Different aggregate values ​​correspond to different words.

[0088] The language model to be trained in the present disclosure may refer to a language model with large-scale parameters and pre-trained on a large amount of text data. In order to shield the inherent influence of the language model, the first aggregate value sequence can be converted into a first obscure word sequence based on the mapping relationship between the aggregate value and the obscure words. This is because the probability of obscure words appearing in the language model is low and is less affected by natural language materials. For example, the language model usually believes that "red" and "color" are close and related, while the association relationship between different obscure words is basically zero for the language model. Therefore, obscure words will not be affected by the existing capabilities of the language model. This makes it possible to establish the relationship between obscure words with less training data. It should be known that it is theoretically feasible to convert the first aggregate value sequence into a non-obscure word sequence based on the mapping relationship between the aggregate value and non-obscure words (such as common words), but this requires more training data to destroy the relationship between the common words that the model has learned.

[0089] For example, a rare word list may be defined, and some of the rare words in the rare word list may include:

[0090] 1041, 1044, 1049, 105, 1050, 1052, 1055, 1058, 106, 1060, 1069, 107, 1070, 1071, 1073, 1074.

[0091] By matching the above rare word list with the CV sequence one by one, we can get the following mapping relationship: 1

[0092] 10 100 1000 1001 1003 1004 1005 1006 101 1010 1013 1019 102 1022 1023

[0093] 1024 1025 103 1032 1034 1035 104

[0094] 1041 1044 1049 105 1050 1052 1055

[0095] 1058 106 1060 1069 107 1070 1071

[0096] 1073 1074

[0097] Based on this mapping relationship, the CV sequence can be converted into a corresponding rare character sequence.

[0098] In step S130, at least one training sample is constructed based on the first word sequence.

[0099] Each training sample includes a second word sequence and a third word sequence in the first word sequence. The third word sequence is located after the second word sequence and adjacent to the two-word sequence, and the second word sequence and the third word sequence each include at least one word. Each training sample can be regarded as a question-answer pair. The second word sequence can be regarded as a question (query) for inputting a language model, and the third word sequence can be regarded as a standard answer (answer) to the question.

[0100] It is reasonable to predict a shorter sequence based on a longer sequence, but the accuracy of predicting a longer sequence based on a shorter sequence cannot be guaranteed. Therefore, the length of the second word sequence can be greater than or equal to the length of the third word sequence.

[0101] Exemplarily, the first word sequence can be subjected to random length extraction to obtain a training sample consisting of a second word sequence and a third word sequence. That is, the starting position of the second word sequence, the length of the second word sequence, and the length of the third word sequence can be randomly determined, and then extracted to obtain a training sample consisting of the second word sequence and the third word sequence. Taking the extraction of the rare word sequence described in the above example, the rare word sequence can be subjected to random length extraction to form a query / response pair (i.e., training sample), that is, the starting position of the sequence extraction, the query length, and the response length are randomly generated. Some results are as follows: query: response: query: response: query: response:

[0102] In step S140, a language model is trained using at least one training sample.

[0103] Training a language model may refer to fine-tuning a language model that has been pre-trained using text corpus, or may refer to training a language model that has not been pre-trained using text corpus.

[0104] During the training (eg, fine-tuning) process, the language model may be adjusted with the goal of reducing the difference between the output obtained by the language model when processing the second word sequence and the third word sequence.

[0105] For example, a loss function can be constructed based on the difference between the output obtained by processing the second word sequence by the language model and the third word sequence, and the model parameters of the language model can be adjusted with the goal of minimizing the function value of the loss function, wherein the function value of the loss function is positively correlated with the difference.

[0106] After training the language model, you can deploy the language model for inference. Use the language model for time series prediction.

[0107] The present disclosure does not impose any special restrictions on the fine-tuning and deployment process, and can be executed according to the existing model fine-tuning or deployment methods. It should be noted that the demand for fine-tuning data volume will increase. According to tests, when there are hundreds of thousands of query / response pairs, the accuracy of remembering historical data is about 95%, and there is basically no generalization ability; when the data is in the tens of millions, the generalization accuracy is 10%. Therefore, continuous optimization is needed to find a model training method that converges quickly. By testing the language model, it is found that the learning ability is good, but the reasoning ability is poor, which proves the feasibility of the method, but the effect still needs to be optimized. Therefore, the present disclosure can also be combined with other means for continuous optimization, so as to find a model training method that converges quickly.

[0108] Figure 2 is a schematic flow chart showing a time series prediction method according to an embodiment of the present disclosure. Figure 2 The method shown can be implemented in software via a computer program, and can also be executed via a specifically configured computing device.

[0109] The language model used in the time series prediction method disclosed in the present invention can be a combination of the above Figure 1 The language model obtained by training the method shown. In addition, the mapping relationship between the first quantity, aggregation value and word mentioned in the time series prediction method disclosed in the present invention is the same as the mapping relationship between the first quantity, aggregation value and word mentioned in the model training process.

[0110] See also Figure 2 In step S210, a first number of continuous second data in the second time series data are aggregated to obtain a second aggregate value sequence consisting of second aggregate values ​​that can reflect the degree of discreteness of the first number of continuous second data.

[0111] Second time series data refers to a set of data recorded in chronological order. Second time series data can be collected, recorded or observed at a certain time interval or timestamp within a period of time. Second time series data can be, but is not limited to, time series data generated in various fields such as financial market analysis, weather forecasting, traffic flow monitoring, and production process monitoring.

[0112] The second time series data may include multiple data records arranged in chronological order, and each data record may include one or more fields. The second data may refer to the field value related to time series prediction in the data record. Taking the prediction of stock transaction prices as an example, the stock transaction information within a period of time (such as 10 days) can be extracted according to the transaction order, and the second time series data consisting of multiple data records including multiple fields such as transaction sequence number, transaction time, transaction price, transaction volume, etc. can be obtained. At this time, the second data in the second time series data may refer to the transaction price.

[0113] Performing aggregating processing on a first number of continuous second data in the second time series data may refer to performing aggregating processing on any first number of continuous second data in the second time series data.

[0114] For more information about aggregation processing, please refer to the above combined Figure 1 For example, the fluctuation coefficient of the first number of consecutive second data in the second time series data may be calculated to obtain the second aggregate value sequence; or the variance of the first number of consecutive second data in the second time series data may be calculated to obtain the second aggregate value sequence; or the standard deviation of the first number of consecutive second data in the second time series data may be calculated to obtain the second aggregate value sequence.

[0115] In step S220, based on the mapping relationship between the aggregate value and the word, the second aggregate value sequence is converted into a fourth word sequence. Exemplarily, the second aggregate value sequence can be converted into a fourth rare word sequence based on the mapping relationship between the aggregate value and the rare word. The mapping relationship based on which the second aggregate value sequence is converted into the fourth word sequence is the same as the mapping relationship based on which the model is trained.

[0116] In step S230, at least one prediction sample is constructed based on the fourth word sequence.

[0117] The prediction sample may be the last word in the fourth word sequence. Alternatively, the prediction sample may be a continuous word sequence including the last word in the fourth word sequence. Generally speaking, the accuracy of prediction using a longer word sequence is higher. Therefore, the constructed prediction sample may preferably be a continuous word sequence including the last word in the fourth word sequence.

[0118] In step S240, the predicted sample is processed using the language model to obtain a fifth word sequence output by the language model, where the fifth word sequence includes at least one word.

[0119] In step S250, based on the mapping relationship between the aggregate values ​​and the words, the fifth word sequence is converted into a third aggregate value sequence.

[0120] Each third aggregate value in the third aggregate value sequence reflects the degree of discreteness of a first number of continuous second data including at least part of the second data within a future period of time.

[0121] In some embodiments, the fluctuation or dispersion of the second data in a future period of time can be analyzed based on the third aggregate value sequence to determine the fluctuation or dispersion of the second time series data in the future period of time.

[0122] In some embodiments, based on the second time series data, the third aggregate value in the third aggregate value sequence may be subjected to an operation opposite to the aggregation process to obtain second data within a future period of time.

[0123] Therefore, the present disclosure can predict the time series data itself, and can also predict the discreteness or fluctuation of the time series data.

[0124] The operations of solving the self-similarity of time series data at the micro level using a large model disclosed in the present invention can be summarized as follows: 1. Changing the time period aggregation of the data sequence to the frequency aggregation of the time series data to retain the micro self-similarity; 2. Encoding the results of the frequency aggregation of the time series data to convert the self-similarity from numbers to text that can be understood by the model; 3. Generating a fine-tuning corpus and a fine-tuning model; 4. Using the fine-tuned model to perform time series prediction.

[0125] The model training method disclosed in the present invention can also be implemented as a model training device. The functional units of the model training device can be implemented by hardware, software, or a combination of hardware and software that implements the principles of the present invention. The following is a brief description of the functional units that the model training device can have and the operations that each functional unit can perform. For the details involved, please refer to the relevant description above, which will not be repeated here. It can be understood by those skilled in the art that the functional units described in this article can be combined or divided into sub-units to implement the principles of the above invention. Therefore, the description herein can support any possible combination, division, or further limitation of the functional units described herein.

[0126] Figure 3 It is a schematic diagram showing the structure of a model training device according to an embodiment of the present disclosure.

[0127] See also Figure 3 , the model training device 300 may include an aggregation module 310, a conversion module 320, a construction module 330 and a training module 340. The aggregation module 310 is used to perform aggregation processing on a first number of continuous first data in the first time series data to obtain a first aggregation value sequence composed of first aggregation values ​​that can reflect the discrete degree of the first number of continuous first data. The conversion module 320 is used to convert the first aggregation value sequence into a first word sequence based on the mapping relationship between the aggregation value and the word. The construction module 330 is used to construct at least one training sample based on the first word sequence. The training module 340 is used to train a language model using at least one training sample; wherein the language model is adjusted with the goal of reducing the difference between the output obtained by the language model processing the second word sequence and the third word sequence.

[0128] Exemplarily, the aggregation module 310 can calculate the coefficient of fluctuation of a first number of consecutive first data in the first time series data; or the aggregation module 310 can calculate the variance of a first number of consecutive first data in the first time series data; or the aggregation module 310 can calculate the standard deviation of a first number of consecutive first data in the first time series data.

[0129] Exemplarily, the conversion module 340 may convert the first aggregate value sequence into a first rare word sequence based on a mapping relationship between aggregate values ​​and rare words.

[0130] The timing prediction method disclosed in the present invention can also be implemented as a timing prediction device. The functional units of the timing prediction device can be implemented by hardware, software, or a combination of hardware and software that implements the principles of the present invention. The following is a brief description of the functional units that the timing prediction device can have and the operations that each functional unit can perform. For the details involved, please refer to the relevant description above, which will not be repeated here. It can be understood by those skilled in the art that the functional units described in this article can be combined or divided into sub-units to implement the principles of the above invention. Therefore, the description in this article can support any possible combination, or division, or further limitation of the functional units described in this article.

[0131] Figure 4 It is a schematic diagram showing the structure of a timing prediction device according to an embodiment of the present disclosure.

[0132] See also Figure 4 The time series prediction device 400 may include an aggregation module 410 , a first conversion module 420 , a construction module 430 , a prediction module 440 and a second conversion module 450 .

[0133] The aggregation module 410 is used to perform aggregation processing on a first number of continuous second data in the second time series data to obtain a second aggregate value sequence composed of second aggregate values ​​that can reflect the discreteness of the first number of continuous second data. The first conversion module 420 is used to convert the second aggregate value sequence into a fourth word sequence based on the mapping relationship between the aggregate value and the word. The construction module 430 is used to construct at least one prediction sample based on the fourth word sequence. The prediction module 440 is used to process the prediction sample using a language model to obtain a fifth word sequence output by the language model, and the fifth word sequence includes at least one word. The second conversion module 450 is used to convert the fifth word sequence into a third aggregate value sequence based on the mapping relationship between the aggregate value and the word.

[0134] In some embodiments, the time series prediction device 400 may further include an analysis module. The analysis module is used to analyze the fluctuation or dispersion degree of the second data within a future period of time based on the third aggregate value sequence.

[0135] In some embodiments, the time series prediction device 400 may further include a calculation module. The calculation module is used to perform an operation opposite to the aggregation process on the third aggregated values ​​in the third aggregated value sequence based on the second time series data to obtain second data within a future period of time.

[0136] Exemplarily, the aggregation module 410 can calculate the coefficient of fluctuation of the first number of consecutive second data in the second time series data; or the aggregation module 410 can calculate the variance of the first number of consecutive second data in the second time series data; or the aggregation module 410 can calculate the standard deviation of the first number of consecutive second data in the second time series data.

[0137] Figure 5 A schematic diagram of the structure of a computing device that can be used to implement the above-mentioned model training method or time series prediction method according to an embodiment of the present disclosure is shown.

[0138] See also Figure 5 , the computing device 500 includes a memory 510 and a processor 520 .

[0139] The processor 520 may be a multi-core processor or may include multiple processors. In some embodiments, the processor 520 may include a general-purpose main processor and one or more special coprocessors, such as a graphics processing unit (GPU), a digital signal processor (DSP), etc. In some embodiments, the processor 520 may be implemented using a customized circuit, such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0140] The memory 510 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 520 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at runtime. In addition, the memory 510 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 510 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0141] The memory 510 stores executable codes, and when the executable codes are processed by the processor 520, the processor 520 can execute the model training method or time series prediction method mentioned above.

[0142] The model training method, time series prediction method, corresponding apparatus and computing device according to the present disclosure have been described in detail above with reference to the accompanying drawings.

[0143] In addition, the method according to the present disclosure may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing the above steps defined in the above method of the present disclosure.

[0144] Alternatively, the present disclosure may also be implemented as a non-temporary machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which executable code (or computer program, or computer instruction code) is stored. When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or computing device, server, etc.), the processor executes the various steps of the above-mentioned method according to the present disclosure.

[0145] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both.

[0146] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the system and method according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0147] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A model training method, characterized in that: include: Aggregating a first number of continuous first data in the first time series data to obtain a first aggregate value sequence consisting of first aggregate values ​​that can reflect the discreteness of the first number of continuous first data; Based on the mapping relationship between the aggregate values ​​and the words, convert the first aggregate value sequence into a first word sequence; Construct at least one training sample based on the first word sequence; The training sample includes a second word sequence and a third word sequence in the first word sequence, the third word sequence is located after the second word sequence and adjacent to the two-word sequence, and the second word sequence and the third word sequence each include at least one word; The language model is trained using the at least one training sample; wherein the language model is adjusted with the goal of reducing the difference between the output obtained by the language model processing the second word sequence and the third word sequence.

2. The method according to claim 1, characterized in that The first number is two.

3. The method according to claim 2, characterized in that The calculation formula of the aggregation process is expressed as: Among them, x1 and x2 represent any two adjacent first data in the first time series data.

4. A time series prediction method, characterized in that: include: Aggregating a first number of consecutive second data in the second time series data to obtain a second aggregate value sequence consisting of second aggregate values ​​that can reflect the discreteness of the first number of consecutive second data; Based on the mapping relationship between the aggregate value and the word, convert the second aggregate value sequence into a fourth word sequence; Constructing at least one prediction sample based on the fourth word sequence; the prediction sample is the last word in the fourth word sequence, or the prediction sample is a continuous word sequence including the last word in the fourth word sequence; Processing the predicted sample using a language model to obtain a fifth word sequence output by the language model, wherein the fifth word sequence includes at least one word; Based on the mapping relationship between the aggregation value and the word, the fifth word sequence is converted into a third aggregation value sequence.

5. The method according to claim 4, characterized in that The language model is a language model trained using the method described in any one of claims 1 to 3.

6. A model training device, characterized in that: include: an aggregation module, configured to perform aggregation processing on a first number of continuous first data in the first time series data to obtain a first aggregation value sequence consisting of first aggregation values ​​that can reflect the discreteness of the first number of continuous first data; a conversion module, configured to convert the first aggregate value sequence into a first word sequence based on a mapping relationship between aggregate values ​​and words; A construction module, used for constructing at least one training sample based on the first word sequence; The training sample includes a second word sequence and a third word sequence in the first word sequence, the third word sequence is located after the second word sequence and adjacent to the two-word sequence, and the second word sequence and the third word sequence each include at least one word; A training module is used to train a language model using the at least one training sample; wherein the language model is adjusted with the goal of reducing the difference between the output obtained by the language model processing the second word sequence and the third word sequence.

7. A time series prediction device, characterized in that: include: an aggregation module, configured to perform aggregation processing on a first number of continuous second data in the second time series data to obtain a second aggregation value sequence consisting of second aggregation values ​​capable of reflecting the discreteness of the first number of continuous second data; A first conversion module, configured to convert the second aggregate value sequence into a fourth word sequence based on a mapping relationship between aggregate values ​​and words; A construction module, configured to construct at least one prediction sample based on the fourth word sequence; the prediction sample is the last word in the fourth word sequence, or the prediction sample is a continuous word sequence including the last word in the fourth word sequence; A prediction module, configured to process the prediction sample using a language model to obtain a fifth word sequence output by the language model, wherein the fifth word sequence includes at least one word; The second conversion module is used to convert the fifth word sequence into a third aggregate value sequence based on the mapping relationship between the aggregate value and the word.

8. A computing device comprising: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 5.

9. A computer program product, comprising executable codes, which, when executed by a processor of an electronic device, causes the processor to perform the method according to any one of claims 1 to 5.

10. A non-transitory machine-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 5.

Images