Time sequence model training method and device and electronic equipment

Abstract

The invention relates to the technical field of machine learning, in particular to a time sequence model training method and device and electronic equipment. In the method, time sequence data is divided into context data and target data according to a preset rule, and data points are randomly shielded in the target data. And inputting the time sequence data into a Transformer encoder comprising a self-supervised learning mechanism, and distinguishing positive sample pairs and negative sample pairs from a time sequence data set based on a time contrast learning strategy. And inputting unshielded data points in the positive sample pair into a decoder, predicting shielded data points by adopting a cross self-attention mechanism, and optimizing a model training process by taking minimization of a preset comparison loss function as a target to obtain a time sequence model. According to the scheme, key features can be efficiently and intelligently extracted from the time sequence data, and then high-precision prediction of future data points is achieved.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a time series model training method, device and electronic device. Background Art

[0002] Time series forecasting is a core topic that has long been the focus of industry and academia. Its core lies in identifying hidden patterns and trends from historical data and making forward-looking predictions about future behaviors or events. With the booming development of deep learning technology, the field of time series forecasting is undergoing a paradigm shift from traditional statistical models to those based on deep learning technology.

[0003] How to efficiently and intelligently extract key features from time series data and achieve high-precision prediction of future data points has become a problem that urgently needs in-depth optimization. Summary of the invention

[0004] The embodiments of the present application provide a time series model training method, device and electronic device for efficiently and intelligently extracting key features from time series data, thereby achieving high-precision prediction of future data points.

[0005] In a first aspect, an embodiment of the present application provides a time series model training method, the method comprising:

[0006] Divide the time series data into context data and target data according to preset rules, and randomly mask data points in the target data, where the context data is historical data and the target data is predicted data points;

[0007] The time series data is input into a Transformer encoder that uses a self-supervised learning mechanism, and positive sample pairs and negative sample pairs are distinguished from the time series data set based on a time contrast learning strategy. The positive sample pairs are continuous time series segments containing context data and target data, and the negative sample pairs are data sets in the time series data that are not adjacent or have low correlation in time except for the context data and the target data.

[0008] The unmasked data points in the positive sample pairs are input into the decoder, and the masked data points are predicted using the cross self-attention mechanism. The model training process is optimized with the goal of minimizing the preset contrast loss function to obtain a time series model. The contrast loss function is calculated using negative sample pairs.

[0009] Optionally, the above contrast loss function is a total loss function composed of the reconstruction loss of temporal masking modeling and the temporal contrast loss of temporal contrast learning

[0010] Optionally, the weights of the reconstruction loss and the temporal contrast loss in the contrast loss function are adjusted using hyperparameters.

[0011] Optionally, the above Transformer encoder contains two branches that share the same network structure and weights.

[0012] Optionally, the above method further includes:

[0013] The downstream task fine-tuning technology is used to adjust the time series model to obtain multiple sub-models corresponding to different categories of downstream tasks.

[0014] Optionally, before dividing the time series data into context data and target data, the method further includes:

[0015] Standardize time series data.

[0016] In a second aspect, an embodiment of the present application provides a time series model training device, the device comprising:

[0017] A data classification module is used to divide the time series data into context data and target data according to preset rules, and randomly mask data points in the target data, wherein the context data is historical data and the target data is predicted data points;

[0018] The data classification module also inputs the time series data into a Transformer encoder that uses a self-supervised learning mechanism, and distinguishes positive sample pairs and negative sample pairs from the time series data set based on a time contrast learning strategy. The positive sample pairs are continuous time series segments containing context data and target data, and the negative sample pairs are data sets in the time series data that are not adjacent or have low correlation in time except for the context data and the target data.

[0019] The processing module is used to input the unmasked data points in the positive sample pairs into the decoder, use the cross self-attention mechanism to predict the masked data points, optimize the model training process with the goal of minimizing the preset contrast loss function, and obtain the time series model. The contrast loss function is calculated using negative sample pairs.

[0020] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the processor implements any one of the time series model training methods described in the first aspect.

[0021] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the time series model training method of any one of the first aspects is implemented.

[0022] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which is executed by a processor to implement a time series model training method as described in any one of the first aspects above.

[0023] The technical effects brought about by any one of the implementation methods in the second to fifth aspects can refer to the technical effects brought about by the corresponding implementation method in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of an application scenario of time series model training provided in an embodiment of the present application;

[0025] Figure 2 A flow chart of a time series model training method provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of the composition architecture of a time series model provided in an embodiment of the present application;

[0027] Figure 4 A schematic diagram of a device for training a time series model provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] Below, some terms in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0030] 1. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0031] 2. Contrastive learning is a self-supervised learning method that learns effective representations of data by maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs. In the fields of computer vision and natural language processing, contrastive learning has been proven to be a powerful unsupervised feature learning technique. In time series analysis, the goal of contrastive learning is to learn representations that can capture the intrinsic structure and patterns of time series while maintaining the temporal characteristics of time series data. By constructing positive and negative sample pairs, the model is trained to distinguish the similarities and differences between different time series. This method can help the model learn richer and more robust time series features, thereby improving the performance of downstream tasks such as classification, clustering, and prediction.

[0032] 3. Mask modeling is a self-supervised learning paradigm that learns data representation by masking part of the input data and training the model to predict these masked parts. This method has achieved great success in the field of natural language processing, especially in language models such as Bidirectional Encoder Representations from Transformers (BERT). In time series analysis, mask modeling can force the model to learn the deep features and intrinsic dependencies of time series data by randomly masking certain parts of the time series data and then using the unmasked parts to reconstruct the masked parts. This method can help the model capture the local features and long-term dependencies of time series data, and has potential application value for tasks such as time series prediction, classification, and clustering. In this way, the model can automatically learn useful feature representations from a large amount of unlabeled time series data without external annotations.

[0033] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0034] The application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. In the description of the present application, unless otherwise specified, the meaning of "multiple" is two or more.

[0035] Time series forecasting is a core topic that has long been the focus of industry and academia. Its core lies in identifying hidden patterns and trends from historical data and making forward-looking predictions about future behaviors or events. With the booming development of deep learning technology, the field of time series forecasting is undergoing a paradigm shift from traditional statistical models to those based on deep learning technology.

[0036] How to efficiently and intelligently extract key features from time series data and achieve high-precision prediction of future data points has become a problem that urgently needs in-depth optimization.

[0037] In order to solve the above problems, an embodiment of the present application provides a time series model training method. In this method, the time series data is divided into context data and target data, and data points are randomly masked in the target data. Wherein, the context data is historical data, and the target data is the predicted data point. A Transformer encoder including a self-supervised learning mechanism is used to distinguish positive sample pairs and negative sample pairs from a time series set based on a time contrast learning strategy. Wherein, the positive sample pair includes a continuous time series fragment of context data and target data. The negative sample pair includes a data set that is not adjacent in time or has low correlation in the time series data except the context data and the target data. The unmasked data points in the positive sample pair are input into the decoder, and the masked data points are predicted using a cross self-attention mechanism. The model training process is optimized with the goal of minimizing the preset contrast loss function to obtain a time series model. Wherein, the contrast loss function is calculated using negative sample pairs.

[0038] In this way, through the self-supervised learning mechanism, the Transformer encoder can automatically extract key features from time series data without the need for a large amount of labeled data, which can reduce the cost of data collection and processing. Randomly masking data points in the target data and training the time series model to predict these masked data points can enable the time series model to learn more robust and comprehensive representations, thereby improving the accuracy of predictions. By distinguishing between positive and negative sample pairs, the time series model can learn the temporality and correlation in the time series data, thereby improving its generalization ability in different scenarios, and achieving efficient and intelligent extraction of key features from time series data, thereby achieving high-precision prediction of future data points.

[0039] Schematic diagram of an application scenario of an optional time series model training method in an embodiment of the present application. Figure 1 As shown, it includes a server 100 and a terminal 101. The server 100 and the terminal 101 can be communicatively connected through a network to implement the time series model training method of the present application.

[0040] The user can use the server 100 to interact with the terminal 101 through the network, such as receiving or sending messages, etc. Various client applications can be installed on the terminal 101, such as program writing applications, web browser applications, search applications, etc.

[0041] It is understandable that in the embodiment of the present application, the server 100 can be implemented as an independent server or a server cluster composed of multiple servers. The terminal 101 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, desktop computers, etc.

[0042] Next, combine Figure 1 The scenario shown introduces the solution of the present application. For example, a flowchart of a time series model training method provided in an embodiment of the present application, optionally, Figure 2 The method flow shown can be Figure 1 The server 100 included in the illustrated scenario may also be executed by the terminal 101. Figure 2 The execution subject of the method flow shown is not limited. Specifically, the following steps may be included:

[0043] Step S201: divide the time series data into context data and target data according to a preset rule, and randomly mask data points in the target data.

[0044] Among them, context data is historical data and target data is predicted data points.

[0045] For example, time series data can be divided into context data and target data according to a certain time point (such as a certain year, month, and day). The context data is the events or observations that occurred before the current time point. The target data is the predicted data point in the time series, representing the future data point. For another example, if the target data is to predict the temperature in the next week, the data of the past few weeks can be used as context data.

[0046] In the above method, by dividing the time series data into context data and target data according to preset rules, the time series model can be provided with information about past states based on the context data, so that the time series model can predict future trends based on this information. Some data points are randomly selected and masked in the target data, so that the subsequent model can predict the values ​​of these masked data points based on the context data and other unmasked target data points, so that the time series model can learn richer representations and stronger prediction capabilities.

[0047] Optionally, before dividing the time series data into context data and target data, the time series data may be standardized to eliminate the dimensional differences between different data and improve the training efficiency and prediction performance of the model.

[0048] Step S202: input the time series data into a Transformer encoder that uses a self-supervised learning mechanism, and distinguish positive sample pairs and negative sample pairs from the time series data set based on a time contrast learning strategy.

[0049] Among them, the positive sample pair is a continuous time series segment containing context data and target data, and the negative sample pair is a set of data in the time series data that is not adjacent in time or has low correlation except for the context data and the target data.

[0050] It can be understood that the time contrastive learning strategy is a method of using the time order and correlation in time series data to construct positive sample pairs and negative sample pairs. The time contrastive learning strategy can use the time order and correlation in time series data to learn the representation of data by comparing data samples at different time points, and construct positive sample pairs and negative sample pairs.

[0051] In the above method, the time contrast learning strategy is used to distinguish positive sample pairs from negative sample pairs, which helps the time series model better capture the temporal characteristics and patterns in the time series data. At the same time, the time series model training method based on self-supervised learning does not require a large amount of labeled data, which can reduce the cost of data collection and processing.

[0052] Optionally, the Transformer encoder can contain two branches that share the same network structure and weights, which helps reduce the time series model parameters and enhance the generalization ability of the model. By sharing weights, the time series model is able to transfer the learned knowledge when processing context and target data. Since context and target data are usually correlated in time series, sharing weights helps the model capture this correlation and better utilize context information to predict target data.

[0053] For example, the composition of the time series model can be found in Figure 3 ,like Figure 3 As shown, the time series model includes:

[0054] 1. Data division:

[0055] Split the time series data into two parts: context data (historical data) and target data (the data points to be predicted).

[0056] 2. Patching:

[0057] The context data and target data are divided into multiple patches, which can be overlapping or non-overlapping. Each patch is mapped to the latent space through a linear layer and a positional encoding is added.

[0058] 3. Encoder:

[0059] The Transformer architecture with self-attention mechanism and feed-forward layer is used as the encoder, which has two branches, processing context data and target data respectively, and sharing weights.

[0060] The encoder includes an input embedding layer (Input Embedding) and N encoder layers. Each of the N encoder layers includes a self-attention mechanism layer (Self-Attention), a residual connection and normalization layer (Add&Norm) and a feedforward neural network layer (Feed Forward).

[0061] 4. Masking:

[0062] The representation of the target data is randomly masked, while the context data is not masked. This allows the model to reconstruct the masked target data segments based on the context data, thereby learning the temporal dependencies in time series data.

[0063] 5.Decoder:

[0064] The feature vectors (keys and values) output by the encoder for representing the context data and partially masked target data, as well as the query vector for representing the target data, are input into the decoder. The decoder uses a cross self-attention mechanism so that the target data can reconstruct the masked part through the context data.

[0065] The decoder includes an input embedding layer (Input Embedding) and N decoder layers. Each of the N decoder layers includes a self-attention mechanism layer (Self-Attention), a residual connection and normalization layer (Add&Norm) and a feedforward neural network layer (Feed Forward).

[0066] 6. Objective Function:

[0067] A reconstruction head is used to receive the output of the encoder, and the time series model training process is optimized by minimizing the preset contrast loss function to obtain a trained time series model.

[0068] Among them, the optimization objectives of temporal self-supervised learning for time series forecasting (TempSSL) include two parts: temporal masked modeling (TMM) and temporal contrastive learning (TCL).

[0069] 6.1TMM: Training time series models by minimizing the reconstruction loss, which is the difference between the reconstructed target data and the original target data.

[0070] 6.2TCL: Training time series models by minimizing the contrastive loss, which is calculated based on the similarity between the context and target data while taking into account the negative samples in the same batch of time series data.

[0071] Step S203: input the unmasked data points in the positive sample pair into the decoder, and use the cross self-attention mechanism to predict the masked data points, so as to optimize the model training process with the goal of minimizing the preset contrast loss function.

[0072] Among them, the contrast loss function is calculated using negative sample pairs.

[0073] In the above method, the cross self-attention mechanism in the Transformer model helps the model to keep its attention on the global context information during the prediction process and generate more accurate predictions. By minimizing the contrast loss function, the time series model can learn how to better predict the masked data points. Since the contrast loss function takes into account the relative relationship between positive and negative samples, the time series model will pay more attention to the features that are critical to distinguishing positive and negative samples during the prediction process, thereby improving the accuracy of the prediction.

[0074] Optionally, the contrastive loss function is a total loss function composed of the reconstruction loss for temporal masking modeling and the temporal contrastive loss for temporal contrastive learning.

[0075] Among them, reconstruction loss is a measure of the difference between the masked data points predicted by the model and the original data points. For example, the reconstruction loss can be obtained by calculating the cosine similarity between the predicted value and the true value. Temporal contrast loss is a measure of the model's ability to distinguish between positive sample pairs and negative sample pairs in feature space. For example, the temporal contrast loss can be obtained by calculating the difference between the similarity between positive sample pairs and the similarity between negative sample pairs.

[0076] Since reconstruction loss focuses on the model's ability to predict a single data point, while time contrast loss focuses on the model's ability to understand the overall structure and temporal relationship of the data, by considering both reconstruction loss and time contrast loss, the time series model can learn the characteristics and patterns of the data more comprehensively during training, thereby improving training efficiency and model performance.

[0077] Optionally, the weights of the reconstruction loss and the temporal contrast loss in the contrastive loss function are adjusted using hyperparameters.

[0078] In the above method, in the contrast loss function, by adjusting the weights of the reconstruction loss and the time contrast loss through hyperparameters, the influence of the reconstruction loss and the time contrast loss in the contrast loss function can be balanced, thereby controlling the trade-off between the prediction ability and the ability to understand the temporal relationship of the time series model.

[0079] Optionally, after the model training is completed, the time series model can be adjusted by using downstream task fine-tuning technology to obtain multiple sub-models corresponding to different categories of downstream tasks, thereby improving the flexibility and accuracy of the model in practical applications.

[0080] Figure 4 A schematic diagram of the structure of a time series model training device provided in an embodiment of the present application. Figure 4 As shown, the device includes: a data classification module 401 and a processing module 402.

[0081] A data classification module 401 is used to classify the time series data into context data and target data according to a preset rule, and randomly mask data points in the target data, wherein the context data is historical data and the target data is predicted data points;

[0082] The data classification module 401 is further used to input the time series data into a Transformer encoder using a self-supervised learning mechanism, and distinguish positive sample pairs and negative sample pairs from the time series data set based on a time contrast learning strategy, where the positive sample pairs are continuous time series segments including context data and target data, and the negative sample pairs are data sets in the time series data that are not adjacent in time or have low correlation except for the context data and the target data;

[0083] The processing module 402 is used to input the unmasked data points in the positive sample pair into the decoder, use the cross self-attention mechanism to predict the masked data points, optimize the model training process with the goal of minimizing the preset contrast loss function, and obtain the time series model. The contrast loss function is calculated using negative sample pairs.

[0084] Optionally, the above contrast loss function is a total loss function composed of the reconstruction loss of temporal masking modeling and the temporal contrast loss of temporal contrast learning

[0085] Optionally, the weights of the reconstruction loss and the temporal contrast loss in the contrast loss function are adjusted using hyperparameters.

[0086] Optionally, the above Transformer encoder contains two branches that share the same network structure and weights.

[0087] Optionally, the processing module 402 is further used for:

[0088] The downstream task fine-tuning technology is used to adjust the time series model to obtain multiple sub-models corresponding to different categories of downstream tasks.

[0089] Optionally, before dividing the time series data into context data and target data, the processing module 402 is further used to:

[0090] Standardize time series data.

[0091] Based on the same technical concept, an electronic device is also provided in an embodiment of the present application, and the electronic device can realize the functions of the aforementioned time series model training device.

[0092] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0093] At least one processor 501, and a memory 502 connected to the at least one processor 501. The specific connection medium between the processor 501 and the memory 502 is not limited in the embodiment of the present application. Figure 5 In the example, the processor 501 and the memory 502 are connected via a bus 500. The bus 500 is Figure 5 The connections between other components are shown in bold lines, and are not intended to be limiting. The bus 500 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 501 can also be called a controller, and there is no limitation on the name.

[0094] In the embodiment of the present application, the memory 502 stores instructions that can be executed by at least one processor 501. The at least one processor 501 can execute a time series model training method discussed above by executing the instructions stored in the memory 502. The processor 501 can implement Figure 4 The functions of each module in the device shown.

[0095] Among them, the processor 501 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 502 and calling data stored in the memory 502, the various functions of the device and processing data, the device can be monitored as a whole.

[0096] In one possible design, the processor 501 may include one or more processing units, and the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a driver interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0097] Processor 501 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of a time series model training method disclosed in the embodiments of the present application can be directly embodied as a hardware processor execution, or a combination of hardware and software modules in the processor.

[0098] The memory 502 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 502 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 502 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 502 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0099] By programming the processor 501, the code corresponding to the time series model training method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 2A time series model training method of the embodiment shown. How to design and program the processor 501 is a technique known to those skilled in the art and will not be described in detail here.

[0100] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0101] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a time series model training method in the above embodiment.

[0102] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0103] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 Flow or flows and / or boxes Figure 1 A device that performs the functions specified in a box or multiple boxes.

[0104] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 Flow or flows and / or boxes Figure 1 The functions specified in one or more boxes.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 Flow or flows and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0106] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A time series model training method, characterized in that: The method comprises: Dividing the time series data into context data and target data according to a preset rule, and randomly masking data points in the target data, wherein the context data is historical data and the target data is predicted data points; The time series data is input into a Transformer encoder using a self-supervised learning mechanism, and positive sample pairs and negative sample pairs are distinguished from the time series data set based on a time contrast learning strategy, wherein the positive sample pairs are continuous time series segments including the context data and the target data, and the negative sample pairs are data sets in the time series data other than the context data and the target data that are not adjacent in time or have low correlation; The unmasked data points in the positive sample pair are input into the decoder, and the masked data points are predicted using the cross self-attention mechanism. The time series model training process is optimized with the goal of minimizing a preset contrast loss function to obtain the time series model. The contrast loss function is calculated using the negative sample pair.

2. The method according to claim 1, characterized in that The contrastive loss function is a total loss function composed of the reconstruction loss of temporal masking modeling and the temporal contrastive loss of temporal contrastive learning.

3. The method according to claim 2, characterized in that The weights of the reconstruction loss and the temporal contrast loss in the contrast loss function are adjusted using hyperparameters.

4. The method according to claim 1, characterized in that: The Transformer encoder contains two branches that share the same network structure and weights.

5. The method according to claim 1, characterized in that The method further comprises: The time series model is adjusted by using downstream task fine-tuning technology to obtain multiple sub-models corresponding to different categories of downstream tasks.

6. The method according to claim 1, characterized in that Before dividing the time series data into context data and target data, the method further includes The time series data is standardized.

7. A time series model training device, characterized in that: The device comprises: A data classification module, used to divide the time series data into context data and target data according to preset rules, and randomly mask data points in the target data, wherein the context data is historical data and the target data is predicted data points; The data classification module further inputs the time series data into a Transformer encoder using a self-supervised learning mechanism, and distinguishes positive sample pairs and negative sample pairs from the time series data set based on a time contrast learning strategy, wherein the positive sample pairs are continuous time series segments including the context data and the target data, and the negative sample pairs are data sets in the time series data other than the context data and the target data that are not adjacent in time or have low correlation; A processing module is used to input the unmasked data points in the positive sample pair into the decoder, use the cross self-attention mechanism to predict the masked data points, optimize the model training process with the goal of minimizing a preset contrast loss function, and obtain the time series model. The contrast loss function is calculated using the negative sample pair.

8. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to cause the computer to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that When the computer program product is called by a computer, the computer executes the method according to any one of claims 1 to 6.

Images