Multi-analysis-task-oriented unsupervised construction method for power time sequence representation model
By masking the power time series data and training the autoencoder architecture, combined with multiple loss functions, a multi-tasking power time series list model is constructed, which solves the model island problem and realizes the different power time series analysis tasks. Model sharing and feature learning improve task adaptability and performance.
Patent Information
- Application Number
- CN202411092723.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-13
AI Technical Summary
The existing technology is difficult to realize model sharing among different power time series analysis tasks, resulting in model island phenomenon and inability to adapt to changes in multiple tasks. Traditional data sharing cannot meet the needs of complex tasks.
The unsupervised construction method of the power time series listing model for multi-analysis tasks is adopted. By masking the initial power time series data, the masking part complementary data pairs are generated. The deep characterization encoding module is trained using the autoencoder architecture, combined with reconstruction-based, Loss functions of context consistency and time consistency, build a general model.
It realizes model sharing among different power time series analysis tasks, improves the generalization ability of the model, is suitable for multi-task analysis, can comprehensively learn data characteristics, adapt to different task needs, and improves the efficiency and performance of the algorithm.
Smart Images
Figure CN119128738A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of time series representation learning, and more specifically, relates to an unsupervised construction method of a power time series representation model for multiple analysis tasks. Background Art
[0002] In actual application scenarios, a complex real-world time series analysis task often needs to be decomposed into multiple basic time series analysis tasks, and multiple subtasks are executed separately to complete the complex task together. At present, different algorithm models are usually established for different tasks and solidified in each independent system, resulting in the phenomenon of model islands, making it difficult to collaboratively handle complex tasks. At most, different methods can only achieve data sharing. Even if the data is exactly the same, model sharing cannot be achieved when completing different tasks. With the emergence of various scenarios, time series tasks have become more and more complex, and the problems that need to be solved often require multiple subtasks. Therefore, traditional data sharing can no longer meet the development trend of future applications, and sharing models between different time series analysis tasks will become inevitable.
[0003] Time series analysis tasks are diverse, and different tasks have different requirements for data features. For example, time series prediction tasks need to infer targets at specific timestamps and pay more attention to fine-grained sample context information. The goal of classification tasks is to distinguish samples of different categories and give priority to instance-level features of samples. When the same data is used for different tasks, the representation learned by a single model has strong limitations. Targeted algorithms are often required for different tasks, which limits the generalization performance of the model and makes it unable to adapt to changes in the target task. Therefore, it is necessary to propose a general representation learning method for time series multi-tasks.
[0004] With the improvement of the representation learning ability of the model, many research methods have begun to explore multi-task learning to further verify the representation learning ability of the model. Based on the multi-periodicity of time series, TimesNet models the intra-periodic and inter-periodic changes of time series, and performs five tasks such as prediction, missing value filling, and classification. TS2Vec implements hierarchical contrastive learning at the timestamp level and instance level respectively, and experiments are conducted on the tasks of time series classification, prediction, and anomaly detection. SimMTM recovers the original time series from multiple randomly masked time series, and experiments are conducted in prediction and classification tasks. Although the above models can be applied to a variety of tasks, they do not truly implement a unified multi-task model because the models used in different tasks are different. Summary of the invention
[0005] In view of the defects of the prior art and the need for improvement, the present invention provides an unsupervised construction method of an electric power time series characterization model for multiple analysis tasks, which aims to achieve the problem of sharing models among different electric power time series analysis tasks.
[0006] To achieve the above object, according to one aspect of the present invention, a method for unsupervised construction of a power time series representation model for multiple analysis tasks is provided, comprising:
[0007] Perform masking processing on each initial power time series data to generate a pair of masked data with complementary masked parts and the same length as the original power time series data; each pair of masked data is used as a training sample;
[0008] Constructing an autoencoder architecture, including a deep representation encoding module to be trained and a trained decoder; using the training sample to train the deep representation encoding module in the autoencoder architecture, and using the trained deep representation encoding module as a time series representation model for multiple analysis tasks;
[0009] Wherein, during the training process, the two masked data in each pair of masked data are respectively input into the two parameter-shared autoencoder architectures; the deep representation encoding module includes: a data embedding layer, used to generate a potential representation of each masked data; an encoder, used to extract a data deep representation including an overall deep representation and a detailed deep representation based on the potential representation; the decoder is used to reconstruct the corresponding initial power time series data based on the data deep representation; after each iteration, the following losses are calculated based on the potential representation and the overall deep representation: a regression loss based on reconstruction, a contrast loss based on context consistency, and a contrast loss based on time consistency.
[0010] Further, a masking strategy is adopted to perform the masking process, specifically:
[0011] Generate a 0-1 sequence with the same size as the original power time series data to be masked as a mask sequence; wherein the positions of "0" are randomly distributed and the ratio is preset; the position of "1" indicates that the data at the corresponding position is reserved data, and the corresponding position of "0" indicates that the original data is masked; invert the mask sequence, swap the positions of "0" and "1", and obtain another complementary mask sequence;
[0012] The original power time series data are multiplied by the two mask sequences respectively to obtain two masked data as the pair of masked data.
[0013] Furthermore, the data embedding layer module structure is a convolutional layer, the encoder is a Transformer encoder, and / or the decoder is a linear layer.
[0014] Furthermore, the total loss function L of the training process is:
[0015]
[0016] In the formula, and are respectively the reconstruction-based regression losses corresponding to the two autoencoder architectures s and w that share parameters, and the mean square error between the initial data corresponding to the masked part and the reconstructed data is used as the regression loss; is the temporal consistency-based contrast loss between the overall deep representation output by the encoder in the autoencoder architecture w and the latent representation output by the data embedding layer in the autoencoder architecture s under a pair of masked data inputs corresponding to the same initial power time series data; is the temporal consistency-based contrast loss between the overall deep representation output by the encoder in the autoencoder architecture s and the latent representation output by the data embedding layer in the autoencoder architecture w under a pair of masked data inputs corresponding to the same initial power time series data; L CC It is a contrast loss based on context consistency. In the calculation of this contrast loss, the two overall deep representations corresponding to the same initial power time series data are taken as a positive sample when calculating the contrast loss, and the two overall deep representations corresponding to any two initial power time series data in the same training batch are taken as a negative sample; α, β and γ represent the weights of the three losses respectively.
[0017] Furthermore, the reconstruction-based regression loss is specifically:
[0018]
[0019]
[0020] Where x represents the initial power time series data, x s ' and x w 'represents the reconstruction results corresponding to the output x reconstructed by the decoder in the two self-encoder architectures s and w of parameter sharing, respectively, mask s and mask w Respectively represent the 0-1 mask sequence corresponding to a pair of masked data of x;
[0021] The contrast loss based on context consistency is specifically:
[0022]
[0023] Among them, c s and c wrepresents the overall deep representation output by the encoders in the two autoencoder architectures s and w with parameter sharing, exp(*) represents an exponential function with base e, Represents the two overall deep representations corresponding to the i-th initial power time series data and The product of Represents one of the overall deep representations corresponding to the i-th initial power time series data One of the overall deep features corresponding to the jth initial power time series data in the same training batch The product of ; N represents the number of training samples in the same training batch;
[0024] The contrast loss based on time consistency is specifically:
[0025]
[0026]
[0027] Among them, z s and z w are the potential representations output by the data embedding layers in the two parameter-sharing autoencoder architectures s and w, respectively, Projection(*) T Representation projection transformation, which transforms the latent representation to the same dimension as the overall deep representation, and are the z corresponding to the i-th initial power time series data s and z w .
[0028] The present invention also provides a method for constructing a power time series analysis model, comprising:
[0029] According to different types of analysis tasks, a corresponding decoder network is constructed; and the decoder network and the unsupervised pre-trained power time series representation model are used to form a power time series analysis model to be trained; wherein the power time series representation model is constructed by the unsupervised construction method of the power time series representation model for multiple analysis tasks as described above;
[0030] According to the analysis task of the decoder network, the representation type of the output represented by the power time series characterization model is selected and determined for the input of the decoder network to train the power time series analysis model, wherein the training process can achieve fine-tuning of the power time series characterization model.
[0031] Further, when the analysis task is a classification task or an anomaly detection task, the representation type is a feature of the overall deep representation output by the encoder in the power time series representation model and a feature after pooling the detailed deep representation output by the encoder;
[0032] When the analysis task is a prediction task or a missing value filling task, the representation type is a detailed deep representation output by the encoder in the power time series representation model.
[0033] The present invention also provides a power time series analysis method, which uses the power time series analysis model constructed by the power time series analysis model construction method as described above to perform time series analysis.
[0034] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned unsupervised construction method of a power time series characterization model for multiple analysis tasks, the above-mentioned construction method of a power time series analysis model, or the above-mentioned power time series analysis method.
[0035] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the unsupervised construction method of a power time series characterization model for multiple analysis tasks as described above, the construction method of a power time series analysis model as described above, or the power time series analysis method as described above.
[0036] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:
[0037] (1) The method of the present invention is an unsupervised construction method for a power time series representation model for multiple analysis tasks, aiming to construct a general model for multiple analysis tasks for power time series data. Based on this model, it can be used to process different tasks of a certain power time series data, truly realizing a unified multi-task model. The method first performs masking processing on each initial power time series data to generate a pair of masked data with complementary masked parts and the same length as the original time series data as training samples, and then uses a data embedding layer and an encoder to generate multiple representations, including latent representation, overall deep representation and detailed deep representation. After that, during iteration, the reconstruction-based regression loss, the context-consistency-based contrast loss and the time-consistency-based contrast loss are combined to jointly constrain the model's ability to comprehensively learn data representation, so that the model has strong generalization ability and is suitable for multi-task analysis. Therefore, the present invention proposes an unsupervised representation learning method for time series multi-tasks to construct a multi-analysis task model, which can comprehensively learn the characteristics of power time series at multiple levels, including potential representation, overall deep representation and detailed deep representation. Downstream tasks can select one or more representations to perform analysis tasks according to actual task needs, that is, reasonably select features that better meet task requirements, thereby providing a unified pre-training model for different downstream tasks, so that the model shows strong effectiveness in different tasks.
[0038] (2) The present invention also proposes a method for constructing an electric power time series analysis model, which realizes the feature selection function in different downstream tasks, so that the constructed analysis model can combine the comprehensive data features currently extracted, and reasonably select features that are more in line with the current task requirements, and select the best features in a targeted manner. The present invention can show strong performance in different electric power time series analysis tasks, and has versatility and effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 A flowchart of an unsupervised construction method of a power time series characterization model for multiple analysis tasks provided by an embodiment of the present invention;
[0040] Figure 2 A structural diagram of a representation learning network provided by an embodiment of the present invention;
[0041] Figure 3 A flowchart of pre-training of a power time series characterization model provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0042] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0043] Embodiment 1
[0044] An unsupervised construction method for power time series representation model for multiple analysis tasks, such as Figure 1 As shown, including:
[0045] Perform masking processing on each initial power time series data to generate a pair of masked data with complementary masked parts and the same length as the original time series data (the masked part can use 0 to replace the original data value); each pair of masked data is used as a training sample;
[0046] Construct an autoencoder architecture, including a deep representation encoding module to be trained and a trained decoder module; use training samples to train the deep representation encoding module in the autoencoder architecture, and use the trained deep representation encoding module as a time series representation model for multiple analysis tasks;
[0047] Among them, during the training process, the two masked data in each pair of masked data are respectively input into two parameter-shared autoencoder architectures; the deep representation encoding module includes: a data embedding layer, which is used to generate a potential representation of each masked data; an encoder, which is used to extract a data deep representation including an overall deep representation and a detailed deep representation based on the potential representation; a decoder is used to reconstruct the corresponding initial power time series data based on the data deep representation; after each iteration, the following losses are calculated based on the potential representation and the overall deep representation: a regression loss based on reconstruction, a contrast loss based on context consistency, and a contrast loss based on time consistency.
[0048] The "overall deep features" described in the method are derived from the summary of the detailed deep features, and the "detailed deep features" are derived from the data itself. The deep representation output by the encoder includes the overall deep representation and the detailed deep representation, where the overall deep representation is used to calculate the contrast loss. When the model is used for downstream tasks, the overall deep representation is used to complete classification tasks; the detailed deep representation is used by the decoder to calculate the reconstruction loss. When the model is used for downstream tasks, the detailed deep representation is used to complete prediction tasks.
[0049] This method uses a pre-training approach, with an autoencoder as the model framework, and combines multiple task objectives to jointly constrain the pre-training model's ability to fully learn data representation, including a regression task based on reconstruction, a contrast task based on contextual consistency, and a contrast task based on temporal consistency. The multi-task pre-training approach enables the model to have a strong generalization ability, extract more comprehensive data features, and adapt more quickly to changes in downstream task requirements, expanding the scope of application of the model.
[0050] This method provides an unsupervised representation learning method for time series multi-tasks. Through the unsupervised representation learning method, the model's ability to comprehensively learn power data representation is improved, and it can fully learn the characteristics of power time series at multiple levels, providing a unified pre-training model for different downstream tasks, solving the model island problem in practical applications, and improving the efficiency of the algorithm.
[0051] The method may also include the following steps: collecting power time series at set sampling time intervals, and performing data preprocessing, wherein the preprocessing operations include missing value filling, outlier processing, and data normalization. Specifically, in the pre-training process, an unsupervised representation learning method is adopted, which only requires the collection of a large number of time series, and does not require other information such as data labels; the missing values of the power time series are filled, and the missing values of the data are reasonably filled to reduce the impact of missing data on model performance; outlier processing is performed on the time series, and a reasonable threshold is set, and the numbers whose data range exceeds the threshold are regarded as outliers, and they are replaced with reasonable values according to the data range; the time series is normalized to reduce the impact of the large difference in the amplitude range of different data on model performance.
[0052] As an example, the sampling time interval for power time series data is preferably set to 1 hour, and 336 data are output as a group every 14 days. Missing value filling is preferably 0 filling. If the missing value exists before and after the missing value, the average of the before and after values is taken. The preferred threshold for outlier processing is the sequence mean plus or minus twice the variance, and the data exceeding the threshold is replaced by the threshold. The data normalization method is preferably mean-variance normalization.
[0053] The deep representation encoding module is trained until convergence, and the model structure and parameters of the deep representation encoding module are retained as the trained representation learning model to complete different downstream tasks.
[0054] As a preferred implementation, a masking strategy is used to perform the above masking process, specifically:
[0055] Generate a 0-1 sequence with the same size as the original time series data to be masked as a mask sequence; wherein the positions of "0" are randomly distributed and the ratio is preset; the position of "1" indicates that the data at the corresponding position is reserved data, and the corresponding position of "0" indicates that the original data is masked; invert the mask sequence, swap the positions of "0" and "1", and obtain another complementary mask sequence;
[0056] The original power time series data are multiplied by the two mask sequences respectively to obtain two masked data as the pair of masked data.
[0057] It should be noted that the masking ratios of the two pairs of masked data may be different or the same.
[0058] As a preferred implementation, the data embedding layer module structure is a convolutional layer, the encoder is a Transformer encoder, and / or the decoder is a linear layer.
[0059] As a preferred implementation, the total loss function L of the above training process is:
[0060]
[0061] In the formula, and The reconstruction-based regression loss corresponding to the two parameter-sharing autoencoder architectures s and w respectively adopts the mean square error between the initial data corresponding to the masked part and the reconstructed data as the regression loss, that is, the reconstruction-based regression loss is the mean square error (MSE) between the decoder output and the original unmasked data, and only the masked position is calculated; is the temporal consistency-based contrast loss between the overall deep representation output by the encoder in the autoencoder architecture w and the latent representation output by the data embedding layer in the autoencoder architecture s under a pair of masked data inputs corresponding to the same initial power time series data; L is the contrast loss based on time consistency between the overall deep representation output by the encoder in the autoencoder architecture s and the potential representation output by the data embedding layer in the autoencoder architecture w under a pair of masked data inputs corresponding to the same initial power time series data. That is, as shown in Figure 2, the contrast loss based on time consistency is to regard the two masked data transformed from the same time series data as a pair of positive samples, and the masked data corresponding to other time series data in the same training batch as a pair of negative samples, and calculate the contrast loss between the overall features of the data output by the encoder; L CCis a contrast loss based on context consistency. In the calculation of this contrast loss, the two overall deep representations corresponding to the same initial power time series data are used as a positive sample when calculating the contrast loss, and the two overall deep representations corresponding to any two initial power time series data in the same training batch are used as a negative sample. That is, Figure 2 As shown in Figure 1, the contrast loss based on context consistency uses the projection features extracted by the data embedding layer to perform cross-view comparison with the overall features of the data output by the encoder; α, β, and γ represent the weights of the three losses, respectively.
[0062] Specifically, Figure 2 The representation learning network structure diagram shown in the figure specifically includes:
[0063] The original input Input is transformed into two masked inputs Input1 and Input2 through a masking strategy. The two inputs are respectively passed through a parameter-sharing data embedding layer, an encoder, and a decoder to obtain potential representations Token1 and Token2 obtained by the data embedding layer, overall deep features Cls Token1 and Cls Token2 obtained by the encoder, and detailed deep features Series Token1 and Series Token2 obtained by the encoder, and reconstruction results Output1 and Output2 obtained by the decoder.
[0064] The model loss during pre-training is calculated by the regression loss based on reconstruction and Contrastive loss based on temporal consistency and And the contrast loss L based on context consistency CC The total loss L is constituted. α, β and γ represent the weights of the three losses respectively, and the preferred weight values of this example are all set to 1.
[0065] More specifically, the regression loss based on reconstruction is:
[0066]
[0067]
[0068] Where x represents the initial power time series data, x s ' and x w 'represents the reconstruction results corresponding to the output x reconstructed by the decoder in the two self-encoder architectures s and w of parameter sharing, respectively, mask s and mask w Respectively represent the 0-1 mask sequence corresponding to a pair of masked data of x;
[0069] The contrast loss based on context consistency is specifically:
[0070]
[0071] Among them, c s and c w represents the overall deep representation output by the encoders in the two autoencoder architectures s and w with parameter sharing, exp(*) represents an exponential function with base e, Represents the two overall deep representations corresponding to the i-th initial power time series data and The product of Represents one of the overall deep representations corresponding to the i-th initial power time series data One of the overall deep features corresponding to the jth initial power time series data in the same training batch The product of ; N represents the number of training samples in the same training batch;
[0072] The contrast loss based on temporal consistency is specifically:
[0073]
[0074]
[0075] Among them, z s and z w are the potential representations output by the data embedding layers in the two parameter-sharing autoencoder architectures s and w, respectively, Projection(*) T Representation projection transformation, which transforms the latent representation to the same dimension as the overall deep representation, and are the z corresponding to the i-th initial power time series data s and z w .
[0076] Embodiment 2
[0077] A method for constructing a power time series analysis model, comprising:
[0078] According to different types of analysis tasks, a corresponding decoder network is constructed; and the decoder network and the unsupervised pre-trained power time series representation model are used to form a power time series analysis model to be trained; wherein the time series representation model is constructed by an unsupervised construction method for a power time series representation model for multiple analysis tasks as described in Example 1;
[0079] According to the analysis task of the decoder network, the representation type of the output represented by the power time series representation model is selected and determined for the input of the decoder network to train the power time series analysis model, wherein the training process can achieve fine-tuning of the power time series representation model.
[0080] like Figure 3 As shown, a flowchart of using the power time series representation model pre-trained in Example 1 for training different downstream tasks specifically includes:
[0081] Load the pre-trained representation model, determine the type of downstream tasks, and design and train corresponding decoder networks for different types.
[0082] Among them, exemplarily, when the downstream task is classification, the preferred classification representation is the overall representation of the data output by the encoder and the representation obtained after pooling the detail features, and the preferred task evaluation indicator is the classification accuracy ACC; when the downstream task is prediction, the preferred prediction representation is the detail feature of the data output by the encoder, and the preferred task evaluation indicator is the mean square error MSE between the predicted value and the target value; when the downstream task is missing value filling, the preferred filling representation is the detail feature of the data output by the encoder, and the preferred task evaluation indicator is the mean square error MSE between the filling value and the target value; when the downstream task is anomaly detection, the preferred anomaly detection representation is the overall feature of the data output by the encoder and the feature after pooling the detail feature pool, the anomaly detection task output is the data representation, the preferred dimension is 128, and the preferred task evaluation indicator is AUC.
[0083] In general, the present invention is proposed to solve the problem of sharing models between different power time series analysis tasks. In order to solve this problem, the present invention adopts a multi-task oriented pre-training method and feature combination strategy: 1) The multi-task oriented pre-training method combines the reconstruction-based regression task, the context consistency-based comparison task and the time consistency-based comparison task in the pre-training stage to jointly constrain the pre-training model's ability to comprehensively learn data representation, so that the model has a strong generalization ability. 2) The feature combination strategy reasonably selects features that are more in line with the task requirements in different downstream tasks, so that the model shows strong effectiveness in different tasks. In addition, the present invention can also solve the problem of model representation learning. By adopting a multi-task oriented pre-training method, it can use corresponding task goals or a combination of task goals to constrain in the pre-training stage according to the needs of subsequent tasks, and enhance the model's representation learning ability in this type of task, thereby showing excellent performance in the task.
[0084] The relevant technical solution is the same as that in Embodiment 1 and will not be described in detail here.
[0085] Embodiment 3
[0086] A power time series analysis method uses the power time series analysis model constructed by the power time series analysis model construction method described in the second embodiment to perform time series analysis.
[0087] The relevant technical solutions are the same as those in Embodiment 1 and Embodiment 2 and will not be described in detail here.
[0088] In order to better illustrate the effectiveness of the above embodiments, the following examples are given:
[0089] The preferred downstream time series analysis tasks in this example are: missing value filling, prediction, classification, and anomaly detection.
[0090] Specifically, the preferred evaluation index for missing value filling is the mean square error (MSE) between the filled value and the actual value, and the smaller the value, the better. The preferred data set is the Electricity data set in the UCI database, which contains electricity consumption data of 321 large industrial users from a certain three years, with a sampling interval of 15 minutes. The preferred experimental parameters are: data length is 504, missing rate is 12.5%, batch size is set to 64, training rounds are 100, optimizer uses AdamW, the initial value of the learning rate is set to 0.001, warmup is used to update the learning rate, and after the 5th round, the learning rate is decayed according to the cosine function.
[0091] The preferred evaluation index for prediction is the mean square error (MSE) between the predicted value and the actual value. The smaller the value, the better. The preferred data set is the Electricity data set in the UCI database. The preferred experimental parameters are: data length is 504, prediction length is 96, batch size is set to 64, training rounds are 100, optimizer uses AdamW, the initial value of the learning rate is set to 0.001, warmup is used to update the learning rate, and the learning rate is decayed according to the cosine function after the 5th round.
[0092] The preferred evaluation index for classification is classification accuracy ACC, and the larger the value, the better. The preferred dataset is the ItalyPowerDemand dataset in the UCR database, which is 12 months of electricity demand data from Italy. The task is to distinguish between October to March and April to September. Data is collected every 15 days, the length of each data is 24, and the category is 2. The preferred experimental parameters are: the batch size is set to 32, the training round is 1000, the optimizer uses AdamW, the initial value of the learning rate is set to 0.002, the learning rate is updated by warmup, and the learning rate is decayed according to the cosine function after the 5th round.
[0093] The preferred evaluation index for anomaly detection is the AUC, which is a special index for binary classification tasks. AUC represents the area under the ROC curve, and the larger the value, the better. The preferred data set is the ElectricDevices data set in the UCR database, which includes electricity consumption data of 7 types of household items such as TVs, kettles, and refrigerators. It is classified only according to the electricity consumption every 15 minutes, and the length of each data is 96. The preferred experimental parameters are: the batch size is set to 32, the training round is 100, the optimizer uses AdamW, the initial value of the learning rate is set to 0.002, the learning rate is updated by warmup, and the learning rate is decayed according to the cosine function after the 5th round.
[0094] The experimental results are shown in Table 1. The representation learning model pre-trained in this example is at the best level in all four tasks compared with the randomly initialized model.
[0095] Table 1 Experimental results of this example on multiple tasks
[0096]
[0097] It should be noted that the time series data involved in the present invention may be time series data in the field of electricity. According to the actual task scenario, the original time series data is collected, and after preprocessing, a training sample training is constructed according to the method of the first embodiment to obtain a characterization model, which can be used for multi-task analysis of the corresponding scenario.
[0098] Embodiment 4
[0099] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned unsupervised construction method of a time series representation model for multiple analysis tasks, the above-mentioned construction method of a time series analysis model, or the above-mentioned time series analysis method.
[0100] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The memory may be used to store computer programs and / or modules, and the processor may perform various functions of the electronic device by running or executing computer programs and / or modules stored in the memory, and calling data stored in the memory.
[0101] The relevant technical solutions are the same as those in Embodiment 1, Embodiment 2 and Embodiment 3, and will not be described in detail here.
[0102] Embodiment 5
[0103] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the unsupervised construction method of a time series characterization model for multiple analysis tasks as described above, the construction method of a time series analysis model as described above, or the time series analysis method as described above.
[0104] Specifically, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, an internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0105] The relevant technical solutions are the same as those in Embodiment 1, Embodiment 2 and Embodiment 3, and will not be described in detail here.
[0106] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. An unsupervised construction method of a power time series representation model for multiple analysis tasks, characterized in that: include: Perform masking processing on each initial power time series data to generate a pair of masked data with complementary masked parts and the same length as the original time series data as a training sample; Constructing an autoencoder architecture, including a deep representation encoding module to be trained and a trained decoder; using the training samples to train the deep representation encoding module in the autoencoder architecture, and using the trained deep representation encoding module as a power time series representation model for multiple analysis tasks; Wherein, during the training process, the two masked data in each pair of masked data are respectively input into the two parameter-shared autoencoder architectures; the deep representation encoding module includes: a data embedding layer, used to generate a potential representation of each masked data; an encoder, used to extract a data deep representation including an overall deep representation and a detailed deep representation based on the potential representation; the decoder is used to reconstruct the corresponding initial power time series data based on the data deep representation; after each iteration, the following losses are calculated based on the potential representation and the overall deep representation: a regression loss based on reconstruction, a contrast loss based on context consistency, and a contrast loss based on time consistency.
2. The unsupervised construction method of the power time series characterization model according to claim 1 is characterized in that: The masking process is performed using a masking strategy, specifically: Generate a 0-1 sequence with the same size as the original time series data to be masked as a mask sequence; wherein the positions of "0" are randomly distributed and the ratio is preset; the position of "1" indicates that the data at the corresponding position is reserved data, and the position of "0" indicates that the original data is masked; invert the mask sequence, swap the positions of "0" and "1", and obtain another complementary mask sequence; The original power time series data are multiplied by the two mask sequences respectively to obtain two masked data as the pair of masked data.
3. The unsupervised construction method of the power time series representation model according to claim 1 is characterized in that: The data embedding layer module structure is a convolutional layer, the encoder is a Transformer encoder, and / or the decoder is a linear layer.
4. The unsupervised construction method of the power time series characterization model according to claim 1 is characterized in that: The total loss function L of the training process is: In the formula, and are respectively the reconstruction-based regression losses corresponding to the two autoencoder architectures s and w that share parameters, and the mean square error between the initial data corresponding to the masked part and the reconstructed data is used as the regression loss; is the temporal consistency-based contrast loss between the overall deep representation output by the encoder in the autoencoder architecture w and the latent representation output by the data embedding layer in the autoencoder architecture s under a pair of masked data inputs corresponding to the same initial power time series data; is the temporal consistency-based contrast loss between the overall deep representation output by the encoder in the autoencoder architecture s and the latent representation output by the data embedding layer in the autoencoder architecture w under a pair of masked data inputs corresponding to the same initial power time series data; L CC It is a contrast loss based on context consistency. In the calculation of this contrast loss, the two overall deep representations corresponding to the same initial power time series data are taken as a positive sample when calculating the contrast loss, and the two overall deep representations corresponding to any two initial power time series data in the same training batch are taken as a negative sample; α, β and γ represent the weights of the three losses respectively.
5. The unsupervised construction method of the power time series characterization model according to claim 4 is characterized in that: The reconstruction-based regression loss is specifically: Where x represents the initial power time series data, x s ' and x w 'represents the reconstruction results corresponding to the output x reconstructed by the decoder in the two self-encoder architectures s and w of parameter sharing, respectively, mask s and mask w Respectively represent the 0-1 mask sequence corresponding to a pair of masked data of x; The contrast loss based on context consistency is specifically: Among them, c s and c w represents the overall deep representation output by the encoders in the two autoencoder architectures s and w with parameter sharing, exp(*) represents an exponential function with base e, Represents the two overall deep representations corresponding to the i-th initial power time series data and The product of Represents one of the overall deep representations corresponding to the i-th initial power time series data One of the overall deep features corresponding to the jth initial power time series data in the same training batch The product of ; N represents the number of training samples in the same training batch; The contrast loss based on time consistency is specifically: Among them, z s and z w are the potential representations output by the data embedding layers in the two self-encoder architectures s and w with parameter sharing, Projection(*) T Representation projection transformation, which transforms the latent representation to the same dimension as the overall deep representation, and are the z corresponding to the i-th initial power time series data s and z w .
6. A method for constructing a power time series analysis model, characterized in that: include: According to different types of analysis tasks, build corresponding decoder networks; The decoder network and the unsupervised pre-trained power time series characterization model constitute a power time series analysis model to be trained; wherein the power time series characterization model is constructed by the unsupervised construction method of a power time series characterization model for multiple analysis tasks as described in any one of claims 1 to 5; According to the analysis task of the decoder network, the representation type of the output represented by the power time series characterization model is selected and determined for the input of the decoder network to train the power time series analysis model, wherein the training process can achieve fine-tuning of the power time series characterization model.
7. The construction method according to claim 6, characterized in that: When the analysis task is a classification task or an anomaly detection task, the representation type is the overall deep representation output by the encoder in the power time series representation model and the feature after pooling the detailed deep representation output by the encoder; When the analysis task is a prediction task or a missing value filling task, the representation type is a detailed deep representation output by the encoder in the power time series representation model.
8. A method for analyzing power time series, characterized in that: The time series analysis is performed using the power time series analysis model constructed by the method for constructing a power time series analysis model as described in claim 6 or 7.
9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, it implements an unsupervised construction method of a power time series characterization model for multiple analysis tasks as described in any one of claims 1 to 5, a construction method of a power time series analysis model as described in claim 6 or 7, or a power time series analysis method as described in claim 8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements an unsupervised construction method of a power time series characterization model for multiple analysis tasks as described in any one of claims 1 to 5, a construction method of a power time series analysis model as described in claim 6 or 7, or a power time series analysis method as described in claim 8.