A method of augmenting a sequence dataset

Abstract

The disclosure discloses a method for expanding a sequence data set, comprising: obtaining a distance matrix of two-by-two sequences in an original sequence data set; obtaining an initial average sequence and an initialized weight array based on the original sequence data set and the distance matrix; obtaining a generated sequence based on the initial average sequence and the initialized weight array; constructing a sequence observation value generation model, and taking the generated sequence as an input of the model, and outputting an observation value corresponding to the generated sequence, so that the generated sequence and the observation value corresponding thereto form a complete sequence data set; and combining the complete sequence data set with the original sequence data set, and performing deduplication to complete expansion of the sequence data set.

Description

Technical Field

[0001] This disclosure pertains to the field of deep learning dataset augmentation, specifically relating to a method for augmenting sequence datasets. Background Technology

[0002] In the field of deep learning, models often perform well on test data when there is sufficient high-quality training data. However, in some domains, acquiring datasets is very costly. For example, acquiring lithium battery charge / discharge datasets requires performing hundreds or thousands of charge / discharge operations on each lithium battery under different fast-charging conditions and the same discharge conditions. Obtaining a sufficiently large lithium battery dataset requires a huge investment of human and material resources. Therefore, small datasets may exist, which can lead to problems such as overfitting on the training set, weak generalization ability on the test set, outliers, and data imbalance. Traditional methods for handling small datasets, such as splitting the dataset, adding noise, and model simplification, can alleviate overfitting, but they cannot expand the original dataset. Adversarial neural networks, which have emerged in recent years, have shown satisfactory results in expanding image datasets, but their performance on small sequence datasets is less than ideal.

[0003] Dynamic Time Warping Barycenter Averaging (DBA) is a global averaging method. DBA iteratively refines an initial (potentially arbitrary) averaging sequence to minimize its squared distance to the average sequence (Dynamic Time Warping, DTW). Compared to traditional nonlinear alignment and averaging filters (NLAAF) and prioritized shape averaging (PSA), DBA boasts the best time performance, with a time complexity of Θ(I·N·T). 2 ), where I represents the number of iterations in the DBA method; while the time complexity of the NLAAF method is Θ(N·T). 3 The time complexity of the PSA method is Θ(N·T). 3 +N 2 ·T 2 In this context, N represents the number of sequences, and T represents the length of each sequence. However, the DBA method can only generate sequence data for a given category and cannot generate the observations corresponding to each sequence. Therefore, implementing sequence data generation based on the dynamic time-warped centroid averaging method is of great significance for predicting sequence data in small datasets. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this disclosure is to provide a method for expanding a sequence dataset. This method can generate observation values ​​corresponding to each sequence in the original sequence dataset, thereby expanding the dataset.

[0005] To achieve the above objectives, this disclosure provides the following technical solutions:

[0006] A method for expanding a sequence dataset includes the following steps:

[0007] S100: Obtain the distance matrix between pairs of sequences in the original sequence dataset;

[0008] S200: Based on the original sequence dataset and distance matrix, obtain the initial average sequence and the initialized weight array;

[0009] S300: Based on the initial average sequence and the initialized weight array, obtain the generated sequence;

[0010] S400: Construct a sequence observation generation model, take the generated sequence as the input of the model, and output the observations corresponding to the generated sequence. The generated sequence and its corresponding observations form a complete sequence dataset.

[0011] S500: Merges the complete sequence dataset with the original sequence dataset, and after deduplication, it completes the expansion of the original sequence dataset.

[0012] Preferably, in step S400, the sequence observation generation model includes:

[0013] An autoencoder is used to extract feature information from the generated sequence;

[0014] The Transformer is used to output observations corresponding to the generated sequence based on the extracted feature information of the generated sequence.

[0015] Preferably, the autoencoder includes an encoder section and a decoder section, wherein,

[0016] The encoder section includes a first input layer, a first fully connected layer, and a second fully connected layer;

[0017] The decoder consists of a second input layer, a third fully connected layer, and an output layer.

[0018] Preferably, the Transformer includes:

[0019] The system consists of a location encoding layer, a multi-head attention layer, a first residual connection and normalization layer, a feedforward linear layer, a second residual connection and normalization layer, and a fully connected layer.

[0020] Preferably, in step S400, the training process of the sequence observation generation model includes:

[0021] S401: Initialize the model's weights and biases;

[0022] S402: Perform forward iterative computation on the generated sequence and output the observed values;

[0023] S403: Calculate the output error using the model loss function;

[0024] S404: Backpropagation is performed based on the error to update the model's weights and biases until the calculated value of the loss function tends to stabilize.

[0025] Preferably, in step S500, the deduplication of the merged complete sequence dataset and the original sequence dataset is performed in the following way: after mixing the complete sequence dataset and the original sequence dataset, sort them according to the size of the observation value corresponding to each sequence, and remove the generated sequences that have duplicate observation values ​​with the original sequence dataset.

[0026] Preferably, the original sequence dataset needs to be standardized before performing step S100.

[0027] Preferably, the original sequence dataset is standardized using the standard score method or the min-max standardization method.

[0028] Preferably, after obtaining the generated sequence through step S300, the generated sequence also needs to be smoothed.

[0029] Preferably, smoothing the generated sequence includes: performing k-th order polynomial fitting on the data points within a sliding window of length W in the generated sequence.

[0030] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0031] 1. Using the dynamic time warping algorithm to calculate the similarity between two sequences to represent the distance between them can solve the problem that traditional distance calculation methods cannot calculate the misalignment of sequences on the x-axis.

[0032] 2. Compared with the dynamic time warping centroid method, the selective averaging method can give greater weight to sequences with similar shapes to the centroid sequence. It focuses on sequences with similar shapes to the initial centroid sequence, avoiding the situation where sequences with significantly different shapes from the centroid sequence have too much influence on the centroid update, thus causing the generated data to lose its original shape features.

[0033] 3. By extracting sequence features through an autoencoder, the dimensionality of the input data is reduced and useful features are extracted.

[0034] 4. By using the Transformer network architecture, the shortcomings of RNN models in parallel computation and CNN models in shallow receptive fields can be overcome, and the training speed is also better than neural network architectures based on recurrent or convolutional layers. Attached Figure Description

[0035] Figure 1 This is a flowchart of a method for expanding a sequence dataset according to an embodiment of this disclosure;

[0036] Figure 2 This is an architecture diagram of a sequence observation generation model provided in another embodiment of this disclosure;

[0037] Figure 3 This is an architecture diagram of a self-encoder provided in another embodiment of this disclosure;

[0038] Figure 4 This is a schematic diagram of a generated sequence provided in another embodiment of this disclosure. Detailed Implementation

[0039] The following will refer to the appendix. Figures 1 to 4 Specific embodiments of this disclosure are described in detail. While specific embodiments of this disclosure are shown in the accompanying drawings, it should be understood that this disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.

[0040] It should be noted that certain terms are used in the specification and claims to refer to specific components. Those skilled in the art will understand that different terms may be used to refer to the same component. This specification and claims do not distinguish components based on differences in terminology, but rather on differences in function. The terms "comprising" or "including" used throughout the specification and claims are open-ended and should be interpreted as "comprising but not limited to." The following descriptions are preferred embodiments for carrying out this disclosure; however, these descriptions are for the purpose of understanding the general principles of the specification and are not intended to limit the scope of this disclosure. The scope of protection of this disclosure is determined by the appended claims.

[0041] To facilitate understanding of the embodiments of this disclosure, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. The accompanying drawings do not constitute a limitation on the embodiments of this disclosure.

[0042] In one embodiment, such as Figure 1 As shown, this disclosure provides a method for expanding a sequence dataset, including the following steps:

[0043] S100: The original sequence dataset is used as the input of the dynamic time warping algorithm, and the similarity between pairs of sequences in the original sequence dataset is output. This similarity is used as the distance metric between pairs of sequences to obtain the distance matrix D representing the distance between pairs of sequences in the original sequence dataset.

[0044] In this step, the length of the original sequence data is L, and the size of the distance matrix D is M×M (M is the number of sequence data in the original data). D(i,j) represents the distance from sequence i to sequence j, where i < M and j < M.

[0045] The pseudocode for calculating the distance matrix D is as follows:

[0046]

[0047]

[0048] In this step, the similarity between pairs of sequences is calculated by using the dynamic time warping algorithm to represent the distance between pairs of sequences. This solves the problem that traditional distance calculation methods (such as Euclidean distance algorithm, Manhattan distance algorithm, etc.) cannot calculate the distance between sequences because the sequences are not aligned on the x-axis.

[0049] It should be noted that the dynamic time warping algorithm in this step is as follows:

[0050] For two sequences I and J of different lengths, find a time warping function j = W(i) that non-linearly maps the time axis i of the test vector I to the time axis j of the template J, and makes the function satisfy the condition that the sum of the distances of all vectors is minimized under optimal time warping.

[0051] By using the dynamic time warping algorithm, the sequence can be locally scaled on the time axis, so that two sequences of different lengths can be better aligned on the time axis, thereby achieving the purpose of calculating the similarity between sequences of different lengths.

[0052] S200: The original sequence dataset and distance matrix D are used as inputs to the selective averaging algorithm to obtain the initial average sequence and the initialized weight array;

[0053] In this step, the initial averaging sequence is randomly selected from the original sequence dataset, and the size of the weight array is 1×M (M is the number of sequence data in the original data). The selective averaging algorithm assigns a weight of 0.5 to the random centroid sequence; the number of nearest neighbor sequences selected from the centroid sequence is 15, and 4 sequences are randomly selected from them, with a total weight of 0.3; after removing the centroid sequence and the 4 random nearest neighbor sequences, the total weight assigned to the other sequences in the original sequence dataset is 0.2.

[0054] The pseudocode for the initialization process of the weight array (i.e., the selective averaging algorithm) is as follows:

[0055]

[0056]

[0057] Among them, W barycenter =0.5, W k_subset =0.3, W other =0.2.

[0058] S300: Based on the initial average sequence and the initialized weight array, a new average sequence is obtained through the dynamic time warping centroid averaging algorithm, i.e., the generated sequence;

[0059] In this step, the one-iteration process of the dynamic time warp centroid averaging algorithm is as follows:

[0060] 1. Calculate the distance matrix Dis between each individual sequence and the initial average sequence that needs to be updated, according to the aforementioned code, in order to find the correlation between the average sequence coordinates and the sequence set coordinates;

[0061] 2. Update each coordinate of the average sequence according to the weight array obtained in step S300 (that is, multiply each coordinate of the average sequence by its corresponding weight) to the centroid of the coordinate associated with it.

[0062] S400: Construct a sequence observation generation model, take the generated sequence as the input of the model, and output the observations corresponding to the generated sequence. The generated sequence and its corresponding observations form a complete sequence dataset.

[0063] In this step, such as Figure 2 As shown, the sequence observation generation model includes an autoencoder and a Transformer. The autoencoder encodes the high-dimensional generated sequence into low-dimensional latent variables, enabling the neural network to learn the most informative features to obtain the feature information of the original high-dimensional input sequence. The Transformer can extract the temporal features of the feature sequence output by the autoencoder and provide corresponding predicted values.

[0064] Among them, such as Figure 3 As shown, the autoencoder includes an encoder part and a decoder part. The encoder part includes a first input layer, a first fully connected layer and a second fully connected layer; the decoder part includes a second input layer, a third fully connected layer and an output layer.

[0065] The Transformer consists of: a positional encoding layer, a multi-head attention layer, a first residual connection and normalization layer, a feedforward linear layer, a second residual connection and normalization layer, and a fully connected layer.

[0066] In this step, the Transformer network architecture can compensate for the shortcomings of RNN models in parallel computation and CNN models in terms of small receptive fields in shallow layers. The reason is that in the Transformer network structure, the computation at time t does not depend on the computation result at time t-1, so there is no sequence dependency. This makes up for the shortcoming of RNNs in parallel computation due to sequence dependencies. When using traditional convolutional neural networks (CNNs), they cannot capture long-distance features due to their small receptive fields. However, the self-attention mechanism in Transformers enables the current data node in the sequence to be connected with any data node, thereby solving the problem of long-distance dependent features.

[0067] S500: Merges the complete sequence dataset with the original sequence dataset, and after deduplication, it completes the expansion of the original sequence dataset.

[0068] In this step, the deduplication of the merged complete sequence dataset and the original sequence dataset is performed in the following way: after mixing the complete sequence dataset and the original sequence dataset, sort them according to the size of the observation value corresponding to each sequence, and remove the generated sequences that have duplicate observation values ​​with the original sequence dataset.

[0069] In another embodiment, step S400, the training process of the sequence observation generation model includes:

[0070] S401: Initialize the model's weights and biases by using a standard normal distribution (mean 0, variance 1) to initialize the weights and assign a bias value of 0.

[0071] S402: The generated sequence is iterated forward to output the observation value. Specifically, one iteration is: x′=a·x+b, where x is the input generated sequence, a is the weight matrix, b is the bias matrix, and x′ is the output observation value.

[0072] S403: Calculate the output error using the model loss function;

[0073] In this step, the loss function is expressed as:

[0074] L=Σ(y true +y predict ) 2 +λΣ(x input -x decode ) 2

[0075] Where, x input Represents the input sequence; x decode This represents the output sequence of the autoencoder; y true For the corresponding observations in the sequence; y predictThe model outputs predicted values; λ is a constant with a value in the range (0, 1).

[0076] S404: Backpropagation is performed based on the error to update the model's weights and biases until the calculated value of the loss function tends to stabilize.

[0077] In another embodiment, the original sequence dataset needs to be standardized before performing step S100.

[0078] In this embodiment, the raw lithium battery charge and discharge sequence dataset is standardized using standard scores, specifically as follows:

[0079]

[0080] Where, x origin Indicates an unnormalized sequence; x z-score This represents the standardized sequence; L represents the sequence length.

[0081] and

[0082]

[0083]

[0084] k represents the sequence number; i represents the sequence number of the k-th data point in the sequence; mean origin std represents the mean of the original sequence data. origin M represents the standard deviation of the original sequence data; M represents the number of sequences.

[0085] After standardization, the values ​​of all sequences at position i in the original sequence dataset will follow a Gaussian distribution with a mean of 0 and a standard deviation of 1.

[0086] It should be noted that since the sequence values ​​in the original sequence dataset and the observations obtained in subsequent steps are of different orders of magnitude, standardizing the original sequence dataset with standard scores can convert the sequence values ​​in the original sequence dataset and the observations generated by the model into standard scores of a unified metric, thereby reducing the error of the complete sequence dataset.

[0087] It should also be noted that, in this embodiment, in addition to using standard scores, the min-max normalization method can also be used to standardize the original sequence dataset. However, the processing effect of using standard scores is relatively better. Therefore, this embodiment prefers to use standard scores.

[0088] In another embodiment, after obtaining the generated sequence through step S300, the generated sequence also needs to be smoothed.

[0089] In this embodiment, the generated sequence can be smoothed using a Savitzky-Gore filter to filter out noise and maintain the shape and bandwidth of the sequence. Specifically, the smoothing of the generated sequence is performed as follows: a k-th order polynomial is fitted to the data points within a sliding window of length W in the generated sequence, where the fitted polynomial is... N is the number of terms in the polynomial, a k This represents the coefficient of the k-th term. After fitting, the smoothed generated sequence can be obtained.

[0090] It should be noted that, in addition to using the Savitsky-Gore filter, the moving average method can also be used to smooth the generated sequence. However, since the Savitsky-Gore filter has a slightly better processing effect, it is preferred to use the Savitsky-Gore filter to smooth the generated sequence.

[0091] The effectiveness of the above method will be explained below using lithium battery capacity sequence data as an example.

[0092] The original lithium battery capacity sequence data training set consisted of 94 sequences, and the test set consisted of 30 sequences. The above method is now used to expand the training set.

[0093] 1. One of the 94 sequences in the training set is:

[0094] x = {-1.4688e-04, -1.3615e04, -1.2695e-04, ..., -3.6823e-03, -3.6555e-03, -3.6279e-03}, after standardization, the standardized sequence is:

[0095] 2. Using the sequence y1 = {5.8204e-02, 5.5344e-02, 4.9425e-02, 4.0623e-02, 2.9220e-02, 1.5601e-02, 2.4838e-04, -1.6275e-02, ...} and the sequence y 10 Taking the sequence {-0.0826, 0.0440, 0.1535, 0.2448, 0.3175, 0.3713, 0.4065, 0.4240, ...} as an example, the similarity between any two sequences calculated using the dynamic time warping method is 12.8576. Therefore, the similarity between sequence y1 and sequence y2 is... 10 The distance matrix between them is D(0,9) = 8.1182.

[0096] 3. If the randomly initialized average sequence is sequence 5, the initialized weight array Weights obtained by the selective averaging method is as follows: The weight of the centroid sequence Weights(4) = 0.5; Select the 15 nearest sequences as the 15 sequences with the smallest distance values in the array D(4,j) (where 0 < j < 93), and the subscripts are marked as jarray. Then randomly select 4 sequences from these 15 sequences, and the subscripts are marked as jsub. The assigned weight is Weights(jsub) = 0.3 / 4. The subscripts of the other sequences except the centroid sequence and the 4 nearest sequences among the 94 sequences are marked as jother, and the assigned weight is Weights(jother) = 0.2 / (94 - 4 - 1).

[0097] 4. If the initial average sequence is sequence 5 and the sequence values are x5 = {-1.5653e-01, -1.5828e-01, -1.5893e-01, -1.5881e-01, -1.5832e-01, -1.5788e-01, -1.5797e-01, -1.5912e-01,...}, after the dynamic programming centroid averaging algorithm iterates, the initial average sequence (i.e., the generated sequence) is updated to

[0098] 5. The above generated sequence After smoothing, the obtained sequence is

[0099] 6. The sequence after smoothing Is used as the input of the sequence observation value generation model, and the predicted life value shown as Figure 4 Is 760 charge-discharge cycles. Then the above generated sequence and 760 discharge cycles constitute a complete sequence data set.

[0100] 7. Mix the generated data with the original data set, and the deduplication operation is as follows:

[0101] Sort all sequences according to the observations. If there are sequences with duplicate observations, keep only one of them. For example, if the sequence x1 = {-3.4244e-02, 3.1816e-02, 8.67680758e-02, 1.3001e-01, 1.6128e-01, 1.8063e-01, 1.8849e-01, 1.8570e-01, ...} has the corresponding observation value y1 = {597}, and the sequence x2 = {-3.3124e-02, 3.0124e-02, 8.3547e-02, 1.0246e-01, 1.5395e-01, 1.8372e-01, 1.7843e-01, 1.8076e-01, ...} has the corresponding observation value y2 = {597}, and y1 and y2 are equal, then we randomly retain one of the data as valid data.

[0102] The final augmented dataset is as follows:

[0103] [x1={5.4771e-5,4.2963e-5,3.3512e-5,2.6207e-5,2.0837e-5,1.7188e-5,…},y1={1852};

[0104] x2={2.3470e-5,1.4696e-5,6.5457e-6,-1.0040e-6,-7.9751e-6,-1.4389e-5,…}, y2={655};

[0105] x3={-7.4080e-7,-1.1730e-5,-2.1841e-5,-3.1108e-5,-3.9562e-5,-4.7236e-5,…}, y3={555};

[0106] x4={-5.4641e-5,-5.2110e-5,-4.9959e-5,-4.8171e-5,-4.6730e-5,-4.5620e-5,…}, y4={693};

[0107] x5={6.7265e-5,7.2880e-5,7.7978e-5,8.2579e-5,8.6702e-5,9.0369e-5,…},y5={969};

[0108] ...

[0109] x 162={-6.7448e-5,-7.6629e-5,-8.4840e-5,-9.2120e-5,-9.8510e-5,-1.0404e-4,…},y 162 ={815};

[0110] x 163 ={3.4136e-5,2.8767e-5,2.3899e-5,1.9510e-5,1.5583e-5,1.2098e-5},y 163 ={1059};

[0111] x 164 ={-3.7905e-5,-3.0467e-5,-2.3676e-5,-1.7511e-5,-1.1949e-5,-6.9684e-6},y 164 ={548}).

[0112] Under the same prediction network settings (Transformer in step S400), prediction models were trained using the original lithium battery capacity sequence and the expanded lithium battery capacity sequence, respectively. The comparison of the lithium battery cycle life prediction results obtained in the test set is shown in Table 1:

[0113] Table 1

[0114] Should it be expanded? no yes Root mean square error 132.470 88.826 Mean absolute percentage error 11.347 8.355

[0115] As shown in Table 1, after the training set was expanded, the root mean square error and mean absolute percentage error of the prediction results for lithium battery cycle life decreased significantly. This indicates that expanding the original lithium battery capacity sequence data using this method has a positive technical effect. In this example, expanding the training set improved the accuracy of lithium battery cycle life prediction, providing more accurate reference values ​​for lithium battery health management. Using the prediction results to plan the maintenance or replacement decisions for lithium battery equipment transforms passive maintenance into condition-based maintenance, effectively reducing the risks faced by lithium battery equipment during operation, further enhancing operational reliability, and saving maintenance effort and costs.

[0116] The foregoing general description of the invention and its specific embodiments should not be construed as a limitation on the technical solution of the invention. Those skilled in the art, based on the disclosure of this application, can add, reduce, or combine the disclosed technical features in the foregoing general description and / or specific embodiments (including examples) without departing from the constituent elements of the invention, to form other technical solutions within the scope of protection of this application.

Claims

1. A method for expanding a sequence dataset, comprising the following steps: S100: Obtain the distance matrix between pairs of sequences in the original sequence dataset; S200: Based on the original sequence dataset and distance matrix, obtain the initial average sequence and the initialized weight array; S300: Based on the initial average sequence and the initialized weight array, obtain the generated sequence; S400: Construct a sequence observation generation model, take the generated sequence as the input of the model, and output the observations corresponding to the generated sequence. The generated sequence and its corresponding observations form a complete sequence dataset. S5 00: Merge the complete sequence dataset with the original sequence dataset, and after deduplication, the original sequence dataset is expanded. in, Before performing step S100, the original sequence dataset needs to be standardized. Standard scores were used to standardize the raw lithium battery charge and discharge sequence dataset. The original lithium battery capacity sequence data was expanded according to the method described above; Expanding the training dataset improves the accuracy of lithium battery cycle life prediction, provides more precise reference values ​​for lithium battery health management, and allows for the use of prediction results to schedule maintenance or replacement decisions for lithium battery equipment, transforming passive maintenance into condition-based maintenance, effectively reducing the risks faced by lithium battery equipment during operation, further enhancing operational reliability, and saving maintenance effort and costs.

2. The method of claim 1, wherein, In step S400, the sequence observation generation model includes: An autoencoder is used to extract feature information from the generated sequence; The Transformer is used to output observations corresponding to the generated sequence based on the extracted feature information of the generated sequence.

3. The method of claim 2, wherein, The autoencoder includes an encoder section and a decoder section, wherein, The encoder section includes a first input layer, a first fully connected layer, and a second fully connected layer; The decoder consists of a second input layer, a third fully connected layer, and an output layer.

4. The method of claim 2, wherein, The Transformer includes: The system consists of a location encoding layer, a multi-head attention layer, a first residual connection and normalization layer, a feedforward linear layer, a second residual connection and normalization layer, and a fully connected layer.

5. The method of claim 2, wherein, In step S400, the training process of the sequence observation generation model includes: S401: Initialize the model's weights and biases; S402: Perform forward iterative computation on the generated sequence and output the observed values; S403: Calculate the output error using the model loss function; S404: Backpropagation is performed based on the error to update the model's weights and biases until the calculated value of the loss function tends to stabilize.

6. The method of claim 1, wherein, In step S500, the deduplication of the merged complete sequence dataset and the original sequence dataset is performed in the following way: after mixing the complete sequence dataset and the original sequence dataset, sort them according to the size of the observation value corresponding to each sequence, and remove the generated sequences that have duplicate observation values ​​with the original sequence dataset.

7. The method of claim 1, wherein, After obtaining the generated sequence through step S300, the generated sequence also needs to be smoothed.

8. The method of claim 7, wherein, Smoothing the generated sequence includes: performing k-order polynomial fitting on the data points within a sliding window of length W in the generated sequence.

Images