A time series data classification method and device based on content-aware embedding, equipment and medium

Abstract

The application discloses a time series data classification method and device based on content-aware embedding, equipment and medium, relates to the technical field of neural networks, and pre-processes time series data of a human body during movement; the pre-processed time series data is segmented to obtain time series blocks; a content-aware embedding sequence is calculated, a sequence block position embedding sequence corresponding to the time series block is determined, and an embedding sequence is generated; a pre-training model is used to extract the embedding sequence to obtain a feature vector; the feature vector and a prototype vector are normalized, the normalized feature is projected to a unit hypersphere, the similarity between the projected normalized feature and the prototype vector is calculated, and a time series data classification prediction result is calculated; the time series data is classified in a linear layer, the corresponding action of the human body during movement is identified, and the problem that the classification performance is poor due to the lack of content-aware capability of the embedding layer and the inability of the output layer to effectively model the category distribution is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neural networks, and in particular to a time series data classification method and device based on content-aware embedding, equipment and medium. BACKGROUND

[0002] TSC (Time Series Classification) is a basic and crucial task in time series analysis, and its goal is to build a model that can assign a predefined class label to the input time series data. The traditional strategy of adjusting at the top level has obvious limitations when applied to time series classification, resulting in suboptimal performance. The main problems are: content-independent embedding representation: although the existing embedding method incorporates time sequence information through position encoding, the generated representation is static and content-independent. This method cannot capture the intrinsic dynamic characteristics of the signal, such as periodicity or spectral energy distribution, and lacks the ability to perceive the content of the time series; the classifier modeling ability is insufficient, and a simple linear output layer essentially learns a set of separating hyperplanes, which lacks explicit modeling of class prototypes, making it difficult for the model to capture and utilize the differences in data distribution between different classes.

[0003] From the above, how to solve the problem of poor classification performance caused by the lack of content-aware ability of the embedding layer and the inability of the output layer to effectively model the class distribution, and improve the accuracy and performance of time series data classification based on content-aware embedding is a problem to be solved in the field. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a time series data classification method and device based on content-aware embedding, which can solve the problem of poor classification performance caused by the lack of content-aware ability of the embedding layer and the inability of the output layer to effectively model the class distribution, and improve the accuracy and performance of time series data classification based on content-aware embedding. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a time series data classification method based on content-aware embedding, comprising:

[0006] Collecting time series data of the human body during exercise, and preprocessing the time series data;

[0007] Segmenting the preprocessed time series data to obtain time series blocks;

[0008] calculating a content-aware embedding sequence of the time series block, determining a sequence block position embedding sequence corresponding to the time series block, and generating an embedding sequence based on the time series block, the content-aware embedding sequence, and the sequence block position embedding sequence;

[0009] extracting the embedding sequence by using a pre-trained model to obtain a feature vector containing deep semantic information;

[0010] normalizing the feature vector and a preset prototype vector to obtain a normalized feature, projecting the normalized feature to a unit hypersphere to calculate a similarity between the projected normalized feature and the prototype vector, and calculating a time series data classification prediction result based on the similarity;

[0011] linearly classifying the time series data by using the time series data classification prediction result to identify a corresponding action of a human body during movement.

[0012] Optionally, time series data of a human body during movement is collected, and the time series data is preprocessed;

[0013] segmenting the preprocessed time series data to obtain a time series block;

[0014] calculating a content-aware embedding sequence of the time series block, determining a sequence block position embedding sequence corresponding to the time series block, and generating an embedding sequence based on the time series block, the content-aware embedding sequence, and the sequence block position embedding sequence;

[0015] extracting the embedding sequence by using a pre-trained model to obtain a feature vector containing deep semantic information;

[0016] normalizing the feature vector and a preset prototype vector to obtain a normalized feature, projecting the normalized feature to a unit hypersphere to calculate a similarity between the projected normalized feature and the prototype vector, and calculating a time series data classification prediction result based on the similarity;

[0017] linearly classifying the time series data by using the time series data classification prediction result to identify a corresponding action of a human body during movement.

[0018] Optionally, the segmenting the preprocessed time series data includes:

[0019] determining a total length of the preprocessed time series data;

[0020] calculating a number of segmented sequence blocks based on the total length;

[0021] If the number of the segmented sequence blocks is an integer, the preprocessed time sequence data is segmented according to the number of the segmented sequence blocks along the time dimension.

[0022] If the number of the segmented sequence blocks is not an integer, the preprocessed time sequence data is padded with zero values at the end of the sequence, and the padded time sequence data is segmented.

[0023] Optionally, the segmentation of the preprocessed time sequence data comprises:

[0024] determining the total length of the preprocessed time sequence data;

[0025] calculating the number of the segmented sequence blocks based on the total length;

[0026] If the number of the segmented sequence blocks is an integer, the preprocessed time sequence data is segmented according to the number of the segmented sequence blocks along the time dimension.

[0027] If the number of the segmented sequence blocks is not an integer, the preprocessed time sequence data is padded with zero values at the end of the sequence, and the padded time sequence data is segmented.

[0028] Optionally, before the extracting of the embedding sequence by using the pre-training model, the method further comprises:

[0029] taking a time sequence data base model as a backbone network, and constructing a pre-training model based on a multi-head self-attention mechanism layer, residual connection and layer normalization, and a feedforward neural network.

[0030] Optionally, the calculating of the similarity between the projected normalized feature and the prototype vector, and the calculating of the time sequence data classification prediction result based on the similarity, comprises:

[0031] calculating the similarity between the projected normalized feature and the prototype vector by using a similarity calculation formula;

[0032] calculating the similarity by using a prediction result calculation formula to obtain the time sequence data classification prediction result;

[0033] The similarity calculation formula is:

[0034] ;

[0035] wherein, the similarity between the normalized feature and each prototype vector, a positive parameter of a learnable distribution concentration, the projected normalized feature, is the transpose matrix of the jth prototype vector;

[0036] The prediction result calculation formula is:

[0037]

[0038] wherein, is the time series data classification prediction result, and W is a linear layer parameter.

[0039] Optionally, the content-aware embedding-based time series data classification method further comprises:

[0040] constructing a cross-entropy classification loss based on the time series data classification prediction result;

[0041] constructing an orthogonal regularization loss based on the prototype vector;

[0042] generating a loss function using the cross-entropy classification loss and the orthogonal regularization loss;

[0043] updating the prototype vector using the loss function.

[0044] In a second aspect, the present application discloses a content-aware embedding-based time series data classification device, comprising:

[0045] a preprocessing module configured to collect time series data of a human body during exercise and preprocess the time series data;

[0046] a segmentation module configured to segment the preprocessed time series data to obtain time series blocks;

[0047] an embedding sequence generation module configured to calculate a content-aware embedding sequence of the time series blocks, determine a sequence block position embedding sequence corresponding to the time series blocks, and generate an embedding sequence based on the time series blocks, the content-aware embedding sequence, and the sequence block position embedding sequence;

[0048] a feature vector extraction module configured to extract the embedding sequence using a pre-trained model to obtain a feature vector containing deep semantic information;

[0049] a similarity calculation module configured to normalize the feature vector and a preset prototype vector to obtain normalized features, project the normalized features to a unit hypersphere, calculate the similarity between the projected normalized features and the prototype vector, and calculate a time series data classification prediction result based on the similarity;

[0050] ​A linear layer classification module is configured to perform linear layer classification on the time series data by using the time series data classification prediction result to identify the corresponding action of the human body in motion.

[0051] In a third aspect, the present application discloses an electronic device, comprising:

[0052] A memory is configured to save a computer program.

[0053] A processor is configured to execute the computer program to implement the aforementioned content-aware embedding-based time series data classification method.

[0054] In a fourth aspect, the present application discloses a computer storage medium configured to save a computer program; wherein the computer program is executed by a processor to implement the steps of the aforementioned content-aware embedding-based time series data classification method.

[0055] It can be seen that the application provides a time series data classification method based on content-aware embedding, which comprises collecting time series data of a human body during movement, preprocessing the time series data; segmenting the preprocessed time series data to obtain time series blocks; calculating content-aware embedding sequences of the time series blocks, determining sequence block position embedding sequences corresponding to the time series blocks, generating embedding sequences based on the time series blocks, the content-aware embedding sequences and the sequence block position embedding sequences; extracting the embedding sequences by using a pre-trained model to obtain feature vectors containing deep semantic information; normalizing the feature vectors and preset prototype vectors to obtain normalized features, projecting the normalized features to a unit hypersphere to calculate the similarity between the projected normalized features and the prototype vectors, calculating a time series data classification prediction result based on the similarity; and performing linear layer classification on the time series data by using the time series data classification prediction result to identify corresponding actions of the human body during movement. The application preprocesses the time series data of the human body during movement to eliminate dimensional differences between different samples, ensures the integrity of the sequence, segments the preprocessed time series data to obtain time series blocks, reduces the computational complexity, calculates the content-aware embedding sequences of the time series blocks, determines the sequence block position embedding sequences corresponding to the time series blocks, generates the embedding sequences based on the time series blocks, the content-aware embedding sequences and the sequence block position embedding sequences, can solve the problem that embedding representation is irrelevant to content, extracts the embedding sequences by using a pre-trained model to obtain feature vectors containing deep semantic information, normalizes the feature vectors and preset prototype vectors to obtain normalized features, projects the normalized features to a unit hypersphere to calculate the similarity between the projected normalized features and the prototype vectors, solves the problem that a traditional linear classifier cannot effectively model the class distribution, calculates a time series data classification prediction result based on the similarity, performs linear layer classification on the time series data by using the time series data classification prediction result, solves the problem of poor classification performance caused by the lack of content-aware capability of the embedding layer and the inability of the output layer to effectively model the class distribution, identifies corresponding actions of the human body during movement, and improves the accuracy and performance of the time series data classification based on content-aware embedding. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0057] Figure 1A content-aware embedding based time series data classification method flowchart disclosed by the application;

[0058] Figure 2 A CAP-Tune adaptation framework chart disclosed by the application;

[0059] Figure 3 A prototype similarity matrix chart obtained by training with cross-regularization disclosed by the application;

[0060] Figure 4 A result chart after training of the orthogonal regularization loss term disclosed by the application;

[0061] Figure 5 A content-aware embedding based time series data classification device structure schematic diagram disclosed by the application;

[0062] Figure 6 An electronic device structure chart provided by the application. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0064] TSC is a basic and crucial task in time series analysis, which aims to build a model that can assign a predefined class label to the input time series data. The traditional strategy of adjusting at the top level has obvious limitations when applied to time series classification, resulting in suboptimal performance. The main problems are as follows: content-agnostic embedding representation: although the existing embedding method incorporates time sequence information through position encoding, the generated representation is static and content-agnostic. This method cannot capture the intrinsic dynamic characteristics of the signal, such as periodicity or spectral energy distribution, and lacks the ability to perceive the content of the time series; the classifier modeling ability is insufficient, and a simple linear output layer essentially learns a set of separating hyperplanes, which lacks explicit modeling of class prototypes, making it difficult for the model to capture and utilize the differences in data distribution between different classes. As can be seen from the above, how to solve the problem of poor classification performance caused by the lack of content-aware ability of the embedding layer and the inability of the output layer to effectively model the class distribution, and improve the accuracy and performance of content-aware embedding based time series data classification are problems to be solved in the field.

[0065] Reference Figure 1As shown, the embodiment of the application discloses a time series data classification method based on content-aware embedding, which can specifically include:

[0066] Step S11: Collecting time series data of human body in the movement process, and pre-processing the time series data.

[0067] In this embodiment, the time series data of human body in the movement process is collected, the Z-score standardization method is used for standardization processing of the time series data, and the forward filling method and the backward filling method are used for missing value processing of the time series data.

[0068] The formula corresponding to the standardization processing is:

[0069]

[0070] Among them, is the time series data after standardization processing, x is any value in the time series data, u is the mean value of all time points in the time series data, is the standard deviation of the time series data.

[0071] In order to eliminate the dimensional difference between different samples and process abnormal data, and provide high-quality input for subsequent model processing, the application proposes a pre-processing process for time series data, specifically: in order to enable the model to be trained stably and avoid the dimension with a larger numerical range in the data dominating the learning of the model parameters, the application adopts the Z-score standardization method to independently process each channel of each time series data input; for the missing values in the data, forward filling, backward filling or linear interpolation method can be used for completion according to the data characteristics, so as to ensure the integrity of the sequence.

[0072] Step S12: Segmenting the pre-processed time series data to obtain time series blocks.

[0073] In this embodiment, the total length of the pre-processed time series data is determined; the number of segmented sequence blocks is calculated based on the total length; if the number of segmented sequence blocks is an integer, the pre-processed time series data is segmented according to the number of segmented sequence blocks along the time dimension; if the number of segmented sequence blocks is not an integer, zero value filling is performed at the end of the pre-processed time series data, and the time series data after filling is segmented to obtain time series blocks.

[0074] ​In this embodiment, the long sequence is converted into a series of patches (local segments) that are easier for the model to process, especially TSFM (Time Series Foundation Models) based on Transformer (a neural network architecture for natural language processing), while reducing computational complexity. For a preprocessed multivariate time series data with D channels and a length of L... Each channel is independently divided into N fixed-size, non-overlapping time series blocks along the time dimension. Each time series block has a length of P. The number of time series blocks is... To ensure the total length of the sequence can be completely divided, if L is not an integer multiple of P, zeros are used to pad the end of the sequence to make its length an integer multiple of the next P. After this step, the preprocessed time series data is converted into time series blocks. .

[0075] Step S13: Calculate the content-aware embedding sequence of the time series block, determine the sequence block position embedding sequence corresponding to the time series block, and generate an embedding sequence based on the time series block, the content-aware embedding sequence, and the sequence block position embedding sequence.

[0076] In this embodiment, a fast Fourier transform is performed on the time series block to obtain the amplitude spectrum; the ratio between the peak value and the root mean square value in the amplitude spectrum is calculated using the diversity intensity index; the time series block is scaled using the ratio to generate a content-aware embedding sequence; the sequence block position embedding sequence corresponding to the time series block is determined; and an embedding sequence is generated based on the time series block, the content-aware embedding sequence, and the sequence block position embedding sequence.

[0077] In this embodiment, a content-aware embedding sequence with content-aware capabilities is generated for each time series block through spectral analysis and diversity intensity index calculation. Specifically, spectral analysis is used for each input time series block. Perform an FFT (Fast Fourier Transform) to obtain the amplitude spectrum. The diversity intensity index is calculated as the ratio between the peak value and the root mean square (RMS) of the amplitude spectrum for each time series block. The formula is as follows:

[0078] ;

[0079] in, It is a ratio. To prevent division by zero by small constants. Lower represents a complex, diverse frequency component, large amount of information, and relatively high represents a single frequency pattern, small amount of information.

[0080] The calculated The time series block is scaled, so as to punish the block with less information and retain the block with rich information. The embedding process is as follows:

[0081] ;

[0082] wherein, and are learnable parameters.

[0083] Finally, the time series block, the content-aware embedding sequence, and the sequence block position embedding sequence are added to form the final input embedding sequence of the TSFM backbone network :

[0084] ;

[0085] wherein, is the time series block, is the content-aware embedding sequence, is the sequence block position embedding sequence.

[0086] Step S14: extracting the embedding sequence by using the pre-trained model to obtain a feature vector containing deep semantic information.

[0087] In the embodiment, before the embedding sequence is extracted by using the pre-trained model, the time series data base model is taken as a backbone network, and the pre-trained model is constructed based on a multi-head self-attention mechanism layer, residual connection and layer normalization, and a feedforward neural network.

[0088] The application utilizes the powerful ability of the pre-trained base model to extract high-level and general feature vectors. The embedding sequence containing content-aware information is input to a pre-trained TSFM as an encoder backbone network. The application adopts GPT-2 (Generative Pre-trained Transformer 2) architecture as the encoder, and the core thereof is composed of multiple stacked Transformer decoder modules. The encoding process can be represented as:

[0089] ;

[0090] wherein, is the embedding sequence fused with content-aware information, and is taken as input In turn, each layer of the GPT-2 encoder is passed through, in each layer, The following core calculation steps are taken to obtain the output sequence .

[0091] The specific process and principles of using a pre-trained model for extraction in this application are as follows:

[0092] MHSA (Multi-Head Self-Attention, multi-head self-attention mechanism): This mechanism is used to capture the complex dependencies between various time series blocks within the input sequence. The embedded input sequence is first linearly transformed into Q (Query), K (Key), and V (Value) matrices. By calculating the dot product similarity of Q and K, the model can dynamically assign attention weights to each block in the sequence, and then perform weighted summation on V using these weights to obtain a representation rich in contextual information. This process can be represented as:

[0093] ;

[0094] The multi-head mechanism performs this process multiple times in parallel and concatenates the results to learn information from different representation subspaces.

[0095] Residual connection and layer normalization: After the self-attention calculation, a residual connection is applied to add the input of this layer to the output of the self-attention layer, followed by layer normalization. This helps to prevent gradient vanishing and accelerate model convergence.

[0096] ;

[0097] FFN (Feed-Forward Network, feed-forward neural network): This network is usually composed of two linear layers and an activation function such as GELU (Gaussian Error Linear Unit, a nonlinear activation function based on Gaussian distribution), which is applied independently to each position in the sequence. It introduces nonlinearity into the model, further enhancing the representation capability.

[0098] Second residual connection and layer normalization: The residual connection and layer normalization are again applied to the output of the feed-forward network to obtain the final output of this Transformer module:

[0099] ;

[0100] In this fine-tuning framework, the parameters of the TSFM backbone network (i.e., the GPT-2 encoder) are usually frozen, i.e., kept unchanged during the training process for downstream tasks. The final output Z of the entire encoding process is a sequence of features containing deep semantic information, and each vector in the sequence is a rich representation of the corresponding input block, which will serve as the basis representation for subsequent classification tasks.

[0101] Step S15: normalizing the feature vector and the preset prototype vector to obtain a normalized feature, projecting the normalized feature to a unit hypersphere to calculate the similarity between the projected normalized feature and the prototype vector, and calculating a time series data classification prediction result based on the similarity.

[0102] In this embodiment, after projecting the normalized feature to the unit hypersphere, the similarity between the projected normalized feature and the prototype vector is calculated using a similarity calculation formula; the similarity is calculated using a prediction result calculation formula to obtain a time series data classification prediction result.

[0103] The similarity calculation formula is:

[0104] ;

[0105] Wherein, is the similarity between the normalized feature and each prototype vector, is a positive parameter of the learnable distribution concentration, is the projected normalized feature, is the transpose matrix of the jth prototype vector.

[0106] The prediction result calculation formula is:

[0107] ;

[0108] Wherein, is the time series data classification prediction result, and W is a linear layer parameter.

[0109] To solve the problem that the traditional linear classifier cannot effectively model the class distribution, the ProtoVMF (von Mises-Fisher) head is designed, and the core idea is that 1. Feature and prototype spherical projection: first, aggregate the feature vector Z output by the TSFM to obtain a feature vector z representing the entire time series, for example, take the feature corresponding to the first [CLS] mark or average pool all block features, and then project the feature vector z and a set of learnable prototype vectors L2 norm normalization is performed, projecting onto the unit hyper-sphere. This makes the model pay more attention to the direction of the features rather than their magnitude.

[0110] ;

[0111] The vMF distribution is used to measure the similarity between a feature and a prototype. For a normalized feature and the j-th prototype vector , the similarity score is computed.

[0112] Step S16: Linear layer classification is performed on the time series data using the time series data classification prediction results to identify the corresponding action of the human body when moving.

[0113] In this embodiment, the time series data classification method based on content-aware embedding further comprises: constructing a cross-entropy classification loss based on the time series data classification prediction results; constructing an orthogonal regularization loss based on the prototype vector; generating a loss function using the cross-entropy classification loss and the orthogonal regularization loss; and updating the prototype vector using the loss function.

[0114] The prototype vector can also be continuously updated using a loss function. The obtained dimension similarity score vector (the number of prototypes) is input into a linear layer to obtain the final classification logits. The loss function consists of two parts: a standard cross-entropy classification loss and an optional orthogonal regularization loss :

[0115] ;

[0116] ;

[0117] wherein is the label value, is the time series data classification prediction result, and n is the number of training set samples. is designed to encourage mutual orthogonality between different prototypes, wherein is a matrix composed of all normalized prototypes , I is the identity matrix, denotes the F (Frobenius) norm of a matrix. This loss term penalizes non-zero similarity between prototypes to learn more diverse and discriminative feature representations.

[0118] The application is applied to a CAP-Tune adaptation framework, which improves the classification performance by reconstructing the embedding layer and the output layer of the TSFM. Figure 2 As shown in the CAP-Tune adaptation framework, it mainly includes time series data preprocessing, block conversion, content-aware embedding, feature extraction, and ProtoVMF head classification.

[0119] 1. Time series data preprocessing: preprocessing the time series data of the human body during the movement process;

[0120] 2. Block conversion: the original time series data is divided into fixed size, non-overlapping time series blocks;

[0121] 3. Content-aware embedding: embedding sequence is generated based on time series blocks, content-aware embedding sequence and sequence block position embedding sequence;

[0122] 4. TSFM feature extraction: the embedding sequence is input into the pre-trained model to extract the feature vector containing deep semantic information.

[0123] 5. ProtoVMF head classification: using the prototype ProtoVMF head instead of the traditional linear classifier. The normalized features are mapped to the unit hypersphere, and the final classification is completed by comparing with the preset prototype vector, identifying the corresponding action of the human body during the movement.

[0124] The application can solve the problem of content-independent of the traditional embedding method, so that the model can learn more discriminative features, and the ProtoVMF head proposed in the application explicitly models the feature distribution of each class on the hypersphere, which can better capture the distribution difference between classes compared with the traditional linear classifier, and constructs a feature space with stronger structure and more explicit probability meaning. The CAP-Tune framework of the application is model-independent, and can be easily applied to a variety of different TSFM backbone networks.

[0125] To verify the effectiveness of the technical solution, GPT4TS is used as the backbone TSFM feature extraction network of the application, and extensive experiments are conducted on LSST, Ering, MotorImagery, and PhonemeSpectra datasets, and the experimental results are shown in Table 1.

[0126] Table 1 Experimental results

[0127]

[0128] To specifically illustrate the technical effect of the prototype vMF head in the present application, an ablation experiment was conducted to verify the effect of the orthogonal regularization loss term using the disclosed LSST dataset. The prototype similarity matrix obtained by training with orthogonal regularization is shown in Figure 3 The result obtained by training with the orthogonal regularization loss term of the present application is shown in Figure 4 It can be seen that, in addition to the diagonal line, there are a large number of areas with high non-diagonal element values in the matrix, which indicates that there is a high similarity between different prototypes, i.e., the prototypes learned by the model are redundant and fail to fully learn diversified features. Figure 4 The matrix in Figure 4 is very close to a unit matrix. Among them, the diagonal element values are all 1 (in red), indicating that the similarity of each prototype to itself is 1; and the values of all non-diagonal elements are close to 0 (in dark blue), indicating that any two different prototypes are mutually orthogonal.

[0129] Experiments prove that the framework proposed in the present application can continuously and significantly improve the performance of various TSFM on a large number of time series classification datasets.

[0130] In addition, the present application can be applied not only in human motion processes, but also in the medical and health care field, the industrial manufacturing field, etc. In the human motion process, by analyzing the time series data in the motion process, the specific action (such as running, jumping, and swinging) performed can be recognized, and technical evaluation and tactical analysis can be performed. In the medical and health care field, by analyzing physiological signal time series data such as ECG (Electro Cardio Gram, electrocardiogram) or EEG (Electroencephalogram, electroencephalogram), early warning of heart disease or automatic detection of epileptic seizures can be achieved. In the industrial manufacturing field, by monitoring sensor time series data such as vibration, temperature, or pressure generated during equipment operation, equipment fault types can be diagnosed in a timely manner, predictive maintenance can be achieved, and stable operation of the production line can be ensured.

[0131] In this embodiment, time series data of a human body during exercise is collected, and the time series data is preprocessed. The preprocessed time series data is segmented to obtain time series blocks. The content-aware embedding sequence of the time series block is calculated, the sequence block position embedding sequence corresponding to the time series block is determined, and the embedding sequence is generated based on the time series block, the content-aware embedding sequence and the sequence block position embedding sequence. The embedding sequence is extracted by using a pre-trained model to obtain a feature vector containing deep semantic information. The feature vector and a preset prototype vector are normalized to obtain a normalized feature, and the normalized feature is projected to a unit hypersphere to calculate the similarity between the projected normalized feature and the prototype vector. The time series data classification prediction result is calculated based on the similarity. The time series data is classified in a linear layer based on the time series data classification prediction result to identify the corresponding action of the human body during exercise. The time series data of the human body during exercise is preprocessed to eliminate the dimensional difference between different samples and ensure the integrity of the sequence. The preprocessed time series data is segmented to obtain time series blocks, which reduces the computational complexity. The content-aware embedding sequence of the time series block is calculated, the sequence block position embedding sequence corresponding to the time series block is determined, and the embedding sequence is generated based on the time series block, the content-aware embedding sequence and the sequence block position embedding sequence, which can solve the problem of embedding representation and content independence. The embedding sequence is extracted by using a pre-trained model to obtain a feature vector containing deep semantic information. The feature vector and a preset prototype vector are normalized to obtain a normalized feature, and the normalized feature is projected to a unit hypersphere to calculate the similarity between the projected normalized feature and the prototype vector. The traditional linear classifier cannot effectively model the class distribution, and the time series data classification prediction result is calculated based on the similarity. The time series data is classified in a linear layer based on the time series data classification prediction result to solve the problem of poor classification performance caused by the lack of content-aware capability of the embedding layer and the inability of the output layer to effectively model the class distribution. The corresponding action of the human body during exercise is identified to improve the accuracy and performance of the content-aware embedding-based time series data classification.

[0132] Referring to Figure 5 The embodiment of the application discloses a time series data classification device based on content-aware embedding, which can specifically include:

[0133] The preprocessing module 11 is used for collecting time series data of a human body during exercise, and preprocessing the time series data.

[0134] The segmentation module 12 is used for segmenting the preprocessed time series data to obtain time series blocks.

[0135] The embedding sequence generation module 13 is configured to calculate a content-aware embedding sequence of the time sequence block, determine a sequence block position embedding sequence corresponding to the time sequence block, and generate an embedding sequence based on the time sequence block, the content-aware embedding sequence, and the sequence block position embedding sequence.

[0136] The feature vector extraction module 14 is configured to extract the embedding sequence by using a pre-trained model to obtain a feature vector containing deep semantic information.

[0137] The similarity calculation module 15 is configured to perform normalization processing on the feature vector and a preset prototype vector to obtain a normalized feature, project the normalized feature to a unit hypersphere, calculate a similarity between the projected normalized feature and the prototype vector, and calculate a time sequence data classification prediction result based on the similarity.

[0138] The linear layer classification module 16 is configured to perform linear layer classification on the time sequence data by using the time sequence data classification prediction result to identify a corresponding action of a human body during movement.

[0139] In this embodiment, time series data of a human body during exercise is collected, and the time series data is preprocessed; the preprocessed time series data is segmented to obtain time series blocks; content-aware embedding sequences of the time series blocks are calculated, sequence block position embedding sequences corresponding to the time series blocks are determined, and embedding sequences are generated based on the time series blocks, the content-aware embedding sequences, and the sequence block position embedding sequences; a pre-trained model is used to extract the embedding sequences to obtain feature vectors containing deep semantic information; the feature vectors and preset prototype vectors are normalized to obtain normalized features, and the normalized features are projected to a unit hypersphere to calculate a similarity between the projected normalized features and the prototype vectors, a time series data classification prediction result is calculated based on the similarity, and linear layer classification is performed on the time series data based on the time series data classification prediction result to identify corresponding actions of the human body during exercise. The time series data of the human body during exercise is preprocessed in this application to eliminate dimensional differences between different samples and ensure the integrity of the sequence. The preprocessed time series data is segmented to obtain time series blocks, the calculation complexity is reduced, content-aware embedding sequences of the time series blocks are calculated, sequence block position embedding sequences corresponding to the time series blocks are determined, embedding sequences are generated based on the time series blocks, the content-aware embedding sequences, and the sequence block position embedding sequences, which can solve the problem of embedding representation being irrelevant to content. A pre-trained model is used to extract the embedding sequences to obtain feature vectors containing deep semantic information, the feature vectors and preset prototype vectors are normalized to obtain normalized features, the normalized features are projected to a unit hypersphere to calculate a similarity between the projected normalized features and the prototype vectors, which solves the problem that a traditional linear classifier cannot effectively model the class distribution. A time series data classification prediction result is calculated based on the similarity, linear layer classification is performed on the time series data based on the time series data classification prediction result, which solves the problem of poor classification performance caused by the lack of content-aware capability of the embedding layer and the inability of the output layer to effectively model the class distribution, identifies corresponding actions of the human body during exercise, and improves the accuracy and performance of content-aware embedding-based time series data classification.

[0140] In some embodiments, the preprocessing module 11 can specifically include:

[0141] A normalization and missing value processing module is configured to perform normalization processing on the time series data by using a Z-score normalization method, and perform missing value processing on the time series data by using a forward filling method and a backward filling method.

[0142] The formula corresponding to the normalization processing is:

[0143] ;

[0144] wherein, is the standardized time series data, x is any value in the time series data, u is the mean value of all time points in the time series data, is the standard deviation of the time series data.

[0145] In some embodiments, the segmentation module 12 can specifically include:

[0146] a total length determination module, configured to determine the total length of the preprocessed time series data;

[0147] a number of segmented sequence blocks calculation module, configured to calculate the number of segmented sequence blocks based on the total length;

[0148] a segmentation module, configured to, if the number of segmented sequence blocks is an integer, segment the preprocessed time series data along the time dimension according to the number of segmented sequence blocks;

[0149] a zero value padding module, configured to, if the number of segmented sequence blocks is not an integer, pad the preprocessed time series data with zero values at the end of the sequence, and segment the padded time series data.

[0150] In some embodiments, the embedded sequence generation module 13 can specifically include:

[0151] a fast Fourier transform module, configured to perform fast Fourier transform on the time series block to obtain an amplitude spectrum;

[0152] a ratio calculation module, configured to calculate the ratio between the peak value in the amplitude spectrum and the root mean square using the diversity intensity index;

[0153] a scaling module, configured to scale the time series block using the ratio to generate a content-aware embedded sequence.

[0154] In some embodiments, the feature vector extraction module 14 can specifically include:

[0155] a pre-trained model construction module, configured to use a time series data base model as a backbone network, and construct a pre-trained model based on a multi-head self-attention mechanism layer, residual connection and layer normalization, and a feedforward neural network.

[0156] In some embodiments, the similarity calculation module 15 can specifically include:

[0157] a specific similarity calculation module, configured to calculate the similarity between the normalized feature after projection and the prototype vector using a similarity calculation formula;

[0158] The time series data classification prediction result calculation module is configured to calculate the similarity by using a prediction result calculation formula to obtain a time series data classification prediction result.

[0159] The similarity calculation formula is as follows:

[0160]

[0161] wherein, is the similarity between the normalized feature and each prototype vector, is a positive parameter of a learnable distribution concentration, is the normalized feature after projection, is the transpose matrix of the jth prototype vector;

[0162] The prediction result calculation formula is as follows:

[0163]

[0164] wherein, is the time series data classification prediction result, and W is a linear layer parameter.

[0165] In some specific embodiments, the time series data classification device based on content-aware embedding can further specifically include:

[0166] The cross-entropy classification loss construction module is configured to construct a cross-entropy classification loss based on the time series data classification prediction result.

[0167] The orthogonal regularization loss construction module is configured to construct an orthogonal regularization loss based on the prototype vector.

[0168] The loss function generation module is configured to generate a loss function by using the cross-entropy classification loss and the orthogonal regularization loss.

[0169] The prototype vector update module is configured to update the prototype vector by using the loss function.

[0170] Figure 6 FIG. 1 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is configured to store a computer program, the computer program is loaded and executed by the processor 21 to implement the related steps in the time series data classification method based on content-aware embedding performed by the electronic device disclosed in any of the preceding embodiments.

[0171] ​​In this embodiment, the power supply 23 is configured to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 is configured to create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which will not be specifically limited herein; the input / output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which will not be specifically limited herein.

[0172] In addition, the memory 22 as a carrier for storing resources can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon include an operating system 221, a computer program 222 and data 223, etc., and the storage mode can be temporary storage or permanent storage.

[0173] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222, so as to realize the operation and processing of the processor 21 on the data 223 in the memory 22, and the operating system 221 can be Windows, Unix, Linux, etc. In addition to the computer program capable of completing the content-aware embedding-based time series data classification method executed by the electronic device 20 disclosed in any one of the preceding embodiments, the computer program 222 can further include a computer program capable of completing other specific work. In addition to the data transmitted by the external device and received by the content-aware embedding-based time series data classification device, the data 223 can also include data collected by the input / output interface 25 itself and the like.

[0174] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be directly implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0175] Further, the embodiments of the present application also disclose a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program is loaded and executed by a processor to implement the steps of the content-aware embedding-based time series data classification method disclosed in any one of the preceding embodiments.

[0176] Finally, it needs to be pointed out that, in this document, relational terms such as first and second and the like can only be used to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between or among the entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0177] The above describes in detail the content-aware embedding-based time series data classification method, device, equipment and storage medium provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges can be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A time-series data classification method based on content-aware embedding, characterized in that, include: Collect time-series data of the human body during movement, and preprocess the time-series data; The time series data is the LSST dataset; The preprocessed time series data is segmented to obtain time series blocks; Calculate the content-aware embedding sequence of the time series block, determine the sequence block position embedding sequence corresponding to the time series block, and generate an embedding sequence based on the time series block, the content-aware embedding sequence, and the sequence block position embedding sequence; The embedded sequence is extracted using a pre-trained model to obtain a feature vector containing deep semantic information; The feature vector and the preset prototype vector are normalized to obtain normalized features. The normalized features are then projected onto a unit hypersphere to calculate the similarity between the projected normalized features and the prototype vector. Based on the similarity, the classification prediction result of the time series data is calculated. The time series data is classified using the classification and prediction results to identify the corresponding actions of the human body during movement. The step of calculating the content-aware embedding sequence of the time series block includes: performing a fast Fourier transform on the time series block to obtain an amplitude spectrum; calculating the ratio between the peak value and the root mean square value in the amplitude spectrum using a diversity intensity index; and scaling the time series block using the ratio to generate a content-aware embedding sequence. The formula for calculating the ratio is: ； in, The ratio, To prevent small constants from being divided by zero, Amplitude spectrum; Before extracting the embedded sequence using the pre-trained model, the method further includes: using the time series data basic model as the backbone network, and constructing a pre-trained model based on a multi-head self-attention mechanism layer, residual connections and layer normalization, and a feedforward neural network.

2. The time series data classification method based on content-aware embedding according to claim 1, characterized in that, The preprocessing of the time series data includes: The time series data were standardized using the Z-score standardization method, and missing values ​​were handled using forward imputation and backward imputation methods. The formula corresponding to the standardization process is: ； in, For standardized time series data, x Any value in the time series data, u This represents the mean of all time points in the time series data. represents the standard deviation of the time series data.

3. The time series data classification method based on content-aware embedding according to claim 1, characterized in that, The segmentation of the preprocessed time series data includes: Determine the total length of the preprocessed time series data; The number of segmented sequence blocks is calculated based on the total length; If the number of segmented sequence blocks is an integer, then the preprocessed time series data is segmented according to the number of segmented sequence blocks and along the time dimension; If the number of segmented sequence blocks is not an integer, then zero values ​​are padded at the end of the preprocessed time series data, and the padded time series data is segmented.

4. The time series data classification method based on content-aware embedding according to claim 1, characterized in that, The calculation of the similarity between the projected normalized features and the prototype vector, and the calculation of time series data classification prediction results based on the similarity, includes: The similarity between the projected normalized features and the prototype vector is calculated using a similarity calculation formula. The similarity is calculated using the prediction result calculation formula to obtain the time series data classification prediction result; The similarity calculation formula is as follows: ； in, The similarity between the normalized features and each prototype vector. is a positive parameter for the learnable distribution concentration. These are the normalized features after projection. For the first j Transpose of a prototype vector; The formula for calculating the prediction result is as follows: ； in, Classification and prediction results for time series data. W These are the parameters for the linear layer.

5. The time series data classification method based on content-aware embedding according to any one of claims 1 to 4, characterized in that, Also includes: Construct a cross-entropy classification loss based on the classification prediction results of time series data; Construct orthogonal regularization loss based on prototype vectors; A loss function is generated using the cross-entropy classification loss and the orthogonal regularization loss. The prototype vector is updated using the loss function.

6. A time-series data classification device based on content-aware embedding, characterized in that, include: The preprocessing module is used to collect time-series data of the human body during movement and to preprocess the time-series data. The time series data is the LSST dataset; The segmentation module is used to segment the preprocessed time series data to obtain time series blocks; An embedding sequence generation module is used to calculate the content-aware embedding sequence of the time series block, determine the sequence block position embedding sequence corresponding to the time series block, and generate an embedding sequence based on the time series block, the content-aware embedding sequence, and the sequence block position embedding sequence. The feature vector extraction module is used to extract the embedded sequence using a pre-trained model to obtain feature vectors containing deep semantic information; The similarity calculation module is used to normalize the feature vector and the preset prototype vector to obtain normalized features, project the normalized features onto a unit hypersphere to calculate the similarity between the projected normalized features and the prototype vector, and calculate the classification prediction result of time series data based on the similarity. The linear layer classification module is used to perform linear layer classification on the time series data using the classification prediction results of the time series data, so as to identify the corresponding actions of the human body during movement; The step of calculating the content-aware embedding sequence of the time series block includes: performing a fast Fourier transform on the time series block to obtain an amplitude spectrum; calculating the ratio between the peak value and the root mean square value in the amplitude spectrum using a diversity intensity index; and scaling the time series block using the ratio to generate a content-aware embedding sequence. The formula for calculating the ratio is: ； in, The ratio, To prevent small constants from being divided by zero, Amplitude spectrum; Before extracting the embedded sequence using the pre-trained model, the method further includes: using the time series data basic model as the backbone network, and constructing a pre-trained model based on a multi-head self-attention mechanism layer, residual connections and layer normalization, and a feedforward neural network.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the content-aware embedding-based time-series data classification method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the content-aware embedding-based time-series data classification method as described in any one of claims 1 to 5.

Images