Semantic learning-based time series signal processing method, device and equipment

Through the time series signal processing method based on semantic learning, the time series semantic generation module, feature extraction module and comparison learning module are used to solve the problem of semantic information extraction in timing sequence analysis and improve the classification recognition performance of timing sequences.

CN118690256BActive Publication Date: 2024-11-22NAT UNIV OF DEFENSE TECH
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411171903.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-22
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

Existing timing sequence analysis methods are difficult to effectively learn and extract semantic information in timing sequences, especially in cases of interference such as length transformation, noise and data loss.

Method used

The timing signal processing method based on semantic learning is adopted, and the timing sequence is sampled and divided and weighted through the timing semantic generation module. The feature extraction module uses the self-attention mechanism to extract local and global correlation features. The comparison learning module calculates the comparison learning loss, so that the model can be made Learn common features between positive samples and separability features between negative samples.

Benefits of technology

It realizes effective extraction and learning of time-series semantic information, improves the model's understanding of time-series semantic information, improves the efficiency and accuracy of classification recognition, and obtains good recognition performance under various interference conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118690256B_ABST
    Figure CN118690256B_ABST
Patent Text Reader

Abstract

The present application relates to a method, device and equipment for processing time series signals based on semantic learning. The method includes: using a time series semantic generation module to sample, divide and weight the time series sequence; the feature extraction module uses a self-attention mechanism to capture the local correlation and global correlation between the weighted divided samples; using a contrastive learning module to calculate the contrastive learning loss of the encoded features, so that the model learns the common features between positive samples and the separable features between negative samples to obtain time series semantic features; the time series semantic features are input into the classification head of the preset time series target classification task to obtain the target recognition result. This method can realize the semantic feature learning of the time series sequence, improve the model's understanding of the time series semantic information contained in the time series sequence, and finally enable the model to obtain time series features with good representation, effectively improve the efficiency and accuracy of classification and recognition, and achieve good recognition performance under various interference conditions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of time series signal processing and computer intelligence technology, and in particular to a time series signal processing method, device and equipment based on semantic learning. Background Art

[0002] Time series analysis is a key technology in the development of science and technology in the world today. This technology has been widely used in many fields, such as financial data analysis and prediction, climate and meteorological prediction, and anomaly monitoring. Time series are usually obtained by a variety of sensors to monitor the surrounding environment or human body conditions. The sequence often contains rich time series semantic information, such as humidity changes, brain waves, etc. However, the semantic information of the time series is hidden in the time series change process of the sequence, and it is difficult to extract the semantic information through explicit modeling. Therefore, how to effectively learn and extract the hidden time series semantic information in the time series is a key and difficult problem in this field.

[0003] Existing time series analysis methods usually use two methods to process time series. The first is the black box method. This method builds an end-to-end network model and learns and extracts information from the time series through a large number of time series samples. The second is the sequence modeling method. This method first models the time-frequency consistency of the time series, and then learns the time domain information and frequency domain information separately through comparative learning. However, the above methods are difficult to represent and learn the semantic information in the time series, so the performance of the above methods is poor under interference such as length transformation, noise and data missing.

[0004] In recent years, natural language processing models have made significant progress, and large models such as chatGPT have been born that have the ability to understand natural language. The key to the success of these models lies in effectively capturing and learning the semantic information contained in natural language through semantic learning methods, thereby improving the model's ability to understand language, and ultimately enabling the model to maintain good performance in the case of long sentences, spelling errors, and missing sentences. The key technology of semantic learning lies in how to divide the input natural language based on phrases and how to give semantic labels to the divided phrases. Summary of the invention

[0005] Based on this, it is necessary to provide a time series signal processing method, device and equipment based on semantic learning to address the above technical problems.

[0006] A time series signal processing method based on semantic learning, the method comprising:

[0007] Acquire a time series sequence; a time series sequence is data obtained by multiple sensors monitoring and sensing the surrounding environment or human body conditions.

[0008] A pre-built semantic learning-based temporal sequence feature representation model is trained using a temporal sequence to obtain a trained temporal sequence feature representation model; the semantic learning-based temporal sequence feature representation model includes: a temporal semantic generation module, a feature extraction module and a contrastive learning module; the temporal semantic generation module is used to sample and divide the temporal sequence, calculate the corresponding semantic weight of each divided sample, weight the divided samples with the sampled semantic weights, and obtain the weighted divided samples; the feature extraction module is used to perform projection encoding on the weighted divided samples, and extract local correlation and global correlation features from the encoding results using a self-attention mechanism to obtain encoding features; the contrastive learning module is used to calculate the contrastive learning loss, and the contrastive learning loss is used to enable the model to learn the common features between positive samples and the separable features between negative samples to obtain temporal semantic features.

[0009] The time series sequence to be processed in the preset time series target classification task is processed using the trained time series feature representation model, and the obtained time series semantic features are input into the classification head of the preset time series target classification task to obtain the target recognition result.

[0010] In one embodiment, a pre-built semantic learning-based time series feature representation model is trained using a time series sequence to obtain a trained time series feature representation model, including:

[0011] The time series is input into the time series semantic generation module to obtain weighted segmentation samples.

[0012] The weighted partitioned samples are input into the feature extraction module to obtain the encoding features.

[0013] The encoded features are input into the contrastive learning module, the loss function is learned through the contrastive learning strategy, and the contrastive loss is used to update the parameters of the time series feature representation model.

[0014] The trained time series feature representation model is connected to the classification head of the preset time series target classification task and the parameters are adjusted in the data set of the preset time series target classification task. The recognition model composed of the adjusted time series feature representation model and the classification head is subjected to classification test using the data set of the preset time series target classification task to obtain the trained time series feature representation model. In the parameter adjustment process, the parameters of the trained time series feature representation model are fixed, and only the parameters of the classification head are updated.

[0015] In one of the embodiments, the temporal semantic generation module includes a sampling partitioning module and a weight generation network; the weight generation network includes a linearization layer, a ReLU activation function and a Sigmoid activation function.

[0016] Input the time series into the time series semantic generation module to obtain weighted segmentation samples, including:

[0017] The time series sequence is input into the sampling and partitioning module of the time series semantic generation module, and the time series sequence is sampled based on the Markov Monte Carlo sampling method to obtain multiple partition samples.

[0018] All the divided samples are input into the weight generation network of the temporal semantic generation module. After being processed by the linearization layer, and then by the ReLU activation function and the Sigmoid activation function, the semantic weight corresponding to each divided sample is obtained.

[0019] Multiply the partitioned sample by the corresponding semantic weight to obtain the weighted partitioned sample.

[0020] In one of the embodiments, the feature extraction module includes: a projection network and a plurality of feature extraction sub-modules.

[0021] The weighted partitioned samples are input into the feature extraction module to obtain the encoding features, including:

[0022] The weighted segmented samples are input into the projection network of the feature extraction module to obtain the projection code.

[0023] The projection code is input into the first feature extraction submodule of the feature extraction module to obtain the first intermediate feature.

[0024] The first intermediate feature is input into the second feature extraction submodule of the feature extraction module, and the output of the second feature extraction submodule is input into the third feature extraction submodule, and so on, until the last feature extraction submodule, to obtain the encoding feature.

[0025] In one embodiment, the feature extraction submodule includes layer normalization, a multi-head self-attention layer, and a feed-forward layer; in the first feature extraction submodule:

[0026] ;

[0027] ;

[0028] ;

[0029] ;

[0030] ;

[0031] ;

[0032] ;

[0033] ;

[0034] in, represents the projection encoding, represents the normalized output of the layer, Representation layer normalization operation, is the query matrix, represents the key matrix, is the value matrix, , , They are , , The projection matrix, Indicates The output of a self-attention head; , , Respectively The projection matrix of the query matrix, key matrix, and value matrix of the self-attention head, represents the number of attention heads, Represents a splicing operation, represents the recovery matrix, represents the input of the feedforward layer, and MLP represents the feedforward layer; represents the first intermediate feature output by the first feature extraction submodule, Represents the output of the multi-head attention layer; represents the output of the feedforward layer, Represents the self-attention mechanism.

[0035] In one embodiment, the feed-forward layer includes a multi-layer perceptron, wherein the multi-layer perceptron includes two fully connected layers.

[0036] In one embodiment, the projection network is composed of fully connected layers.

[0037] In one embodiment, the encoding features are input into a contrastive learning module, a loss function is learned through a contrastive learning strategy, and a parameter of a temporal sequence feature representation model is updated using contrastive loss, including:

[0038] The encoded features are processed using a contrastive learning strategy and then projected to obtain a projection vector.

[0039] The projection vector is divided into positive and negative samples using a positive and negative sample division method based on a time series sequence to obtain a positive sample set and a negative sample set; the positive and negative sample division method based on a time series sequence refers to treating the divided samples from the same sequence as positive sample pairs, and treating the divided samples from different sequences as negative sample pairs.

[0040] According to the projection vector, the positive sample set and the projection vector set of the encoding features obtained in the same batch, the contrastive learning loss function is determined as:

[0041] ;

[0042] in, represents the contrastive learning loss, Indicates Anchor points, represents the set of encoded features excluding anchor points, Indicates The encoding features of anchor points, represents a positive sample, represents the temperature coefficient, Represents the set of projection vectors of the coded features obtained in the same batch, represents the positive sample set, Represents the total number of Monte Carlo sampling.

[0043] The loss contrast loss function is used to update all learnable parameters in the time series feature representation model.

[0044] A time series signal processing device based on semantic learning, the device comprising:

[0045] The training sample acquisition module is used to acquire time series; the time series is data obtained by multiple sensors monitoring the surrounding environment or human body conditions.

[0046] The time series feature representation model training module is used to train the pre-built time series feature representation model based on semantic learning with the time series to obtain a trained time series feature representation model; the time series feature representation model based on semantic learning includes: a time series semantic generation module, a feature extraction module and a contrastive learning module; the time series semantic generation module is used to sample and divide the time series, calculate the corresponding semantic weight of each divided sample, and weight the divided samples with the sampled semantic weights to obtain weighted divided samples; the feature extraction module is used to perform projection encoding on the weighted divided samples, and extract local correlation and global correlation features from the encoding results using the self-attention mechanism to obtain encoding features; the contrastive learning module is used to calculate the contrastive learning loss, and uses the contrastive learning loss to enable the model to learn the common features between positive samples and the separable features between negative samples to obtain time series semantic features.

[0047] The signal processing module based on the time series feature representation model is used to process the time series sequence to be processed in the preset time series target classification task using the trained time series feature representation model, and input the obtained time series semantic features into the classification head of the preset time series target classification task to obtain the target recognition result.

[0048] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0049] The above-mentioned method, device, machine equipment and storage medium for processing time series signals based on semantic learning, the method includes: using a time series semantic generation module to sample, divide and weight the time series sequence, using a feature extraction module based on a self-attention mechanism to capture the local correlation and global correlation between the weighted divided samples; using a contrastive learning module to calculate the contrastive learning loss of the encoding features, so that the time series feature representation model learns the common features between positive samples and the separable features between negative samples to obtain time series semantic features, and inputs the time series semantic features into the classification head of the preset time series target classification task to obtain the target recognition result. The time series feature representation model based on semantic learning proposed by this method can realize the semantically basic feature learning of the time series sequence, improve the model's understanding of the time series semantic information contained in the time series sequence, and finally enable the model to obtain time series features with good representation, improve the efficiency and accuracy of classification and recognition, and achieve good recognition performance under various interference conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of a method for processing a time series signal based on semantic learning in one embodiment;

[0051] Figure 2 The present invention is a block diagram and a flow chart of a temporal sequence feature representation model based on semantic learning in one embodiment, wherein (a) is a block diagram of a temporal sequence feature representation model based on semantic learning, (b) is an overall flow chart of a temporal sequence feature representation model based on semantic learning, and (c) is an overall flow chart of a contrastive learning module;

[0052] Figure 3 is a structural block diagram of a time series signal processing device based on semantic learning in one embodiment;

[0053] Figure 4 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] In one embodiment, Figure 1As shown, a time series signal processing method based on semantic learning is provided, and the method should include the following steps:

[0056] Step 100: Acquire a time series; the time series is data obtained by multiple sensors monitoring and sensing the surrounding environment or human body conditions.

[0057] Specifically, taking the sleep EEG sequence as an example, the time series samples were collected from 83 healthy subjects during 153 full nights of sleep. Each EEG sequence was sampled at a frequency of 100 Hz, and each EEG data was associated with a specific sleep stage, including wakefulness, non-rapid eye movement (including three sub-states), and rapid eye movement.

[0058] Step 102: using a time series sequence to train a pre-built time series feature representation model based on semantic learning, to obtain a trained time series feature representation model; the time series feature representation model based on semantic learning includes: a time series semantic generation module, a feature extraction module and a contrastive learning module; the time series semantic generation module is used to sample and divide the time series sequence, calculate the corresponding semantic weight according to each divided sample, and weight the divided samples by the sampled semantic weight to obtain the weighted divided samples; the feature extraction module is used to perform projection encoding on the weighted divided samples, and extract local correlation and global correlation features from the encoding results by using a self-attention mechanism to obtain encoding features; the contrastive learning module is used to calculate the contrastive learning loss, and the contrastive learning loss is used to enable the model to learn the common features between positive samples and the separability features between negative samples to obtain time series semantic features;

[0059] Specifically, the temporal semantic generation module is used to divide and sample the input temporal sequence based on Monte Carlo sampling, and to assign weights to the divided samples based on the training loss function. The assigned weights will eventually converge with the training iterations.

[0060] The feature extraction module is built based on the long-range processing capability of the self-attention mechanism, and performs feature extraction on the weighted partitioned samples to obtain the encoded features.

[0061] The contrastive learning module is built based on the contrastive learning strategy. Specifically, the module regards the encoding features from the same sequence as the same batch of positive samples, and the encoding features obtained from different sequences as negative samples. The module uses the contrastive learning loss function to enable the model to learn the common features between positive samples and the separable features between negative samples.

[0062] The time series feature representation model based on semantic learning can effectively extract and learn the semantic information in the time series and provide good separable feature representation for downstream tasks. The overall block diagram of the time series feature representation model based on semantic learning is shown in the figure. Figure 2 As shown, (a) is a composition block diagram of the time series feature representation model based on semantic learning, (b) is the overall process of the time series feature representation model based on semantic learning, and (c) is the overall process of the contrastive learning module.

[0063] Step 104: The time series sequence to be processed in the preset time series target classification task is processed using the trained time series feature representation model, and the obtained time series semantic features are input into the classification head of the preset time series target classification task to obtain the target recognition result.

[0064] In the above-mentioned method for processing time series signals based on semantic learning, the method includes: using a time series semantic generation module to sample, divide and weight the time series sequence, using a feature extraction module based on a self-attention mechanism to capture the local correlation and global correlation between the weighted divided samples; using a contrastive learning module to calculate the contrastive learning loss of the encoded features, so that the time series feature representation model learns the common features between positive samples and the separable features between negative samples to obtain time series semantic features, and inputs the time series semantic features into the classification head of the preset time series target classification task to obtain the target recognition result. The time series feature representation model based on semantic learning proposed by this method can realize the semantically basic feature learning of the time series sequence, improve the model's understanding of the time series semantic information contained in the time series sequence, and finally enable the model to obtain time series features with good representation, improve the efficiency and accuracy of classification and recognition, and achieve good recognition performance under various interference conditions.

[0065] In one embodiment, step 102 includes the following specific steps:

[0066] Step 200: Input the time series sequence into the time series semantic generation module to obtain weighted division samples.

[0067] Step 202: Input the weighted divided samples into a feature extraction module to obtain coding features.

[0068] Step 204: Input the encoded features into the contrastive learning module, learn the loss function through the contrastive learning strategy, and use the contrastive loss to update the parameters of the time series feature representation model.

[0069] Step 206: Connect the trained time series feature representation model to the classification head of the preset time series target classification task and adjust the parameters in the data set of the preset time series target classification task; perform classification test on the recognition model composed of the time series feature representation model and the classification head after the parameter adjustment using the data set of the preset time series target classification task to obtain the trained time series feature representation model; wherein, during the parameter adjustment process, the parameters of the trained time series feature representation model are fixed, and only the parameters of the classification head are updated.

[0070] Specifically, the model training process of this method can be divided into three steps. The input time series batch is processed by the time series semantic generation module to obtain the corresponding weighted partition samples. ; Secondly, the obtained weighted partitioned samples are subjected to feature extraction by the feature extraction module to obtain the coding features ; Finally, in the contrastive learning module, the contrastive loss is calculated based on the obtained encoding features , so as to update the parameters of the time series feature representation model.

[0071] The test process of this embodiment is specifically divided into two steps. First, the trained semantic learning-based temporal sequence feature representation model is connected to the recognition and classification head of the preset temporal target classification task and the parameters are adjusted in the data set of the preset temporal target classification task. The parameter adjustment process fixes the parameters of the semantic learning-based temporal sequence feature representation model and only updates the parameters of the recognition and classification head. After the parameter adjustment is completed, the parameters of the semantic learning-based temporal sequence feature representation model and the recognition and classification head are fixed, and a classification test is performed in the preset temporal target classification task data set.

[0072] In one embodiment, the temporal semantic generation module includes a sampling partitioning module and a weight generation network; the weight generation network includes a linearization layer, a ReLU activation function and a Sigmoid activation function; step 200 includes: inputting the temporal sequence into the sampling partitioning module of the temporal semantic generation module, sampling the temporal sequence based on the Markov Monte Carlo sampling method to obtain multiple partitioned samples; inputting all the partitioned samples into the weight generation network of the temporal semantic generation module, after being processed by the linearization layer, and then after being processed by the ReLU activation function and the Sigmoid activation function, the semantic weight corresponding to each partitioned sample is obtained; multiplying the partitioned sample with the corresponding semantic weight to obtain a weighted partitioned sample.

[0073] Specifically, the temporal semantic generation module is responsible for sampling, dividing and weighting the input temporal sequence, and finally obtaining the weighted divided samples. The overall process of this module is as follows Figure 2 As shown in (b) in the figure, the main process of the temporal semantic generation module is introduced below.

[0074] Assume that in The input length in the training iteration is The time series is , the sequence is divided into samples after sampling , the divided samples are obtained from the entire time series. Represents the partition sample The sampling process is determined by the parameter vector Determine. The parameter vector , Indicates the starting time point of the divided sample on the entire sample segment, Indicates the length of the sequence of divided samples. In actual implementation, the length of the sequence of divided samples is Use a fixed length, starting time point By uniform distribution Sampling is obtained. In this way, the prior probability of dividing the sample is It can be expressed as:

[0075] (1)

[0076] The integral term in formula (1) is often not calculable. Therefore, this application is implemented by Monte Carlo sampling, which can be specifically expressed as:

[0077] (2)

[0078] in, Represents the total number of Monte Carlo sampling. Then, the temporal semantic generation module will assign weights to the partitioned samples. The weight calculation process is:

[0079] (3)

[0080] in, represents the weight calculation network, where represents the learning parameters, Indicates the corresponding weights calculated by the weight network based on the input partitioned samples. The weight calculation network parameters will be based on the model's loss function to update.

[0081] In the During the training iteration, the learning parameters of the weight network will be The loss function calculated by the partitioned samples obtained in the first time is updated, thereby affecting the Therefore, this embodiment constructs a Markov process, which can be specifically expressed as:

[0082] (4)

[0083] in, Indicates The loss function calculated is: Indicates that based on parameters The entire time series is divided and sampled. Indicates The loss function calculated by The sampling division results are determined by It represents the weight .

[0084] Formula (4) shows that the temporal semantic generation module contains two processes: the first process is to sample and divide the entire input sequence to obtain different sampling and division samples; the second process is that the weight network will calculate the corresponding semantic weight based on each division sample. Finally, the weighted division sample will be the final output of the temporal semantic generation module, which can be specifically expressed as:

[0085] (5)

[0086] in, Represents the weighted partition samples.

[0087] In one embodiment, the feature extraction module includes: a projection network, and multiple feature extraction sub-modules; step 202 includes: inputting the weighted divided samples into the projection network of the feature extraction module to obtain projection coding; inputting the projection coding into the first feature extraction sub-module of the feature extraction module to obtain a first intermediate feature; inputting the first intermediate feature into the second feature extraction sub-module of the feature extraction module, inputting the output of the second feature extraction sub-module into the third feature extraction sub-module, and so on, until the last feature extraction sub-module to obtain the coding feature.

[0088] In one embodiment, the feature extraction submodule includes layer normalization, a multi-head self-attention layer, and a feed-forward layer; in the first feature extraction submodule:

[0089] (6)

[0090] (7)

[0091] (8)

[0092] (9)

[0093] (10)

[0094] (11)

[0095] (12)

[0096] (13)

[0097] in, represents the projection encoding, represents the normalized output of the layer, Representation layer normalization operation, is the query matrix, represents the key matrix, is the value matrix, , , They are , , The projection matrix, Indicates The output of the self-attention head, , , Respectively The projection matrix of the query matrix, key matrix, and value matrix of the self-attention head, , represents the number of attention heads, Represents a splicing operation, Represents the recovery matrix, which is used to ensure that the input and output matrices are consistent in dimension. represents the input of the feedforward layer, and MLP represents the feedforward layer; represents the feature extraction submodule, represents the first intermediate feature output by the first feature extraction submodule, Represents the output of the multi-head attention layer; represents the output of the feedforward layer, Represents the self-attention mechanism.

[0098] Specifically, the self-attention mechanism has excellent representation capabilities for long-range dependencies, and the self-attention mechanism can capture local and global correlations between partitioned samples. In order to effectively mine the intrinsic correlation information between weighted samples, a feature extraction module is constructed based on the self-attention mechanism. The feature extraction module mainly includes two processes: sample encoding process and feature extraction process. The sample encoding process is to project and position-encode the input weighted partitioned samples, and align the input encoded data dimensions. Specifically, the weighted partitioned samples output by the temporal semantic generation module will first be projected, which can be specifically expressed as:

[0099] (14)

[0100] in, represents a projection network consisting of fully connected layers, Then it represents the output projection code. Further, the output projection code will go through the feature extraction process to obtain the final feature encoding result. Specifically, the feature extraction process consists of multiple feature extraction submodules, each of which contains layer normalization, multi-head self-attention mechanism layer and feedforward layer. The main components of the feature extraction submodule will be introduced in detail below.

[0101] The layer normalization operation can effectively prevent the gradient vanishing and explosion problems caused by the network with a deep number of layers. This application uses the method shown in formula (6) to perform layer normalization on the input sequence.

[0102] The output of the layer normalization will be input into the multi-head self-attention layer. This layer first Perform three linear projections respectively and get Dimensional query matrix , Dimensional key matrix and Dimensional value matrix , as shown in formulas (7) to (9).

[0103] in, Then we get The three matrices are scaled by a self-attention mechanism based on cosine similarity measurement. The attention weights between each weighted partition sample are calculated with the softmax function and the value matrix is ​​re-weighted. The output from the attention layer is: .

[0104] In order to enable the model to learn more projection modes to better adapt to the characteristic modes of different modal data, the multi-head self-attention layer queries the matrix , key matrix Sum Matrix Project them to multiple feature subspaces respectively, and concatenate the attention results calculated in each subspace to obtain the final multi-head attention layer output. The multi-head attention layer output is shown in formula (10) to formula (11). Finally, the output of the multi-head self-attention layer will be input into the feedforward layer. The feedforward layer includes a multi-layer perceptron, which can extract and filter features through high-order mapping and low-dimensional recovery. The specific process can be expressed as formula (12). The output of the overall feature extraction submodule is shown in formula (13).

[0105] The feature extraction module includes: a projection network and multiple feature extraction submodules; the final output of the feature extraction module can be expressed as:

[0106] (15)

[0107] in, Represents the encoded features of the final output.

[0108] In one embodiment, the feed-forward layer includes a multi-layer perceptron, wherein the multi-layer perceptron includes two fully connected layers.

[0109] In one embodiment, the projection network is composed of fully connected layers.

[0110] In one embodiment, step 204 includes: processing the coding feature using a contrastive learning strategy and then projecting it to obtain a projection vector; dividing the projection vector into positive and negative samples using a positive and negative sample division method based on a time series sequence to obtain a positive sample set and a negative sample set; the positive and negative sample division method based on the time series sequence refers to treating the divided samples from the same sequence as positive sample pairs, and treating the divided samples from different sequences as negative sample pairs; according to the projection vector, the positive sample set, and the projection vector set of the coding feature obtained in the same batch, determining the contrastive learning loss function as:

[0111] (16)

[0112] in, represents the contrastive learning loss, Indicates anchor points (pre-labeled data), represents the set of encoded features excluding anchor points, Indicates The encoding features of anchor points, represents a positive sample, represents the temperature coefficient, Represents the set of projection vectors of the coded features obtained in the same batch, represents the positive sample set, Represents the total number of Monte Carlo sampling;

[0113] The loss contrast loss function is used to update all learnable parameters in the time series feature representation model.

[0114] Specifically, the contrastive learning module will build a contrastive learning paradigm based on weighted partitioning samples based on contrastive learning strategies. By dividing the encoded features obtained by feature extraction into positive and negative samples, this module will calculate the contrastive learning loss of the partitioned samples, so that the time series feature representation model based on semantic learning can learn the common features between positive samples and the difference features between negative samples. Specifically, in the In the iterations, for a batch size of After the time series samples are processed by the time series semantic generation module and the feature extraction module, the obtained encoding feature batch can be expressed as:

[0115] (17)

[0116] in, Indicates the batch number. Indicates the sequence number of Monte Carlo sampling.

[0117] According to the general contrastive learning strategy, the obtained batch of encoded features will be further projected, and the process can be expressed as:

[0118] (18)

[0119] in, represents the projection vector obtained by the final projection, represents the projection network, and the network parameters are To simplify the description, let the set Indicates the encoding features obtained in the same batch.

[0120] This application further constructs a positive and negative sample division method based on time series, aiming to enable the model to mine common features within the same sequence and learn separable features between different sequences. Specifically, this application regards the divided samples from the same sequence as positive sample pairs, and the divided samples from different sequences as negative sample pairs. The mathematical description of this process is:

[0121] (19)

[0122] in, represents the positive sample set, represents the set of positive sample numbers, It can map the input encoding features to their corresponding input time series. Therefore, the contrastive learning loss is shown in formula (16), and the obtained contrastive loss function will update all learnable parameters in the time series feature representation model based on semantic learning.

[0123] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0124] In a verification example, recognition is performed under different data interference conditions, and the recognition results are shown in Tables 1 to 3. It can be seen from the data in Tables 1 to 3 that the use of this method to characterize the time series effectively improves the accuracy of downstream recognition tasks. Among them, for the target recognition results under length transformation, the accuracy of the present invention is improved by about 15%-16% compared with the most advanced method; under the conditions of noise interference and data loss, the present invention can achieve stable recognition performance.

[0125] Table 1 Results of time series target recognition under length transformation

[0126]

[0127] Note: Acc, Pre, Re and F1 in the table represent accuracy, precision, recall and F1 value respectively.

[0128] Table 2 Results of time series target recognition under noise interference

[0129]

[0130] Table 3 Results of time series target recognition under missing data

[0131]

[0132] In one embodiment, Figure 3 As shown, a time series signal processing device based on semantic learning is provided, comprising: a training sample acquisition module, a time series feature representation model training module and a signal processing module based on the time series feature representation model, wherein:

[0133] The training sample acquisition module is used to acquire time series; the time series is data obtained by multiple sensors monitoring the surrounding environment or human body conditions.

[0134] The time series feature representation model training module is used to train the pre-built time series feature representation model based on semantic learning with the time series to obtain a trained time series feature representation model; the time series feature representation model based on semantic learning includes: a time series semantic generation module, a feature extraction module and a contrastive learning module; the time series semantic generation module is used to sample and divide the time series, calculate the corresponding semantic weight of each divided sample, and weight the divided samples with the sampled semantic weights to obtain weighted divided samples; the feature extraction module is used to perform projection encoding on the weighted divided samples, and extract local correlation and global correlation features from the encoding results using the self-attention mechanism to obtain encoding features; the contrastive learning module is used to calculate the contrastive learning loss, and uses the contrastive learning loss to enable the model to learn the common features between positive samples and the separable features between negative samples to obtain time series semantic features.

[0135] The signal processing module based on the time series feature representation model is used to process the time series sequence to be processed in the preset time series target classification task using the trained time series feature representation model, and input the obtained time series semantic features into the classification head of the preset time series target classification task to obtain the target recognition result.

[0136] In one of the embodiments, the time series feature representation model training module is also used to input the time series sequence into the time series semantic generation module to obtain weighted divided samples; input the weighted divided samples into the feature extraction module to obtain encoding features; input the encoding features into the contrastive learning module, learn the loss function through the contrastive learning strategy, and use the contrastive loss to update the parameters of the time series feature representation model; connect the trained time series feature representation model with the classification head of the preset time series target classification task and adjust the parameters in the data set of the preset time series target classification task, and use the data set of the preset time series target classification task to perform classification test on the recognition model composed of the adjusted time series feature representation model and the classification head to obtain the trained time series feature representation model; wherein, during the parameter adjustment process, the parameters of the trained time series feature representation model are fixed, and only the parameters of the classification head are updated.

[0137] In one embodiment, the temporal semantic generation module includes a sampling partitioning module and a weight generation network; the weight generation network includes a linearization layer, a ReLU activation function and a Sigmoid activation function; the temporal sequence feature representation model training module is also used to input the temporal sequence into the sampling partitioning module of the temporal semantic generation module, and obtain multiple partition samples by sampling the temporal sequence based on the Markov Monte Carlo sampling method; all the partition samples are input into the weight generation network of the temporal semantic generation module, and after being processed by the linearization layer, and then after being passed through the ReLU activation function and the Sigmoid activation function, the semantic weight corresponding to each partition sample is obtained; the partition sample is multiplied by the corresponding semantic weight to obtain a weighted partition sample.

[0138] In one embodiment, the feature extraction module includes: a projection network, multiple feature extraction sub-modules; a time series feature characterization model training module, which is also used to input the weighted divided samples into the projection network of the feature extraction module to obtain projection coding; input the projection coding into the first feature extraction sub-module of the feature extraction module to obtain a first intermediate feature; input the first intermediate feature into the second feature extraction sub-module of the feature extraction module, input the output of the second feature extraction sub-module into the third feature extraction sub-module, and so on, until the last feature extraction sub-module to obtain the coding feature.

[0139] In one embodiment, the feature extraction submodule in the time series feature representation model training module includes layer normalization, a multi-head self-attention layer, and a feedforward layer; in the first feature extraction submodule, the specific calculation principle is shown in formulas (6) to (13).

[0140] In one of the embodiments, the feedforward layer in the time series feature representation model training module includes a multilayer perceptron, wherein the multilayer perceptron includes two fully connected layers.

[0141] In one of the embodiments, the projection network in the time series feature representation model training module is composed of fully connected layers.

[0142] In one of the embodiments, the time series feature representation model training module is further used to process the coding feature using a contrastive learning strategy and then project it to obtain a projection vector; divide the projection vector into positive and negative samples using a positive and negative sample division method based on the time series to obtain a positive sample set and a negative sample set; the positive and negative sample division method based on the time series refers to treating the divided samples from the same sequence as positive sample pairs, and treating the divided samples from different sequences as negative sample pairs; determine the contrastive learning loss function shown in formula (15) based on the projection vector, the positive sample set and the projection vector set of the coding features obtained in the same batch; and use the loss contrast loss function to update all learnable parameters in the time series feature representation model.

[0143] For the specific limitations of the timing signal processing device based on semantic learning, please refer to the limitations of the timing signal processing method based on semantic learning above, which will not be repeated here. Each module in the above-mentioned timing signal processing device based on semantic learning can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0144] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a time series signal processing method based on semantic learning is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0145] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0146] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in any of the above method embodiments when executing the computer program.

[0147] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0148] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A time series signal processing method based on semantic learning, characterized in that: The method comprises: Acquire a time series; the time series is data obtained by multiple sensors monitoring and sensing the surrounding environment or human body conditions; The time series sequence is used to train the pre-constructed time series feature representation model based on semantic learning to obtain a trained time series feature representation model; the time series feature representation model based on semantic learning includes: a time series semantic generation module, a feature extraction module and a contrastive learning module; the time series semantic generation module is used to sample and divide the time series sequence, calculate the corresponding semantic weight according to each divided sample, sample the semantic weight to weight the divided sample, and obtain the weighted divided sample; the feature extraction module is used to perform projection encoding on the weighted divided sample, and use the self-attention mechanism to extract local correlation and global correlation features from the encoding result to obtain the encoding feature; the contrastive learning module is used to regard the encoding features from the same sequence as the same batch of positive samples, regard the encoding features obtained from different sequences as negative samples, calculate the contrastive learning loss of positive samples and negative samples, and use the contrastive learning loss to enable the model to learn the common features between positive samples and the separable features between negative samples to obtain the time series semantic features; The time series sequence to be processed in the preset time series target classification task is processed by using the trained time series feature representation model, and the obtained time series semantic features are input into the classification head of the preset time series target classification task to obtain the target recognition result; Among them, the temporal semantic generation module includes a sampling partitioning module and a weight generation network; the weight generation network includes a linearization layer, a ReLU activation function and a Sigmoid activation function; the temporal semantic generation module is also used to input the temporal sequence into the sampling partitioning module, and obtain multiple partition samples by sampling the temporal sequence based on the Markov Monte Carlo sampling method; all the partition samples are input into the weight generation network, and after being processed by the linearization layer, and then after being passed through the ReLU activation function and the Sigmoid activation function, the semantic weight corresponding to each partition sample is obtained; the partition sample is multiplied by the corresponding semantic weight to obtain the weighted partition sample.

2. The method according to claim 1, characterized in that The time series sequence is used to train a pre-built time series feature representation model based on semantic learning to obtain a trained time series feature representation model, including: Inputting the time series sequence into the time series semantic generation module to obtain weighted segmentation samples; Inputting the weighted divided samples into the feature extraction module to obtain coding features; Inputting the encoded features into the contrastive learning module, learning the loss function through the contrastive learning strategy, and using the contrastive loss to update the parameters of the time series feature representation model; The trained time series feature representation model is connected to the classification head of the preset time series target classification task and the parameters are adjusted in the data set of the preset time series target classification task. The recognition model composed of the adjusted time series feature representation model and the classification head is subjected to classification test using the data set of the preset time series target classification task to obtain the trained time series feature representation model. In the parameter adjustment process, the parameters of the trained time series feature representation model are fixed, and only the parameters of the classification head are updated.

3. The method according to claim 2, characterized in that The feature extraction module includes: a projection network and a plurality of feature extraction submodules; The weighted divided samples are input into the feature extraction module to obtain coding features, including: Inputting the weighted divided samples into the projection network of the feature extraction module to obtain projection codes; Inputting the projection code into a first feature extraction submodule of the feature extraction module to obtain a first intermediate feature; The first intermediate feature is input into the second feature extraction submodule of the feature extraction module, and the output of the second feature extraction submodule is input into the third feature extraction submodule, and so on, until the last feature extraction submodule, to obtain the encoding feature.

4. The method according to claim 3, characterized in that The feature extraction submodule includes layer normalization, multi-head self-attention layer and feed-forward layer; In the first feature extraction submodule: in, represents the projection encoding, represents the normalized output of the layer, Representation layer normalization operation, is the query matrix, represents the key matrix, is the value matrix, , , They are , , The projection matrix, represents the output of the i-th self-attention head, , , are the projection matrices of the query matrix, key matrix, and value matrix of the i-th self-attention head, respectively. represents the number of attention heads, Represents a splicing operation, represents the recovery matrix, represents the input of the feedforward layer, and MLP represents the feedforward layer; represents the first intermediate feature output by the first feature extraction submodule, Represents the output of the multi-head attention layer; represents the output of the feedforward layer, Represents the self-attention mechanism.

5. The method according to claim 4, characterized in that The feed-forward layer includes a multi-layer perceptron, wherein the multi-layer perceptron includes two fully connected layers.

6. The method according to claim 3, characterized in that The projection network consists of fully connected layers.

7. The method according to claim 1, characterized in that The encoding feature is input into the contrastive learning module, a loss function is learned through a contrastive learning strategy, and a parameter of the time series feature representation model is updated using contrastive loss, including: Processing the encoding feature using a contrastive learning strategy and then projecting it to obtain a projection vector; The projection vector is divided into positive and negative samples by using a positive and negative sample division method based on a time series sequence to obtain a positive sample set and a negative sample set; the positive and negative sample division method based on a time series sequence refers to treating the divided samples from the same sequence as positive sample pairs, and treating the divided samples from different sequences as negative sample pairs; According to the projection vector, the positive sample set and the projection vector set of the coding features obtained in the same batch, the contrastive learning loss function is determined as: in, represents the contrastive learning loss, Indicates Anchor points, represents the set of encoded features excluding anchor points, represents the encoding feature of the a-th anchor point, represents a positive sample, represents the temperature coefficient, Represents the set of projection vectors of the coded features obtained in the same batch, represents the positive sample set, Represents the total number of Monte Carlo sampling; A loss contrast loss function is used to update all learnable parameters in the temporal sequence feature representation model.

8. A time series signal processing device based on semantic learning, characterized in that: The device comprises: A training sample acquisition module is used to acquire a time series; the time series is data obtained by multiple sensors monitoring the surrounding environment or human body conditions; A time series feature representation model training module is used to use the time series sequence to train a pre-built time series feature representation model based on semantic learning to obtain a trained time series feature representation model; the time series feature representation model based on semantic learning includes: a time series semantic generation module, a feature extraction module and a contrastive learning module; the time series semantic generation module is used to sample and divide the time series sequence, calculate the corresponding semantic weight according to each divided sample, sample the semantic weight to weight the divided sample, and obtain the weighted divided sample; the feature extraction module is used to perform projection encoding on the weighted divided sample, and use the self-attention mechanism to extract local correlation and global correlation features from the encoding result to obtain encoding features; the contrastive learning module is used to regard the encoding features from the same sequence as the same batch of positive samples, regard the encoding features obtained from different sequences as negative samples, calculate the contrastive learning loss of positive samples and negative samples, and use the contrastive learning loss to enable the model to learn the common features between positive samples and the separability features between negative samples to obtain time series semantic features; A signal processing module based on a temporal sequence feature representation model is used to process the to-be-processed temporal sequence in a preset temporal target classification task using a trained temporal sequence feature representation model, and input the obtained temporal semantic features into a classification head of the preset temporal target classification task to obtain a target recognition result; Among them, the temporal semantic generation module includes a sampling partitioning module and a weight generation network; the weight generation network includes a linearization layer, a ReLU activation function and a Sigmoid activation function; the temporal semantic generation module is also used to input the temporal sequence into the sampling partitioning module, and obtain multiple partition samples by sampling the temporal sequence based on the Markov Monte Carlo sampling method; all the partition samples are input into the weight generation network, and after being processed by the linearization layer, and then after being passed through the ReLU activation function and the Sigmoid activation function, the semantic weight corresponding to each partition sample is obtained; the partition sample is multiplied by the corresponding semantic weight to obtain the weighted partition sample.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Time sequence action positioning method based on multi-time-resolution tense semantic aggregation network

    CN114821420A

  • Training method of power grid dispatching model and terminal

    CN115099606A

  • Point cloud semantic segmentation method and device

    CN116310310A

  • Feature fusion model training and sample retrieval method and device and computer equipment

    CN116580257A

  • Commodity sequence recommendation method based on time sequence perception self-attention and comparative learning

    CN117196763A