Time series data class increment classification method based on dual-channel fusion

Abstract

The invention discloses a time series data class increment classification method based on dual-channel fusion, and relates to the technical field of time series data class increment classification, and the method comprises the steps: training a time series data class increment classification model based on dual-channel fusion by taking the original data of a plurality of task classes as the data of a learning stage; the time series data class increment classification model based on the two-channel fusion after training in the previous learning stage is finished is used as a teacher model, and the time series data class increment classification model based on the two-channel fusion in the current learning stage is used as a student model for training, so that priori knowledge can be explicitly reserved, and disastrous forgetting can be overcome; and through the use of a joint optimization loss function, a mixed distillation strategy fusing feature-level and logic-level knowledge distillation is adopted, and more structured and more robust feature distribution is formed between incremental tasks, so that the method has stronger adaptability to cross-task distribution drift.

Description

Technical Field

[0001] The present invention relates to the technical field of incremental classification of time series data, and in particular to a method for incremental classification of time series data based on dual-channel fusion. Background Art

[0002] Time series data is ubiquitous in fields such as finance, transportation, and climate science. While existing methods have achieved promising performance in time series classification tasks, most still adhere to the traditional machine learning paradigm, which assumes static data. However, real-world temporal data often changes continuously in a streaming fashion. Inspired by this, Time Series Class Incremental Learning (TSCIL) was proposed to enable machines to continuously accumulate and optimize temporal knowledge.

[0003] A core challenge facing the TSCIL task is overcoming catastrophic forgetting—the phenomenon in which learning new information overwrites previously acquired knowledge. Existing methods primarily mitigate this problem by replaying old samples (Kwon, YD, Chauhan, J., & Mascolo, C. (2021). Fasticarl: Fast incremental classifier and representation learning with efficient budget allocation in audio sensing applications. In Proc. Interspeech (pp. 356–360)) or memorizing category prototypes (Qiao, Z., Hu, M., Jiang, X., Suganthan, PN, & Savitha, R. (2023). Classincremental learning on multivariate time series via shape-aligned temporal distillation. In Proc. ofIEEE Conference on Acoustics, Speech and Signal Processing (pp. 1–5)). However, these methods rely solely on learning one-dimensional temporal representations, making it difficult to capture key features such as trends, periodicity, and peaks. The limited representation capacity makes the feature space more susceptible to distribution drift when learning new tasks, thereby exacerbating catastrophic forgetting.

[0004] In the field of time series classification, a range of methods aim to reformulate the problem of image classification by transforming one-dimensional time series data into two-dimensional texture images. Research has shown that this transformation helps model global sequence patterns. However, directly applying this transformation to time series class incremental learning (TSCIL) can significantly lose local temporal structure, leading to inaccurate classification results. Summary of the Invention

[0005] The present invention discloses a time series data incremental classification method based on dual-channel fusion to overcome the above technical problems.

[0006] In order to achieve the above object, the technical solution of the present invention is: A time series data incremental classification method based on dual-channel fusion includes the following steps: S1: Acquisition The original time series data of task categories; The original time series data of task categories are divided into The data of each learning stage includes the original time series data of several task categories; Indicates the total number of task categories; represents the total number of learning stages; S2: Establishing an incremental classification model for time series data based on dual-channel fusion; the incremental classification model for time series data based on dual-channel fusion includes a time series feature extractor, an image feature extractor, and a cross-modal gated fusion module; S3: Based on the first learning stage The original time series data of task categories are used to obtain the feature representation of local time series patterns using a time series feature extractor; The index number of the task category in the first learning stage; S4: Based on the first learning stage The original time series of each task category is used to obtain a two-dimensional line graph image, and the image feature extractor is used to obtain the semantic feature representation of the image branch; S5: obtaining fusion features based on the feature representation representing the local temporal pattern and the semantic feature representation of the image branch based on a cross-modal gated fusion module; S6: Based on the fusion features, the original time series data is obtained based on the classifier module. The predicted probability of each task category; S7: Based on the classification constraint loss function, the incremental classification model for time series data based on dual-channel fusion is trained in the first learning phase; S8: Based on the The original time series data of the learning stage is repeatedly executed from S3 to S6, and based on the joint optimization loss function and the -1 learning stage of training based on dual-channel fusion of time series data incremental classification model, the time series data incremental classification model based on dual-channel fusion is sequentially Training of learning stages; to obtain the final trained time series data incremental classification model based on dual-channel fusion; S9: According to the new time series data, the final trained time series data incremental classification model based on dual-channel fusion is used to obtain the new time series data belonging to the first The probability of task categories is used to classify new time series data, where The index number indicating the task category.

[0007] Furthermore, the joint optimization loss function is expressed as follows: .

[0008] Where: represents the total loss of the joint optimization loss function; Both represent the weights of loss components; represents the feature distillation loss; represents the classification constraint loss; Represents the logits distillation loss represents the regularization constraint loss.

[0009] Furthermore, the method for obtaining the characteristic distillation loss is as follows: First, get the teacher model in Class average over task categories:

[0010] in: Indicates that the teacher model Class average over task categories; Indicates that it belongs to The total number of training samples for each task category; Indicates that it belongs to A collection of indexes of training samples of task categories; Indicates that it belongs to The index of the training samples of the task category; Indicates that it belongs to Task category A two-dimensional line graph image of a set of training sample indices; represents the teacher model feature extractor; Secondly, the teacher model is Normalize the class average over the task categories:

[0011] in: Indicates Unit vector in direction; Indicates the operation of calculating the two norm; Finally, the feature distillation loss is obtained as follows:

[0012] in: represents the student model feature extractor; Represents the teacher model feature extractor.

[0013] Furthermore, the logits distillation loss is expressed as follows:

[0014] =

[0015] in: represents the logits distillation loss; Represents the KL divergence calculation function; Represents the prediction results of the teacher model.

[0016] Furthermore, the regularized constraint loss is expressed as follows;

[0017] Where: The index number of the parameter in the temporal feature extractor; represents the total number of parameters in the temporal feature extractor; Represents the first The importance of the parameters; Represents the first parameters; represents the first parameters; The index number of the parameter in the image feature extractor; Represents the total number of parameters in the image feature extractor; Represents the first The importance of the parameters; Represents the first parameters; Represents the first parameters.

[0018] Furthermore, the formula used to obtain the feature representation that characterizes the local temporal pattern is as follows: =

[0019]

[0020] in: Represents the original time series data; represents the set of real numbers; Indicates the length of input data; Indicates the number of input data channels; Represents the intermediate time series characteristics, i.e. The output of a one-dimensional convolution module; Indicates the index number of the one-dimensional convolution module; Represents the total number of one-dimensional convolution modules in the temporal feature extractor; (·) indicates the A convolution block sequence consisting of a 1D convolution block, a normalization module, an activation function, a pooling module, and Dropout.

[0021] Furthermore, the computational process for obtaining the semantic feature representation of the image branch is as follows: =

[0022] =

[0023] = +

[0024] in: Represents the output of the first convolution operation in the ResNet Basic block; Represents the output of the second convolution operation in the ResNetBasic block; represents the activation function; Represents batch normalization operation; Represents the first convolution operation in the ResNet Basic block; Represents the second convolution operation in the ResNet Basic block; Represents the output of ResNet Basic block; represents the downsampling operation;

[0025]

[0026] in: represents the output of adaptive evaluation pooling; Represents the feature vector of the flattened output; Represents a flattening operation;

[0027] in: Semantic feature representation of image branches; Represents batch normalization operation; represents the learnable weights of the linear layer; Represents bias in the linear layer.

[0028] Furthermore, the formula used to obtain the fusion features is as follows: = +

[0029] in: Representation of features that characterize local temporal patterns; Semantic feature representation of image branches; represents fusion features; represents a learnable weight vector; Represents element-wise multiplication operation; Represents normalization operation; is with A vector of all 1s of the same dimension.

[0030] Furthermore, the formula used to obtain the final probability distribution is as follows:

[0031] in: Indicates that the original time series data belongs to The probability of task categories, ; The index number indicating the task category; represents a classifier consisting of a single-layer MLP and Softmax; Represents a linear classification layer.

[0032] Furthermore, the classification constraint loss function is expressed as follows:

[0033] Where: represents the classification constraint loss function; The index number indicating the task category; Indicates the total number of task categories; Indicates that the original time series data belongs to The probability of a task category.

[0034] Beneficial effects: The present invention provides an incremental classification method for time series data based on dual-channel fusion. By taking the original data of several task categories as the data of a learning stage, the incremental classification model of time series data based on dual-channel fusion is trained, and the incremental classification model of time series data based on dual-channel fusion after the training of the previous learning stage is used as the teacher model, and the incremental classification model of time series data based on dual-channel fusion in the current learning stage is trained as the student model. It can explicitly retain prior knowledge and overcome catastrophic forgetting, and through the use of a joint optimization loss function, a hybrid distillation strategy that integrates feature-level and logic-level knowledge distillation is adopted to form a more structured and robust feature distribution between incremental tasks, so that it has stronger adaptability to distribution drift across tasks. When applied to incremental learning of time series, the present invention solves the problem of significant loss of local time series structure, and the accuracy of classification results is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0036] Figure 1 This is a flowchart of a method for incremental classification of time series data based on dual-channel fusion provided by the present invention; Figure 2 This is a network structure diagram of incremental classification of time series data based on dual-channel fusion provided by the present invention; Figure 3 It is a schematic diagram of the generation of the teacher class average provided by the present invention; Figure 4 It is a schematic diagram of the specific operation of the CFG fusion module in the network; Figure 5 Schematic diagram of comparison of classification results on UCI-HAR in an embodiment of the present invention; Figure 6 Schematic diagram of comparison of classification results on DSA in an embodiment of the present invention; Figure 7 Schematic diagram of the visual comparison of the results of the embodiment of the present invention on the DSA dataset with other methods. DETAILED DESCRIPTION

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0038] This embodiment introduces a time series data incremental classification method based on dual-channel fusion, including the following steps: Figure 1 As shown: S1: Acquisition The original time series data of task categories; The original time series data of task categories are divided into Data from a learning phase; wherein the data from the learning phase includes original time series data of several task categories; Indicates the total number of task categories; represents the total number of learning stages; S2: Establishing a time series data incremental classification model based on dual-channel fusion; the time series data incremental classification model based on dual-channel fusion includes a time series feature extractor, an image feature extractor, and a cross-modal gated fusion module; Figure 2 As shown; Specifically, in this embodiment, DSA (Human Movement Dataset) and UCI-HAR (Human Activity Recognition Dataset) are used as basic data. Taking the UCI-HAR dataset as an example, the dataset contains six human activity categories (such as walking, going upstairs, going downstairs, standing, sitting, and lying down), that is, six task categories. The time series data corresponding to each category in UCI-HAR is obtained as the original time series data of the task category. In order to construct a class incremental learning scenario, this embodiment The original time series data of each task category is divided into three consecutive learning stages (Task 1, Task 2, Task 3). Each learning stage introduces original time series data of two mutually exclusive categories (for example: Task 1: {walk, go upstairs}; Task 2: {go downstairs, stand}; Task 3: {sit, lie down}). During the training process, the incremental classification model for time series data based on dual-channel fusion can only access the category data corresponding to the current learning stage. During the testing phase, all learned categories (from the first learning stage to the current learning stage) must be jointly evaluated. Through technologies such as dynamic expansion of classification heads or feature replay, the incremental classification model for time series data based on dual-channel fusion in this embodiment can effectively maintain the ability to discriminate the time series features of historical categories while gradually learning new categories, thereby alleviating the problem of catastrophic forgetting.

[0039] Furthermore, during the training of the incremental classification model for time series data based on dual-channel fusion, a teacher-student knowledge transfer framework is employed: when using Task 2, the task category data from the second learning phase, the model trained using Task 1 from the first learning phase serves as the teacher model, while the model trained using Task 2 from the current second learning phase serves as the student model. This progressive learning mechanism enables the continuous transfer of knowledge from previous tasks to new tasks.

[0040] S3: Based on the first learning stage The original time series data of the task category is extracted by the time series feature extractor. Extract the time series branch features from the original time series data of each task category to obtain the feature representation that represents the local time series pattern ;in, The index number of the task category in the first learning stage; Specifically, this embodiment uses multiple one-dimensional convolution modules to process the original time series data of each task category. Processing is performed to obtain the feature representation that characterizes the local temporal pattern , that is, local sensitive temporal feature representation , where the feature extractor of the temporal branch consists of four one-dimensional convolution modules, each of which consists of a 1D convolution block, a normalization module, an activation function, a pooling module, and Dropout; Preferably, the formula used to obtain the feature representation characterizing the local temporal pattern is as follows: In the feature extractor CNNEncoder of the time series branch, the original time series data is first input Converted into a channel-first form, and then extracted features of different scales through multiple layers of one-dimensional convolution modules. The output of each one-dimensional convolution module is used as a set of intermediate time series features. : = (1)

[0041] in: Represents the original time series data; represents the set of real numbers; Indicates the length of input data; Indicates the number of input data channels; Represents the intermediate time series characteristics, i.e. The output of a one-dimensional convolution module; Indicates the index number of the one-dimensional convolution module; Represents the total number of one-dimensional convolution modules in the temporal feature extractor; (·) indicates the A convolution block sequence consisting of a 1D convolution block, a normalization module, an activation function, a pooling module, and Dropout.

[0042] Specifically, the output of the last one-dimensional convolution module is the feature representation of the local temporal pattern to be obtained. After the features processed by the convolutional block sequence are finally obtained, average pooling is performed to obtain multi-scale temporal features under different receptive fields. Finally, after the features of all scales are obtained, they are projected to the same dimension through a linear mapping layer to enhance the ability to align different scales.

[0043] S4: According to the original time series , get the two-dimensional line chart image , using the image feature extractor to obtain the semantic feature representation of the image branch ; Specifically, the original time series data Convert to a 2D line chart image The method used is a conventional method in this field and will not be described in detail here. Among Dimensions are plotted with different color curves, and different channels are plotted with different color curves. Multiple images are combined into a unified grid to preserve the multi-dimensional structural information of the time series. To maintain the performance of this embodiment (i.e., to ensure the semantic feature representation of image branches, high aggregation of similar samples in the feature space, and maximum separation of heterogeneous samples), and to facilitate the processing of line graphs, the image size here is set to 512×512. In addition, for each channel of the time series data, we assign it a separate color and place it independently in a separate grid; Specifically, the two-dimensional line graph image The image is input into the image feature extractor for deep convolutional coding. The image feature extractor here uses the classic convolutional neural network structure ResNet32 based on residual blocks; its core structure ResNet Basic block is shown below: Converted to channel priority format, , and normalize and resize. Represents a two-dimensional line chart image; Indicates the height of the two-dimensional line chart image; Indicates the width of the 2D line chart image; Then, spatial features at different levels are extracted by stacking the initial convolutional layer and multiple residual structures. The output of each residual stacking stage is defined as a set of intermediate image features : = (2) in: Represents the index number of the ResNet Basic block in the image feature extractor; Represents the total number of ResNet Basic blocks in the image feature extractor; Represents the intermediate image features, that is, the output of the ResNetBasic block; Represents an image feature extractor composed of multiple stacked ResNet Basic blocks. Each ResNet Basic block contains two layers of convolution and residual connection structure.

[0044] Preferably, the computation process for obtaining the semantic feature representation of an image branch is as follows: = (3) = (4) = + (5) in: Represents the output of the first convolution operation in the ResNet Basic block; Represents the output of the second convolution operation in the ResNetBasic block; represents the activation function; Represents batch normalization operation; Represents the first convolution operation in the ResNet Basic block; Represents the second convolution operation in the ResNet Basic block; Represents the output of ResNet Basic block; Represents the downsampling operation; first, the input Perform two convolution operations and then use the obtained features After downsampling The final output of the ResNetBasic block is obtained above. Apply adaptive average pooling to obtain a global image feature representation with a spatial dimension of 1×1: (6) (7) in: represents the output of adaptive evaluation pooling; Represents the feature vector of the flattened output; Represents a flattening operation; Get the feature vector of the flattened output , The output dimension of the representation feature is projected into a unified representation space through a linear mapping layer to enhance the image's alignment with other modalities or tasks in subsequent modules: (8) in: Semantic feature representation of image branches; Represents batch normalization operation; represents the learnable weights of the linear layer; Indicates bias in the linear layer; Specifically, in the incremental classification model of time series data based on dual-channel fusion in this embodiment, the feature representation of the local time series pattern extracted is firstly and semantic feature representation of the image branch Composition of dual channels: First, convert the multi-dimensional time series into a two-dimensional line chart image: For the original time series data We plot the time series data of each dimension into an independent line graph, with the horizontal axis representing time and the vertical axis representing value. Different colors are used to enhance the distinction between the dimensions. Then, these line graphs are arranged in a fixed grid to form a two-dimensional line graph image. The conversion process can be formalized as ,in Represents a conversion function that maps a one-dimensional time series to a two-dimensional image.

[0045] S5: Feature representation based on the local temporal pattern and semantic feature representation of the image branch , based on the cross-modal gated fusion module, obtain fusion features ; Specifically, the feature representation of the local temporal pattern is and semantic feature representation of the image branch , input to the cross-modal gated fusion module (CGF module) to obtain fusion features ; Preferably, the formula used to obtain the fusion feature is as follows: = + (9) in: Represents the feature representation of the local temporal pattern, which is input by the feature extractor of the temporal branch to the original temporal time series Encoded; The semantic feature representation of the image branch is encoded by the image feature extractor on the input 2D line graph image; represents fusion features; represents a learnable weight vector; Represents element-wise multiplication operation; Represents normalization operation; is with A vector of all 1s of the same dimension.

[0046] Specifically, we first express the feature representation of the local temporal pattern The semantic feature representation of the image branch is multiplied element-wise and then normalized by the Softmax function. Next, a learnable weight vector with the same dimension as the feature vector is used. The fusion results are weighted and combined with the original time series features Perform weighted summation, such as Figure 4 As shown; S6: Based on the fusion features , based on the classifier module, obtain the original time series data belonging to the The predicted probability of each task category; Preferably, the formula used to obtain the final probability distribution is as follows: (10) in: Indicates that the original time series data belongs to The probability of task categories, ; The index number indicating the task category; represents a classifier consisting of a single-layer MLP and Softmax; represents a linear classification layer; Specifically, we will integrate the features Input to a classifier module consisting of a single layer MLP and Softmax Projection and classification are performed in to obtain the final classification probability cloth.

[0047] S7: Based on the classification constraint loss function, the incremental classification model for time series data based on dual-channel fusion is trained in the first learning phase; Preferably, the classification constraint loss function is expressed as follows: Specifically, in this embodiment, cross entropy loss is used to supervise the learning process of each new task. , and its cross entropy loss is expressed as follows:

[0048] Where: represents the classification constraint loss function; The index number indicating the task category; Indicates the total number of task categories; Indicates that the original time series data belongs to The probability of task categories; S8: Based on the The original time series data of the learning stage is repeatedly executed from S3 to S6, and based on the joint optimization loss function and the -1 learning stage of training based on dual-channel fusion of time series data incremental classification model, the time series data incremental classification model based on dual-channel fusion is sequentially training of the learning stage; to obtain the final trained incremental classification model of time series data based on dual-channel fusion; wherein the incremental classification model of time series data based on dual-channel fusion is sequentially trained. The model trained in the learning stage is the final trained incremental classification model for time series data based on dual-channel fusion.

[0049] Preferably, the joint optimization loss function is expressed as follows: .

[0050] represents the total loss of the joint optimization loss function; Both represent the weights of loss components; represents the feature distillation loss; represents the classification constraint loss; Represents the logits distillation loss represents the regularization constraint loss; Specifically, the joint optimization loss function of this embodiment includes the cross-entropy loss of the current task, the feature distillation loss of the image branch, the logits distillation loss of the output layer, and the regularization loss at the parameter level, which can achieve the retention of the learned content in the teacher model and the adaptation to new tasks.

[0051] Preferably, the characteristic distillation loss The method to obtain is as follows: Specifically, feature distillation plays a key role in alleviating catastrophic forgetting, because the representation space is the basis of the classification task. In this embodiment, we only perform feature distillation on the image branch, not on the time series branch. Before each new task begins, that is, when using the first The original time series data of the first learning stage is used to train the time series data incremental classification model based on dual-channel fusion. The teacher model (i.e. -1 learning stage after training the incremental classification model for time series data based on dual-channel fusion) is used to construct the mean features of each category of the image branch, construct the direction consistency constraint, and calculate the direction consistency loss to guide the current image features to align with the original representation, thereby maintaining the continuity of the feature space. In this embodiment, when training the incremental classification model for time series data based on dual-channel fusion based on the original time series data of each learning stage, the first -The time series data incremental classification model based on dual-channel fusion after training for 1 learning stage is used as the teacher model, and the model being trained in the current task is used as the student model.

[0052] Specifically, direction-aware feature distillation encourages the feature vector extracted by the student model to be as consistent as possible with the class mean feature direction of a certain category in the teacher model. Before training the current task, we first calculate the feature mean vector of the teacher model on each category, such as Figure 3 As shown, the formula is as follows: (11) in: Indicates that the teacher model Class average over task categories; Indicates that it belongs to The total number of training samples for each task category; Indicates that it belongs to A collection of indexes of training samples of task categories; Indicates that it belongs to The index of the training samples of the task category; Indicates that it belongs to Task category A two-dimensional line graph image of a set of training sample indices; represents the teacher model feature extractor; Next, the class average vector is L2 normalized to obtain the unit direction vector: (12) in: Indicates Unit vector in direction; Indicates the operation of calculating the two norm; Later, during the student model training process, we achieve directional consistency by minimizing the following loss function:

[0053] in: represents the student model feature extractor; represents the teacher model feature extractor; Compared with traditional direct alignment methods, this direction-aware feature distillation mechanism can more effectively preserve old task knowledge while adapting to the learning of new tasks.

[0054] Preferably, the logits distillation loss It is expressed as follows: Specifically, although knowledge distillation at the feature level helps maintain the distribution structure of the representation space, its effect on maintaining decision knowledge in the classifier is still limited. We hope that the classification probability output by the student model is consistent with that of the teacher model. Among them, logits distillation is used to compare the classification probability outputs of the teacher model and the student model, and to maintain the consistency of the output distribution by minimizing the KL divergence, thereby encouraging the decision distribution of the student model to be consistent with the teacher model, thereby enhancing the stability of the classification decision and reducing catastrophic forgetting. Logits distillation loss It is expressed as follows:

[0055] =

[0056] in: represents the logits distillation loss; Represents the KL divergence calculation function; Represents the prediction results of the teacher model; In this way, we are able to effectively preserve the decision boundary of the teacher model without interfering with the student model's ability to learn new tasks, thereby mitigating forgetting.

[0057] Preferably, the regularized constraint loss It is expressed as follows; Specifically, this embodiment introduces a selective parameter regularization mechanism to evaluate the contribution of each parameter to the previous task and add stronger retention constraints to key parameters to prevent them from being over-updated in the new task. Because it contains both time series branches and image branches, it is necessary to selectively regularize the parameters of the two branches separately. Specifically, the regularization constraint loss It is expressed as follows:

[0058] Where: The index number of the parameter in the temporal feature extractor; represents the total number of parameters in the temporal feature extractor; Represents the first The importance of the parameters; Represents the first parameters; represents the first parameters; The index number of the parameter in the image feature extractor; Represents the total number of parameters in the image feature extractor; Represents the first The importance of the parameters; Represents the first parameters; Represents the first parameters; Specifically, this embodiment evaluates the importance of each parameter to the learned task based on the model parameters in the current learning stage, and uses the parameters to update the important parameters when performing new tasks in the current learning stage, thereby effectively alleviating the catastrophic forgetting problem.

[0059] Specifically, the calculation of the regularized constraint loss measures the gradient change of the output feature with respect to each parameter, i.e., its sensitivity. The greater the impact of a parameter on the output, the greater its importance, indicating that the parameter is crucial to the teacher model and should be retained when training new tasks. Conversely, it indicates that the parameter is highly adaptable and more suitable for adapting to new tasks.

[0060] S9: According to the new time series data, the final trained time series data incremental classification model based on dual-channel fusion is used to obtain the new time series data belonging to the first The probability of each task category is used to classify new time series data.

[0061] Specifically, the trained model is applied to the test sample, and dual-branch feature extraction and fusion are performed to output the final prediction result. At the same time, the average accuracy (AT) and average forgetting rate (FT) of the model on all tasks are evaluated to verify the continuous learning ability and stability of the method under the condition of no sample playback. The prediction results of the label are as follows Figure 7 As shown, the total accuracy achieved by this embodiment on the DSA dataset is demonstrated.

[0062] This example compares the results with several state-of-the-art incremental time series learning methods, including five mainstream no-replay methods: Naive, LwF, MAS, EWC, and DT2W. To ensure fairness, all comparison methods were tested under the same no-replay setting, using the same model capacity and hyperparameter configurations. The experiments were conducted on two public datasets, UCI-HAR and DSA, and the model performance was comprehensively evaluated using the average accuracy (AT) and forgetting rate (FT).

[0063] Comparison results Figure 5 and Figure 6The results show that this embodiment achieved optimal performance on all evaluation indicators of both datasets. Specifically, on UCI-HAR, this embodiment improved the average accuracy by 2.1% compared to the second-best method, DT2W, and by 12.4% on the DSA dataset. In terms of the FT indicator, which measures the degree of forgetting, this embodiment also achieved the lowest value, significantly outperforming other methods and demonstrating excellent anti-forgetting capabilities. In particular, as subsequent tasks were gradually added, this embodiment demonstrated a gradually enhanced performance advantage, reflecting its strong robustness and adaptability in long-term incremental learning.

[0064] In this embodiment, by considering learning a comprehensive and robust feature space, the problem of catastrophic forgetting caused by distribution drift in the incremental learning process of time series data (TSCIL) is solved. A joint representation framework including time series and images is established, namely, an incremental classification model for time series data based on dual-channel fusion. The entire incremental classification model for time series data based on dual-channel fusion is a progressive mutual guidance network (PMGNet), which is divided into mutual guidance of primitive visual features and combined visual features at multiple different scales. The distribution drift caused by TSCIL is addressed by enhancing the robustness of feature representation, and a joint optimization loss function is adopted. A hybrid distillation strategy that integrates feature-level and logic-level knowledge distillation is adopted. This not only enhances the representation learning ability, but also forms a more structured and robust feature distribution between incremental tasks, making it more adaptable to distribution drift across tasks.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A time series data incremental classification method based on dual-channel fusion, characterized by: The steps include: S1: Acquisition The original time series data of task categories; The original time series data of task categories are divided into The data of each learning stage includes the original time series data of several task categories; Indicates the total number of task categories; represents the total number of learning stages; S2: Establishing an incremental classification model for time series data based on dual-channel fusion; the incremental classification model for time series data based on dual-channel fusion includes a time series feature extractor, an image feature extractor, and a cross-modal gated fusion module; S3: Based on the first learning stage The original time series data of task categories are used to obtain the feature representation of local time series patterns using a time series feature extractor; The index number of the task category in the first learning stage; S4: Based on the first learning stage The original time series of each task category is used to obtain a two-dimensional line graph image, and the image feature extractor is used to obtain the semantic feature representation of the image branch; S5: obtaining fusion features based on the feature representation representing the local temporal pattern and the semantic feature representation of the image branch based on a cross-modal gated fusion module; S6: Based on the fusion features, the original time series data is obtained based on the classifier module. The predicted probability of each task category; S7: Based on the classification constraint loss function, the incremental classification model for time series data based on dual-channel fusion is trained in the first learning phase; S8: Based on the The original time series data of the learning stage is repeatedly executed from S3 to S6, and based on the joint optimization loss function and the -1 learning stage of training based on dual-channel fusion of time series data incremental classification model, the time series data incremental classification model based on dual-channel fusion is sequentially Training of learning stages; to obtain the final trained time series data incremental classification model based on dual-channel fusion; S9: According to the new time series data, the final trained time series data incremental classification model based on dual-channel fusion is used to obtain the new time series data belonging to the first The probability of task categories is used to classify new time series data, where The index number indicating the task category.

2. The method for incremental classification of time series data based on dual-channel fusion according to claim 1 is characterized in that: The joint optimization loss function is expressed as follows: Where: represents the total loss of the joint optimization loss function; Both represent the weights of loss components; represents the feature distillation loss; represents the classification constraint loss; Represents the logits distillation loss represents the regularization constraint loss.

3. The method for incremental classification of time series data based on dual-channel fusion according to claim 2 is characterized in that: The method for obtaining the characteristic distillation loss is as follows: First, get the teacher model in Class average over task categories: in: Indicates that the teacher model Class average over task categories; Indicates that it belongs to The total number of training samples for each task category; Indicates that it belongs to A collection of indexes of training samples of task categories; Indicates that it belongs to The index of the training samples of the task category; Indicates that it belongs to Task category A two-dimensional line graph image of a set of training sample indices; represents the teacher model feature extractor; Secondly, the teacher model is Normalize the class average over the task categories: in: Indicates Unit vector in direction; Indicates the operation of calculating the two norm; Finally, the feature distillation loss is obtained as follows: in: represents the student model feature extractor; Represents the teacher model feature extractor.

4. The method for incremental classification of time series data based on dual-channel fusion according to claim 2 is characterized in that: The logits distillation loss is expressed as follows: = in: represents the logits distillation loss; Represents the KL divergence calculation function; Represents the prediction results of the teacher model.

5. The method for incremental classification of time series data based on dual-channel fusion according to claim 2 is characterized in that: The regularized constraint loss is expressed as follows; Where: The index number of the parameter in the temporal feature extractor; represents the total number of parameters in the temporal feature extractor; Represents the first The importance of the parameters; Represents the first parameters; represents the first parameters; The index number of the parameter in the image feature extractor; Represents the total number of parameters in the image feature extractor; Represents the first The importance of the parameters; Represents the first parameters; Represents the first parameters.

6. The method for incremental classification of time series data based on dual-channel fusion according to claim 1 is characterized in that: The formula used to obtain the feature representation that characterizes the local temporal pattern is as follows: = in: Represents the original time series data; represents the set of real numbers; Indicates the length of input data; Indicates the number of input data channels; Represents the intermediate time series characteristics, i.e. The output of a one-dimensional convolution module; Indicates the index number of the one-dimensional convolution module; Represents the total number of one-dimensional convolution modules in the temporal feature extractor; (·) indicates the A convolution block sequence consisting of a 1D convolution block, a normalization module, an activation function, a pooling module, and Dropout.

7. The method for incremental classification of time series data based on dual-channel fusion according to claim 1 is characterized in that: The computational process for obtaining the semantic feature representation of the image branch is as follows: = = = + in: Represents the output of the first convolution operation in the ResNet Basic block; Represents the output of the second convolution operation in ResNet Basicblock; represents the activation function; Represents batch normalization operation; Represents the first convolution operation in the ResNet Basic block; Represents the second convolution operation in the ResNetBasic block; Represents the output of ResNet Basic block; represents the downsampling operation; in: represents the output of adaptive evaluation pooling; Represents the feature vector of the flattened output; Represents a flattening operation; in: Semantic feature representation of image branches; Represents batch normalization operation; represents the learnable weights of the linear layer; Represents bias in the linear layer.

8. The method for incremental classification of time series data based on dual-channel fusion according to claim 7 is characterized in that: The formula used to obtain the fusion features is as follows: = + in: Representation of features that characterize local temporal patterns; Semantic feature representation of image branches; represents fusion features; represents a learnable weight vector; Represents element-wise multiplication operation; Represents normalization operation; is with A vector of all 1s of the same dimension.

9. The method for incremental classification of time series data based on dual-channel fusion according to claim 8, characterized in that: The formula used to obtain the final probability distribution is as follows: in: Indicates that the original time series data belongs to The probability of task categories, ; The index number indicating the task category; represents a classifier consisting of a single-layer MLP and Softmax; Represents a linear classification layer.

10. The method for incremental classification of time series data based on dual-channel fusion according to claim 1, characterized in that: The classification constraint loss function is expressed as follows: Where: represents the classification constraint loss function; The index number indicating the task category; Indicates the total number of task categories; Indicates that the original time series data belongs to The probability of a task category.

Images