A time series data class incremental classification method based on double-channel fusion

Abstract

The application discloses a kind of based on dual-channel fusion's time series data class incremental classification method, it is related to time series data class incremental classification technical field, by the original data of several task categories, as a learning stage data, based on dual-channel fusion's time series data class incremental classification model is trained, and the based on dual-channel fusion's time series data class incremental classification model after the previous learning stage training is ended as teacher model, based on dual-channel fusion's time series data class incremental classification model of current learning stage is trained as student model, can explicitly retain prior knowledge and overcome catastrophic forgetting, and by the use of joint optimization loss function, using the mixed distillation strategy of fusion feature level and logic level knowledge distillation, more structured, more robust feature distribution is formed between incremental tasks, so that it has stronger adaptability to cross-task distribution drift.

Description

Technical Field

[0001] This invention relates to the field of incremental classification technology for time-series data, and in particular to an incremental classification method for time-series data based on dual-channel fusion. Background Technology

[0002] Time series data is ubiquitous in fields such as finance, transportation, and climate science. Although existing methods have achieved satisfactory performance in time series classification tasks, most still follow the traditional machine learning paradigm under static assumptions. However, real-world time data often changes continuously in a streaming form. Inspired by this, Time Series Class Incremental Learning (TSCIL) has been proposed to enable machines to continuously accumulate and optimize time series knowledge.

[0003] The core challenge of the TSCIL task is overcoming catastrophic forgetting—that is, the learning of new knowledge overwrites previously acquired knowledge. Existing methods mainly alleviate this problem by replaying old samples (Kwon, YD, Chauhan, J., & Mascolo, C. (2021). Fasticarl: Fast incremental classifier and representation learning with efficient budget allocation in audio sensing applications. In Proc. Interspeech (pp. 356–360)) or remembering class prototypes (Qiao, Z., Hu, M., Jiang, X., Suganthan, PN, & Savitha, R. (2023). Classincremental learning on multivariate time series via shape-aligned temporal distillation. In Proc. of IEEE Conference on Acoustics, Speech and Signal Processing (pp. 1–5)). However, these methods rely only on learning one-dimensional temporal representations, making it difficult to capture key features such as trends, periodicity, and peaks. Limited representational capacity makes the feature space more susceptible to distribution drift when learning new tasks, thus exacerbating catastrophic forgetting.

[0004] In the field of time series classification, a number of methods have been dedicated to redefining it as an image classification problem by transforming one-dimensional time series data into two-dimensional textured images. Research shows that this transformation helps model global sequence patterns. However, directly applying such transformations to Time Series Class Incremental Learning (TSCIL) may lead to a significant loss of local temporal structure, resulting in inaccurate classification results. Summary of the Invention

[0005] This invention discloses an incremental classification method for time-series data based on dual-channel fusion to overcome the aforementioned technical problems.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] An incremental classification method for time-series data based on dual-channel fusion includes the following steps:

[0008] S1: Obtain The original time-series data for each task category; and The original time-series data for each task category are divided into: Data for each learning stage, where each learning stage includes raw time-series data for several task categories; Indicates the total number of task categories; This indicates the total number of learning stages;

[0009] S2: Establish an incremental classification model for time-series data based on dual-channel fusion; the incremental classification model for time-series data based on dual-channel fusion includes a time-series feature extractor, an image feature extractor, and a cross-modal gating fusion module;

[0010] S3: Based on the first learning stage... The original time-series data for each task category were used to extract feature representations of local time-series patterns using a time-series feature extractor; among them... The index number representing the task category of the first learning stage;

[0011] S4: Based on the first learning stage... The original time series of each task category is used to obtain a two-dimensional line graph image, which is then used to extract the semantic feature representation of the image branches using an image feature extractor.

[0012] S5: Based on the feature representation of the local temporal pattern and the semantic feature representation of the image branch, obtain the fused features using the cross-modal gating fusion module;

[0013] S6: Based on the fusion features and the classifier module, determine which category the original time-series data belongs to. Predicted probabilities for each task category;

[0014] S7: Based on the classification constraint loss function, train the first learning stage of the incremental classification model for time series data based on dual-channel fusion.

[0015] S8: Based on the first The original time-series data from each learning stage are used to repeatedly execute S3~S6, and the loss function is optimized based on the joint optimization loss function and the first... The incremental classification model for time-series data based on dual-channel fusion, trained after one learning phase, is then subjected to the following steps: Training in several learning phases to obtain the final trained incremental classification model for time-series data based on dual-channel fusion;

[0016] S9: Based on the new time-series data, the final trained incremental classification model based on dual-channel fusion is used to determine the class of the new time-series data. The probability of each task category is used to classify new time-series data. Indicates the index number of the task category.

[0017] Furthermore, the joint optimization loss function is expressed as follows:

[0018] .

[0019] In the formula: This represents the total loss of the joint optimization loss function; Both represent the weights of the loss components; Indicates characteristic distillation loss; Represents the classification constraint loss; Indicates logits distillation loss This represents the loss due to regularization constraints.

[0020] Furthermore, the method for obtaining the characteristic distillation loss is as follows:

[0021] First, obtain the teacher model in the... Average number of classes across task categories:

[0022]

[0023] in: The teacher model is represented in the first... Average number of classes across task categories; Indicates belonging to the first The total number of training samples for each task category; Indicates belonging to the first A set of indices of training samples for each task category; Indicates belonging to the first Indexes of training samples for each task category; Indicates belonging to the first The first task category A two-dimensional line graph image of a set of training sample indices; This represents the feature extractor for the teacher model;

[0024] Secondly, the teacher model will be used in the first... Normalize the class average across task categories:

[0025]

[0026] in: Indicates in Unit vector in the direction; This indicates the operation of calculating the L2 norm;

[0027] Finally, the feature distillation loss is obtained as follows:

[0028]

[0029] in: This represents the feature extractor for the student model. This represents the feature extractor for the teacher model.

[0030] Furthermore, the logits distillation loss is expressed as follows:

[0031]

[0032] =

[0033] in: This indicates the logits distillation loss; This represents the KL divergence calculation function; This represents the prediction results of the teacher model.

[0034] Furthermore, the regularization constraint loss is expressed as follows;

[0035]

[0036] In the formula: Indicates the index number of the parameter in the temporal feature extractor; This represents the total number of parameters in the temporal feature extractor; The first in the temporal feature extractor The importance of each parameter; The first in the temporal feature extractor One parameter; This represents the first time-series feature extractor in the teacher model. One parameter; Indicates the index number of the parameter in the image feature extractor; This represents the total number of parameters in the image feature extractor; This represents the first feature extractor in the image. The importance of each parameter; This represents the first feature extractor in the image. One parameter; The first image feature extractor in the teacher model represents the first... One parameter.

[0037] Furthermore, the formula used to obtain the feature representation of the local temporal pattern is as follows:

[0038] =

[0039]

[0040] in: Represents the original time series data; Represents the set of real numbers; Indicates the length of the input data; Indicates the number of input data channels; Representing intermediate time series characteristics, i.e., the first... The output of a one-dimensional convolutional module; Indicates the index number of the one-dimensional convolution module; This represents the total number of one-dimensional convolutional modules in the temporal feature extractor. (·) indicates the first A sequence of convolutional blocks consisting of 1D convolutional blocks, normalization modules, activation functions, pooling modules, and Dropout.

[0041] Furthermore, the calculation process for obtaining the semantic feature representation of the image branch is as follows:

[0042] =

[0043] =

[0044] = +

[0045] in: This represents the output of the first convolution operation in the ResNet Basic block; This represents the output of the second convolution operation in the ResNetBasic block; Indicates the activation function; This indicates a batch normalization operation; This represents the first convolution operation in the ResNet Basic block; This represents the second convolution operation in the ResNet Basic block; This represents the output of the ResNet Basic block; Indicates a downsampling operation;

[0046]

[0047]

[0048] in: This represents the output of adaptive evaluation pooling; This represents the feature vector output after flattening; Indicates the flattening operation;

[0049]

[0050] in: Semantic feature representation of image branches; This indicates a batch normalization operation; This represents the learnable weights of a linear layer; This represents paranoia in a linear layer.

[0051] Furthermore, the formula used to obtain the fusion features is as follows:

[0052] = +

[0053] in: This represents the feature representation of a local temporal pattern; Semantic feature representation of image branches; Indicates fusion characteristics; Represents a learnable weight vector; This indicates element-wise multiplication. This indicates a normalization operation; Is with Vectors of all dimensions consisting of only 1s.

[0054] Furthermore, the formula used to obtain the final probability distribution is as follows:

[0055]

[0056] in: This indicates that the original time series data belongs to the first... The probability of each task category. ; Index number indicating the task category; This represents a classifier composed of a single-layer MLP and Softmax. This represents a linear classification layer.

[0057] Furthermore, the classification constraint loss function is expressed as follows:

[0058]

[0059] In the formula: Represents the classification constraint loss function; Index number indicating the task category; Indicates the total number of task categories; This indicates that the original time series data belongs to the first... The probability of each task category.

[0060] Beneficial Effects: This invention provides an incremental classification method for time-series data based on dual-channel fusion. It trains a dual-channel fusion-based incremental classification model by using the original data from several task categories as a learning stage. The model trained in the previous learning stage is used as the teacher model, and the model trained in the current learning stage is used as the student model. This method explicitly preserves prior knowledge and overcomes catastrophic forgetting. Furthermore, by jointly optimizing the loss function, it employs a hybrid distillation strategy that integrates feature-level and logical-level knowledge distillation, resulting in a more structured and robust feature distribution across incremental tasks, thus enhancing its adaptability to distribution drift across tasks. When applied to incremental learning of time series data, this invention solves the problem of significant loss of local temporal structure, greatly improving the accuracy of classification results. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart of the method for incremental classification of time-series data based on dual-channel fusion provided by the present invention;

[0063] Figure 2 This is a network structure diagram of incremental classification of time-series data based on dual-channel fusion provided by the present invention;

[0064] Figure 3 This is a schematic diagram illustrating the generation of the average value for the teacher class provided by the present invention;

[0065] Figure 4 This is a schematic diagram illustrating the specific operation of the CFG fusion module in the network;

[0066] Figure 5 This is a schematic diagram showing the comparison of classification results in this embodiment of the invention on UCI-HAR.

[0067] Figure 6 This is a schematic diagram showing the comparison of classification results on DSA in an embodiment of the present invention.

[0068] Figure 7 A schematic diagram illustrating the visualization comparison of results with other methods on the DSA dataset in this embodiment of the invention. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] This embodiment introduces an incremental classification method for time-series data based on dual-channel fusion, including the following steps: Figure 1 As shown:

[0071] S1: Obtain The original time-series data for each task category; and The original time-series data for each task category are divided into the following categories: Data for each learning stage; wherein the data for each learning stage includes raw time-series data for several task categories; Indicates the total number of task categories; This indicates the total number of learning stages;

[0072] S2: Establish an incremental classification model for time-series data based on dual-channel fusion; the incremental classification model for time-series data based on dual-channel fusion includes a time-series feature extractor, an image feature extractor, and a cross-modal gating fusion module; such as Figure 2 As shown;

[0073] Specifically, in this embodiment, DSA (Human Motion Dataset) and UCI-HAR (Human Activity Recognition Dataset) are used as the basic data. Taking the UCI-HAR dataset as an example, this dataset contains six categories of human activities (such as walking, going upstairs, going downstairs, standing, sitting, and lying down), i.e., six task categories. The time-series data corresponding to each category in UCI-HAR is obtained as the raw time-series data for each task category. To construct the class incremental learning scenario, this embodiment... The original temporal data for each task category is divided into three consecutive learning stages (Task 1, Task 2, Task 3). Each learning stage introduces original temporal data for two mutually exclusive categories (e.g., Task 1: {walking, going upstairs}; Task 2: {going downstairs, standing}; Task 3: {sitting, lying down}). During training, the incremental classification model based on dual-channel fusion of temporal data can only access the category data corresponding to the current learning stage. During the testing stage, all learned categories (from the first learning stage to the current learning stage) need to be jointly evaluated. By dynamically expanding the classification head or using feature replay techniques, the incremental classification model based on dual-channel fusion of temporal data in this embodiment effectively maintains its ability to discriminate historical category temporal features while gradually learning new categories, thereby mitigating the catastrophic forgetting problem.

[0074] Furthermore, during the training of the incremental classification model for time-series data based on dual-channel fusion, a teacher-student knowledge transfer framework is adopted: when using Task 2 data from the second learning stage, the model trained on Task 1 data from the first learning stage is used as the teacher model, and the model trained on Task 2 data from the current second learning stage is used as the student model. Through this progressive learning mechanism, knowledge is continuously transferred from historical tasks to new tasks.

[0075] S3: Based on the first learning stage... The original time-series data of each task category are processed using a time-series feature extractor. Temporal branch features are extracted from the raw time-series data of each task category to obtain feature representations that characterize local temporal patterns. ;in, The index number representing the task category of the first learning stage;

[0076] Specifically, this embodiment uses multiple one-dimensional convolutional modules to process the raw time-series data of each task category. The process yields feature representations that characterize local temporal patterns. That is, locally sensitive temporal feature representation The feature extractor of the temporal branch consists of four one-dimensional convolutional modules, each of which consists of a 1D convolutional block, a normalization module, an activation function, a pooling module, and Dropout.

[0077] Preferably, the formula used to obtain the feature representation of the local temporal pattern is as follows:

[0078] In the CNNEncoder, a feature extractor in the temporal branch, the original temporal data is first processed... The data is converted to a channel-priority form, and then processed through multiple one-dimensional convolutional modules to extract features at different scales. The output of each one-dimensional convolutional module serves as a set of intermediate temporal features. :

[0079] = (1)

[0080]

[0081] in: Represents the original time series data; Represents the set of real numbers; Indicates the length of the input data; Indicates the number of input data channels; Representing intermediate time series characteristics, i.e., the first... The output of a one-dimensional convolutional module; Indicates the index number of the one-dimensional convolution module; This represents the total number of one-dimensional convolutional modules in the temporal feature extractor. (·) indicates the first A sequence of convolutional blocks consisting of 1D convolutional blocks, normalization modules, activation functions, pooling modules, and Dropout.

[0082] Specifically, the output of the last one-dimensional convolutional module is the feature representation of the local temporal pattern that is to be obtained. After obtaining the features processed by the convolutional block sequence, average pooling is performed to acquire multi-scale temporal features under different receptive fields. Finally, after obtaining the features at all scales, they are projected onto the same dimension through a linear mapping layer to enhance the alignment ability between different scales.

[0083] S4: Based on the original time series Obtain a 2D polyline image To obtain semantic feature representations of image branches using an image feature extractor. ;

[0084] Specifically, the raw time series data Convert to a 2D line graph image The method used is a conventional method in this field and will not be described in detail here. Each variable channel, i.e. Among them Dimensions are plotted using curves of different colors, with different channels plotted as curves of different colors. Multiple images are combined into a unified grid to preserve the multi-dimensional structural information of the time series. To maintain the performance of this embodiment (i.e., ensuring the semantic feature representation of image branches, high aggregation of similar samples in the feature space, and maximum separation of dissimilar samples), and to facilitate the processing of line charts, the image size is set to 512×512. Furthermore, for each channel of the time series data, we assign it a unique color and place it independently in a separate grid.

[0085] Specifically, the two-dimensional line graph image The input is fed into the image feature extractor, where it undergoes deep convolutional encoding. The image feature extractor here uses the classic ResNet32 convolutional neural network structure based on residual blocks; its core ResNet Basic block structure is represented as follows: Convert to channel-first format. And then normalization and size adjustment were performed. Among them, Represents a two-dimensional line graph image; Indicates the height of a two-dimensional line graph; Indicates the width of a two-dimensional line chart image;

[0086] Subsequently, spatial features at different levels are extracted through an initial convolutional layer and multiple stacked residual structures. The output of each residual stacking stage is defined as a set of intermediate image features. :

[0087] = (2)

[0088] in: Indicates the index number of the ResNet Basic block in the image feature extractor; This represents the total number of ResNet Basic blocks in the image feature extractor; This represents the intermediate image features, i.e., the output of the ResNetBasic block; This represents an image feature extractor composed of multiple stacked ResNet Basic blocks, each of which contains two layers of convolutional and residual connection structures.

[0089] Preferably, the calculation process for obtaining the semantic feature representation of the image branch is as follows:

[0090] = (3)

[0091] = (4)

[0092] = + (5)

[0093] in: This represents the output of the first convolution operation in the ResNet Basic block; This represents the output of the second convolution operation in the ResNetBasic block; Indicates the activation function; This indicates a batch normalization operation; This represents the first convolution operation in the ResNet Basic block; This represents the second convolution operation in the ResNet Basic block; This represents the output of the ResNet Basic block; This indicates a downsampling operation; first, the input... Perform two convolution operations, and then use the resulting features. Compared with downsampling The final output is obtained by performing residual connections. The final output of the ResNetBasic block obtained above is shown below. Applying adaptive average pooling yields a global image feature representation with a spatial dimension of 1×1:

[0094] (6)

[0095] (7)

[0096] in: This represents the output of adaptive evaluation pooling; This represents the feature vector output after flattening; Indicates the flattening operation;

[0097] The flattened feature vector is obtained. , The output dimension represents the features; it is projected onto a unified representation space through a linear mapping layer to enhance the image's alignment with other modalities or tasks in subsequent modules.

[0098] (8)

[0099] in: Semantic feature representation of image branches; This indicates a batch normalization operation; This represents the learnable weights of a linear layer; This indicates paranoia in the linear layer;

[0100] Specifically, in the dual-channel fusion-based incremental classification model for time-series data in this embodiment, the extracted features representing local time-series patterns are first represented... Semantic feature representation of image branches Composing a dual-channel system: First, the multi-dimensional time series is converted into a two-dimensional line graph image: for the original time series data We plotted the time series data for each dimension as a separate line chart, with the horizontal axis representing time and the vertical axis representing values. Different colors were used to enhance the distinction between the different dimensions. Then, these line charts were arranged according to a fixed grid and combined into a single two-dimensional line chart image. This transformation process can be formalized as ,in This represents a transformation function that maps a one-dimensional time series to a two-dimensional image.

[0101] S5: Based on the feature representation of the local temporal pattern Semantic feature representation of image branches Based on the cross-modal gating fusion module, fusion features are obtained. ;

[0102] Specifically, the feature representation of local temporal patterns Semantic feature representation of image branches The input is fed into the cross-modal gated fusion module (CGF module) to obtain fused features. ;

[0103] Preferably, the formula used to obtain the fusion features is as follows:

[0104] = + (9)

[0105] in: The feature representation representing the local temporal pattern is obtained by the feature extractor of the temporal branch from the input original temporal time series. It is encoded; The semantic feature representation of the image branch is encoded by the image feature extractor from the input two-dimensional polyline image; Indicates fusion characteristics; Represents a learnable weight vector; This indicates element-wise multiplication. This indicates a normalization operation; Is with Vectors of all dimensions consisting of only 1s.

[0106] Specifically, firstly, the feature representation of local temporal patterns... The semantic feature representations of the image branches are multiplied element-wise and then normalized using the Softmax function. Next, a learnable weight vector with the same dimension as the feature vector is used. The fusion results are weighted and compared with the original time series features. Perform a weighted summation, such as Figure 4 As shown;

[0107] S6: Based on the fusion features Based on the classifier module, the original time series data is obtained as belonging to the first... Predicted probabilities for each task category;

[0108] Preferably, the formula used to obtain the final probability distribution is as follows:

[0109] (10)

[0110] in: This indicates that the original time series data belongs to the first... The probability of each task category. ; Index number indicating the task category; This represents a classifier composed of a single-layer MLP and Softmax. Represents a linear classification layer;

[0111] Specifically, we will integrate the features The input is fed into a classifier module consisting of a single-layer MLP and Softmax. The projection and classification are performed to obtain the final classification probability distribution.

[0112] S7: Based on the classification constraint loss function, train the first learning stage of the incremental classification model for time series data based on dual-channel fusion.

[0113] Preferably, the classification constraint loss function is expressed as follows:

[0114] Specifically, in this embodiment, cross-entropy loss is used to supervise the learning process for each new task. Given input... Its cross-entropy loss is expressed as follows:

[0115]

[0116] In the formula: Represents the classification constraint loss function; Index number indicating the task category; Indicates the total number of task categories; This indicates that the original time series data belongs to the first... The probability of each task category;

[0117] S8: Based on the first The original time-series data from each learning stage are used to repeatedly execute S3~S6, and the loss function is optimized based on the joint optimization loss function and the first... The incremental classification model for time-series data based on dual-channel fusion, trained after one learning phase, is then subjected to the following steps: The training process involves several learning phases to obtain the final trained incremental classification model for time-series data based on dual-channel fusion. The incremental classification model for time-series data based on dual-channel fusion is then subjected to the following steps sequentially: The trained model from each learning stage becomes the final trained incremental classification model for time-series data based on dual-channel fusion.

[0118] Preferably, the joint optimization loss function is expressed as follows:

[0119] .

[0120] This represents the total loss of the joint optimization loss function; Both represent the weights of the loss components; Indicates characteristic distillation loss; Represents the classification constraint loss; Indicates logits distillation loss This represents the loss due to regularization constraints;

[0121] Specifically, the joint optimization loss function in this embodiment includes the cross-entropy loss of the current task, the feature distillation loss of the image branch, the logits distillation loss of the output layer, and the regularization loss at the parameter level, which can achieve the preservation of the content learned in the teacher model and the adaptation to new tasks.

[0122] Preferably, characteristic distillation loss The method to obtain it is as follows:

[0123] Specifically, feature distillation plays a crucial role in mitigating catastrophic forgetting because the representation space is fundamental to classification tasks. In this embodiment, we perform feature distillation only on the image branch, not the time series branch. Before each new task begins, i.e., when using the... The original time-series data from each learning stage are used to train an incremental classification model for time-series data based on dual-channel fusion. The teacher model (i.e., the first...) is then extracted. In this embodiment, when training the incremental classification model for time-series data based on dual-channel fusion after the first training stage, the mean features of each category in the image branch are used to construct directional consistency constraints and calculate directional consistency loss. This is used to guide the current image features to align with the original representation, thereby maintaining the continuity of the feature space. The incremental classification model based on dual-channel fusion for time-series data, trained after one learning phase, is used as the teacher model, while the model currently being trained in the task is used as the student model.

[0124] Specifically, orientation-aware feature distillation encourages the student model to extract feature vectors that are as consistent as possible with the orientation of the class mean feature of a certain category in the teacher model. Before training the current task, we first calculate the feature mean vector of the teacher model for each category, such as... Figure 3 As shown, the formula is as follows:

[0125] (11)

[0126] in: The teacher model is represented in the first... Average number of classes across task categories; Indicates belonging to the first The total number of training samples for each task category; Indicates belonging to the first A set of indices of training samples for each task category; Indicates belonging to the first Indexes of training samples for each task category; Indicates belonging to the first The first task category A two-dimensional line graph image of a set of training sample indices; This represents the feature extractor for the teacher model;

[0127] Next, the average vector is L2 normalized to obtain the unit direction vector:

[0128] (12)

[0129] in: Indicates in Unit vector in the direction; This indicates the operation of calculating the L2 norm;

[0130] Subsequently, during the student model training process, we achieve directional consistency by minimizing the following loss function:

[0131]

[0132] in: This represents the feature extractor for the student model. This represents the feature extractor for the teacher model;

[0133] This orientation-aware feature distillation mechanism, compared to traditional direct alignment methods, can more effectively retain knowledge from old tasks while adapting to the learning of new tasks.

[0134] Preferably, the logits distillation loss It is expressed as follows:

[0135] Specifically, while feature-level knowledge distillation helps maintain the distribution structure of the representation space, its effect on preserving decision knowledge in the classifier remains limited. We aim for the classification probabilities output by the student model to be consistent with those of the teacher model. Logits distillation is used to compare the classification probability outputs of the teacher and student models. By minimizing the KL divergence, it maintains the consistency of the output distribution, encouraging the student model's decision distribution to align with the teacher model, thereby enhancing the stability of classification decisions and reducing catastrophic forgetting. The logits distillation loss... It is expressed as follows:

[0136]

[0137] =

[0138] in: This indicates the logits distillation loss; This represents the KL divergence calculation function; This represents the prediction results of the teacher model;

[0139] In this way, we can effectively preserve the decision boundaries of the teacher model without interfering with the student model's ability to learn new tasks, thereby slowing down forgetting.

[0140] Preferably, the regularization constraint loss It is represented as follows;

[0141] Specifically, this embodiment introduces a selective parameter regularization mechanism to evaluate the contribution of each parameter to the previous task and adds stronger retention constraints to key parameters to prevent them from being over-updated in the new task. Because it includes both time series and image branches, selective regularization needs to be applied to the parameters of both branches separately. Specifically, the regularization constraint loss... It is expressed as follows:

[0142]

[0143] In the formula: Indicates the index number of the parameter in the temporal feature extractor; This represents the total number of parameters in the temporal feature extractor; The first in the temporal feature extractor The importance of each parameter; The first in the temporal feature extractor One parameter; This represents the first time-series feature extractor in the teacher model. One parameter; Indicates the index number of the parameter in the image feature extractor; This represents the total number of parameters in the image feature extractor; This represents the first feature extractor in the image. The importance of each parameter; This represents the first feature extractor in the image. One parameter; The first image feature extractor in the teacher model represents the first... One parameter;

[0144] Specifically, this embodiment evaluates the importance of each parameter to the learned task based on the model parameters in the current learning stage, and effectively alleviates the catastrophic forgetting problem by updating the important parameters when performing new tasks in the current learning stage.

[0145] Specifically, the regularization constraint loss is calculated by measuring the gradient change of the output feature with respect to each parameter, i.e., its sensitivity. The greater the influence of a parameter on the output, the greater its importance, indicating that the parameter is crucial to the teacher model and should therefore be retained when training new tasks. Conversely, a smaller influence indicates higher adjustability and greater suitability for adapting to new tasks.

[0146] S9: Based on the new time-series data, the final trained incremental classification model based on dual-channel fusion is used to determine the class of the new time-series data. The probability of each task category is used to classify new time-series data.

[0147] Specifically, the trained model is applied to test samples, performing dual-branch feature extraction and fusion to output the final prediction result. Simultaneously, the model's average accuracy (AT) and average forgetting rate (FT) are evaluated across all tasks to verify the method's continuous learning ability and stability under conditions without sample replay. The label prediction results are as follows: Figure 7 As shown, the overall accuracy achieved by this embodiment on the DSA dataset is illustrated.

[0148] This embodiment compares the model with several state-of-the-art time-series incremental learning methods, including five mainstream replay-free methods: Naive, LwF, MAS, EWC, and DT2W. To ensure fairness, all comparison methods were performed under the same replay-free settings, using the same model capacity and hyperparameter configurations. Experiments were conducted on two public datasets, UCI-HAR and DSA, to comprehensively evaluate the model's performance on both average accuracy (AT) and forgetting rate (FT).

[0149] Comparison results Figure 5 and Figure 6 This demonstrates that the proposed implementation achieves state-of-the-art performance across all evaluation metrics for both datasets. Specifically, on the UCI-HAR dataset, the proposed implementation achieves a 2.1% improvement in average accuracy compared to the second-best method, DT2W, and a further 12.4% improvement on the DSA dataset. The proposed implementation also achieves the lowest value on the Fourier Transform (FT) metric, which measures the degree of forgetting, significantly outperforming other methods and exhibiting superior resistance to forgetting. Particularly noteworthy is the progressively increasing performance advantage demonstrated by the addition of subsequent tasks, showcasing its strong robustness and adaptability in long-term incremental learning.

[0150] In this embodiment, by considering the learning of a comprehensive and robust feature space, the problem of catastrophic forgetting caused by distribution drift during incremental learning of time-series data in TSCIL is addressed. A joint representation framework including temporal and image data is established, namely a time-series data incremental classification model based on dual-channel fusion. The entire dual-channel fusion-based time-series data incremental classification model is a Progressive Mutual Guidance Network (PMGNet), which consists of mutual guidance of primitive visual features and combined visual features at multiple different scales. By enhancing the robustness of feature representation to address the distribution drift caused by TSCIL, a joint optimization loss function is adopted, employing a hybrid distillation strategy that fuses feature-level and logical-level knowledge distillation. This not only enhances representation learning capabilities but also forms a more structured and robust feature distribution across incremental tasks, thereby making it more adaptable to cross-task distribution drift.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An incremental classification method for time-series data based on dual-channel fusion, characterized in that, Includes the following steps: S1: Obtain The original time-series data for each task category; and The original time-series data for each task category are divided into: Data for each learning stage, where each learning stage includes raw time-series data for several task categories; Indicates the total number of task categories; This indicates the total number of learning stages; S2: Establish an incremental classification model for time-series data based on dual-channel fusion; the incremental classification model for time-series data based on dual-channel fusion includes a time-series feature extractor, an image feature extractor, and a cross-modal gating fusion module; S3: Based on the first learning stage... The original time-series data for each task category were used to extract feature representations of local time-series patterns using a time-series feature extractor; among them... The index number representing the task category of the first learning stage; S4: Based on the first learning stage... The original time series of each task category is used to obtain a two-dimensional line graph image, which is then used to extract the semantic feature representation of the image branches using an image feature extractor. S5: Based on the feature representation of the local temporal pattern and the semantic feature representation of the image branch, obtain the fused features using the cross-modal gating fusion module; S6: Based on the fusion features and the classifier module, determine which category the original time-series data belongs to. Predicted probabilities for each task category; S7: Based on the classification constraint loss function, train the first learning stage of the incremental classification model for time series data based on dual-channel fusion. S8: Based on the first The original time-series data from each learning stage are used to repeatedly execute S3~S6, and the loss function is optimized based on the joint optimization loss function and the first... The incremental classification model for time-series data based on dual-channel fusion, trained after one learning phase, is then subjected to the following steps: Training in several learning phases to obtain the final trained incremental classification model for time-series data based on dual-channel fusion; S9: Based on the new time-series data, the final trained incremental classification model based on dual-channel fusion is used to determine the class of the new time-series data. The probability of each task category is used to classify new time-series data. Indicates the index number of the task category.

2. The incremental classification method for time-series data based on dual-channel fusion according to claim 1, characterized in that, The joint optimization loss function is expressed as follows: In the formula: This represents the total loss of the joint optimization loss function; Both represent the weights of the loss components; Indicates characteristic distillation loss; Represents the classification constraint loss; Indicates logits distillation loss This represents the loss due to regularization constraints.

3. The incremental classification method for time-series data based on dual-channel fusion according to claim 2, characterized in that, The method for obtaining the characteristic distillation loss is as follows: First, obtain the teacher model in the... Average number of classes across task categories: in: The teacher model is represented in the first... Average quantity across task categories; Indicates belonging to the first The total number of training samples for each task category; Indicates belonging to the first A set of indices of training samples for each task category; Indicates belonging to the first Indexes of training samples for each task category; Indicates belonging to the first The first task category A two-dimensional line graph image of a set of training sample indices; This represents the feature extractor for the teacher model; Secondly, the teacher model will be used in the first... Normalize the class average across task categories: in: Indicates in Unit vector in the direction; This indicates the operation of calculating the L2 norm; Finally, the feature distillation loss is obtained as follows: in: This represents the feature extractor for the student model. This represents the feature extractor for the teacher model.

4. The incremental classification method for time-series data based on dual-channel fusion according to claim 2, characterized in that, The distillation loss of logits is expressed as follows: = in: This indicates the logits distillation loss; This represents the KL divergence calculation function; This represents the prediction results of the teacher model.

5. The incremental classification method for time-series data based on dual-channel fusion according to claim 2, characterized in that, The regularization constraint loss is expressed as follows; In the formula: Indicates the index number of the parameter in the temporal feature extractor; This represents the total number of parameters in the temporal feature extractor; The first in the temporal feature extractor The importance of each parameter; The first in the temporal feature extractor One parameter; This represents the first time-series feature extractor in the teacher model. One parameter; Indicates the index number of the parameter in the image feature extractor; This represents the total number of parameters in the image feature extractor; This represents the first feature extractor in the image. The importance of each parameter; This represents the first feature extractor in the image. One parameter; The first image feature extractor in the teacher model represents the first... One parameter.

6. The incremental classification method for time-series data based on dual-channel fusion according to claim 1, characterized in that, The formula used to obtain the feature representation of local temporal patterns is as follows: = in: Represents the raw time series data; Represents the set of real numbers; Indicates the length of the input data; Indicates the number of input data channels; Representing intermediate time series characteristics, i.e., the first... The output of a one-dimensional convolutional module; Indicates the index number of the one-dimensional convolution module; This represents the total number of one-dimensional convolutional modules in the temporal feature extractor. (·) indicates the first A sequence of convolutional blocks consisting of 1D convolutional blocks, normalization modules, activation functions, pooling modules, and Dropout.

7. The incremental classification method for time-series data based on dual-channel fusion according to claim 1, characterized in that, The calculation process for obtaining the semantic feature representation of the image branch is as follows: = = = + in: This represents the output of the first convolution operation in the ResNet Basic block; This represents the output of the second convolution operation in the ResNet Basicblock; Indicates the activation function; This indicates a batch normalization operation; This represents the first convolution operation in the ResNet Basic block; This represents the second convolution operation in the ResNetBasic block; This represents the output of the ResNet Basic block; Indicates a downsampling operation; in: This represents the output of adaptive evaluation pooling; This represents the feature vector output after flattening; Indicates the flattening operation; in: Semantic feature representation of image branches; This indicates a batch normalization operation; This represents the learnable weights of a linear layer; This represents paranoia in a linear layer.

8. The incremental classification method for time-series data based on dual-channel fusion according to claim 7, characterized in that, The formula used to obtain the fusion features is as follows: = + in: This represents the feature representation of a local temporal pattern; Semantic feature representation of image branches; Indicates fusion characteristics; Represents a learnable weight vector; This indicates element-wise multiplication. This indicates a normalization operation; Is with Vectors of all dimensions consisting of only 1s.

9. The incremental classification method for time-series data based on dual-channel fusion according to claim 8, characterized in that, The formula used to obtain the final probability distribution is as follows: in: This indicates that the original time series data belongs to the first... The probability of each task category. ; Index number indicating the task category; This represents a classifier composed of a single-layer MLP and Softmax. This represents a linear classification layer.

10. The incremental classification method for time-series data based on dual-channel fusion according to claim 1, characterized in that, The classification constraint loss function is expressed as follows: In the formula: Represents the classification constraint loss function; Index number indicating the task category; Indicates the total number of task categories; This indicates that the original time series data belongs to the first... The probability of each task category.

Images