A class imbalance time series domain adaptation method and system based on multi-expert consensus entropy

Abstract

The application discloses a class imbalance time series domain self-adaptive system based on multi-expert consensus entropy, and comprises a random enhancement module, a feature extraction module and a multi-expert committee module.The random enhancement module is used for performing random enhancement of source domain data and target domain data in multiple different types, wherein the source domain data is labeled time series data, and the target domain data is unlabeled time series data.The feature extraction module is used for performing feature extraction on the enhanced source domain data and target domain data.The multi-expert committee module comprises multiple expert classification sub-modules and is used for performing classification prediction on the source domain data and target domain data processed by the feature extraction module.

Description

Technical Field

[0001] This invention relates to the field of machine learning, specifically to the field of transfer learning, and more specifically, to a method and system for adaptive learning of imbalanced time series domains based on multi-expert consensus entropy. Background Technology

[0002] Behavior recognition based on wearable devices (such as smart bracelets, health trackers, and smartwatches—electronic devices worn on the body that acquire information from time-series signals collected by sensors during human activity) plays a crucial role in ubiquitous computing scenarios. Accurate inference of human activity behavior relies on training models using sufficient time-series data acquired through wearable devices. However, when the availability of acquired data is limited, and there are distributional discrepancies between training data (source domain) and test data (target domain), obtaining the expected results solely through model training becomes challenging. Traditional domain adaptation methods can alleviate this problem by adjusting features between the source and target domains.

[0003] However, in the real world, limitations in data collection or inherent characteristics often lead to imbalanced label distribution (i.e., class imbalance). Take wearable behavior recognition for Parkinson's disease as an example: the tremor level in Parkinson's patients is typically correlated with the severity of their condition. However, high-level tremors are relatively rare among patients, making it more difficult to obtain such data. Conversely, data on mild tremors are relatively easier to obtain, as this is more common in patients with milder conditions. In this situation, due to the imbalance between different classes, the model may tend to focus more on the relatively easily obtainable low-tremor level data during training, while performing poorly on high-level tremor data. This results in highly unreliable pseudo-labels, and using them for domain adaptation, which is based on domain alignment, leads to error accumulation. Because the model favors the class with the larger sample size, this label distribution imbalance can cause the model to perform poorly on minority classes during training and testing, resulting in poor predictive performance when faced with rare events. Therefore, this label distribution imbalance limits the application of domain adaptation in tasks with large data class differences, such as Parkinson's disease behavior recognition.

[0004] In existing technologies, to address the problem of label distribution imbalance, reference 2 proposes an image augmentation-based committee consensus algorithm. For each target instance, it generates a random, label-preserving image transformation committee. Then, it calculates the consistency between the model predictions of the original and augmented versions. Next, it minimizes the prediction entropy on highly consistent target instances (increasing model confidence) and maximizes the prediction entropy on highly inconsistent target instances (decreasing model confidence). This encourages the model to increase its confidence in predicting unlabeled target data. This makes its success highly dependent on its initialization. With good initialization, after training in the source domain, the classes can be reasonably aligned between the source and target domains. This method effectively alleviates the bias caused by class imbalance to some extent.

[0005] However, the above methods have the following problems.

[0006] First, the effectiveness of committee consensus depends on the degree of label imbalance. As label imbalance increases, the committee will exhibit bias towards most categories.

[0007] Secondly, this method relies on image augmentation (a method of generating new images by performing various transformations on the original image, including rotation, flipping, scaling, cropping, and color transformation), which is not directly applicable to time-series data acquired from personal wearable devices as described above. To adapt to time-series data, a sufficiently large semantically invariant augmentation space specific to the classification task is also needed.

[0008] Thirdly, unlike general image enhancement which can be applied to most classification tasks, time-series data may fail when the transformation is detrimental to the task. For example, in tasks involving classifying noisy patterns, introducing additional noise can lead to semantic inconsistencies.

[0009] References:

[0010] [1]Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li, "Fixmatch: Simplifying semi-supervised learning with consistency and confidence," in NeurlPS, 2020.

[0011] [2]Viraj Prabhu,Shivam Khare,Deeksha Kartik,and JudyHoffman,"Sentry:Selective entropy optimization via committee consistency for unsuperviseddomain adaptation,"in Proc.ICCV,2021.

[0012] [3]Weili Shi,Ronghang Zhu,and Sheng Li,"Pairwise adversarial trainingfor unsupervised class-imbalanced domain adaptation,"in Proc.ACM SIGKDD,2022.

[0013] [4]Vladimir N Vapnik,"An overview of statistical learning theory,"IEEE Trans.Neural Netw.,1999.

[0014] [5]Yaroslav Ganin,Evgeniya Ustinova,Hana Ajakan,Pascal Germain,HugoLarochelle, Laviolette,Mario Marchand,and Victor Lempitsky,"Domainadversarial training of neural networks,"J.Mach.Learn.Res.,2016.

[0015] [6]Kuniaki Saito,Kohei Watanabe,Yoshitaka Ushiku,andTatsuya Harada,"Maximum classifier discrepancy for unsupervised domain adaptation,"inProc.CVPR,2018.

[0016] [7]Shuang Li, Fangrui Lv, Binhui Xie, Chi Harold Liu, Jian Liang, and Chen Qin, "Bi-classifier determinacy maximization for unsupervised domain adaptation," in Proc. AAA1, 2021. Summary of the Invention

[0017] To address the aforementioned issues, this invention proposes a class-imbalanced time series domain adaptation method and system based on multi-expert consensus entropy. By introducing a multi-expert committee mechanism, boundary adjustments are made according to the imbalanced distribution of the source domain. This avoids bias in the minority class by the multi-expert committee while encouraging the model to exhibit strong generalization performance across a wider range of label distribution shifts. Furthermore, this application introduces a time-frequency dynamic consensus enhancement mechanism to dynamically adapt to the classification task while addressing domain shifts in the source / target domain.

[0018] This invention provides a quasi-imbalanced time series domain adaptive system based on multi-expert consensus entropy, comprising:

[0019] The random augmentation module is used to perform various types of random augmentation on source domain / target domain data, wherein the source domain data is labeled time series data and the target domain data is unlabeled time series data;

[0020] The feature extraction module is used to extract features from the enhanced source / target domain data;

[0021] The multi-expert committee module includes multiple expert classification sub-modules, which are used to classify and predict the source / target domain data after it has been processed by the feature extraction module.

[0022] According to the system described in the embodiments of the present invention, wherein,

[0023] The enhancement methods used by the random enhancement module include no enhancement, time-domain enhancement, frequency-domain enhancement, and dynamic weighted enhancement in both the time and frequency domains.

[0024] According to the system described in the embodiments of the present invention, wherein,

[0025] The multi-expert committee module includes at least one source distribution expert classification submodule and one balanced distribution expert classification submodule.

[0026] According to the system described in the embodiments of the present invention, wherein,

[0027] The expert committee module also includes a randomized expert submodule.

[0028] The present invention also provides a method for adaptive domain management of imbalanced time series based on the system described in any of the above embodiments, the method comprising:

[0029] Step S1: Obtain time series data for the source domain and target domain respectively;

[0030] Step S2: Perform classification training on the time series data of the source domain / target domain respectively to obtain the sum of the cross-entropy function of the source domain and the uniform entropy function of the target domain, and backpropagate to the system;

[0031] Step S3: Adjust the training parameters of the system based on the results obtained in step 2;

[0032] Step S4: Repeat steps S2 and S3 to minimize the sum of the cross-entropy function and the uniform entropy function.

[0033] The method according to embodiments of the present invention, wherein,

[0034] The source domain training in step S2 includes:

[0035] Step A1: Use the time-frequency enhancement module to perform various types of random enhancements on the source domain data.

[0036] Step A2: The randomly augmented data enters the feature extraction module for feature extraction.

[0037] Step A3: Input the features obtained in step A2 into multiple expert classification sub-modules to obtain different prediction outputs. Calculate the corresponding cross-entropy based on the prediction output of each expert classification sub-module and the label of the source domain data, and add the multiple cross-entropies obtained.

[0038] The method according to embodiments of the present invention, wherein,

[0039] The target domain training in step S2 includes:

[0040] Step B1: Use the time-frequency enhancement module to perform various types of random enhancement on the acquired target domain data.

[0041] Step B2: The randomly augmented data enters the feature extraction module for feature extraction.

[0042] Step B3: Input the features obtained in step B2 into multiple expert classification sub-modules to obtain different prediction outputs, and calculate the consistency entropy of the different prediction outputs based on the obtained multiple prediction outputs.

[0043] The method according to embodiments of the present invention, wherein,

[0044] The training in step S2 is represented as follows:

[0045]

[0046] in, For the overall training function, Denotes uniform entropy, δ 1-K This represents the generalized Kronecker function, used to control the sign of the uniform entropy. and Let y represent the data spaces of the source and target domains, respectively. s Here, represents the label of the source domain, Aug() is the enhancement function, K represents the number of expert classification submodules, and i represents the i-th expert classification submodule among the K expert classification submodules. Training functions representing different expert classification submodules.

[0047] According to the method described in this embodiment of the invention, in step S2, the expert classification submodule makes different adjustments to the decision boundary, as follows:

[0048]

[0049] Among them, E i π represents the i-th expert classification submodule. i Indicates E i Logit adjustments were performed.

[0050] The present invention provides a storage medium storing a computer program that, when executed by a processor, can be used to implement the methods described in any of the above embodiments.

[0051] The present invention provides an electronic device, including a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, can be used to implement the methods described in any of the above embodiments. Attached Figure Description

[0052] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0053] Figure 1 This diagram illustrates the class imbalance problem in existing domain adaptive learning techniques, where... Figure 1 (a) in the dataset is a balanced dataset. Figure 1 (b) in the dataset is an imbalanced dataset. Figure 1 (c) in the dataset represents a highly imbalanced dataset.

[0054] Figure 2 A schematic diagram of a class-imbalanced time series domain adaptive method and system based on multi-expert consensus entropy according to embodiments of this application is shown, wherein, Figure 2 (a) in the diagram is the system architecture diagram. Figure 2(b) in the diagram is a schematic diagram of the method. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0056] Before describing the embodiments of the present invention in detail, some of the terms used therein are explained as follows:

[0057] Transfer learning, also known as inductive transfer or domain adaptation, aims to apply knowledge or patterns learned in one domain (source domain) or task to different but related domains (target domain) or problems.

[0058] The source domain refers to the domain in which existing knowledge resides during transfer learning.

[0059] The target domain refers to the domain in which the new knowledge to be learned in transfer learning resides.

[0060] A multi-class decision boundary is a boundary learned by a deep learning model in a multi-class classification task. It is used to distinguish input data of different classes, that is, the decision boundary that the model makes for samples of different classes.

[0061] Domain adaptation, a type of transfer learning, is used in machine learning when the source and target domains have different data distributions but the tasks are the same. It allows a model trained on source domain data with high accuracy to be applied to a target domain with less data.

[0062] Figure 1 This diagram illustrates the class imbalance problem in existing domain adaptive learning techniques, where... Figure 1 (a) in the dataset is a balanced dataset. Figure 1 (b) in the dataset is an imbalanced dataset. Figure 1 (c) in the dataset represents a highly imbalanced dataset.

[0063] Figure 1 In each diagram, the triangle on the left represents an instance of category 1 in the source domain, and the circle on the right represents an instance of category 2. The number of instances of category 1 is greater than the number of instances of category 2. Figure 1 In each figure, solid-lined circles represent instances of the target domain, while circles without solid lines represent enhanced instances of the target domain.

[0064] based on Figure 1The class imbalance problem exists in image processing, as mentioned in the background section. Reference 2 has proposed a committee consensus algorithm based on image enhancement. This algorithm generates additional enhancement instances (such as...) within the target domain. Figure 1 (As shown by the circles without solid lines in the figures), it selectively minimizes entropy when there is consistency and maximizes entropy when there is inconsistency to illustrate committee consistency. However, it also has the limitations mentioned in the background art.

[0065] Based on the above issues, Figure 2 A schematic diagram of a class-imbalanced time series domain adaptive method and system based on multi-expert consensus entropy according to embodiments of this application is shown, wherein, Figure 2 (a) in the diagram is the system architecture diagram. Figure 2 (b) in the diagram is a schematic diagram of the method.

[0066] I. System Structure

[0067] First, let's interpret the symbols in the diagram: Let the data spaces of the source and target domains be respectively... and Consider classification tasks for target domains that have the same label set as the source domain. Where V is the number of categories (i.e., the target domain classification task and the source domain classification task have the same set of labels (e.g., the source domain classifies cats and dogs, and the target domain also classifies cats and dogs, with no other categories)). The source domain with labels is... The unlabeled target domain is Where N s and N t The number of instances in the domain. Feature By feature extraction module F (i.e. Figure 2 The shared feature extractor in (a) is generated and used as input to different expert classification submodules.

[0068] In this application, the source and target domains belong to the same category (same classification task), share the same category space, and fall under the same use case. According to one embodiment of the invention, both the source and target domains belong to Parkinson's tremor level identification. The source domain data, for example, originates from public databases such as those from hospitals. The target domain data, for example, originates from time-series data related to personal symptoms acquired from sensors in personal wearable devices. Given that the data acquired in this application is time-series data, in both cases, existing image enhancement methods are ineffective for classifying time-series data.

[0069] First, when the impact of the transformation on the task is negligible, for example, when the noise is very small and can be ignored.

[0070] Second, when the enhancement method is detrimental to the task. For example, in a task involving classifying noise patterns, introducing additional noise may lead to semantic inconsistencies.

[0071] Therefore, based on the above issues, such as Figure 2 As shown in (a) of this application, the model includes: a stochastic enhancement module (corresponding to...) Figure 2 (a) The random augmentation function Aug() is used to randomly augment the source / target domain data.

[0072] Specifically, based on the first problem, in one embodiment of this application, the random enhancement module uses a time-frequency enhancement library to enhance the source / target domain data. The time-frequency domain library may contain any enhancement operations that have a significant impact on the time-frequency domain.

[0073] According to one embodiment of this application, based on the CL-HAR enhancement library designed for time-series signals, a time-frequency enhancement library is proposed, containing 7 time-domain enhancement methods and 5 frequency-domain enhancement iterations. During adaptive training, K different enhancement techniques are used to enhance the source / target domain data to fully utilize the information of the signal in both the time and frequency domains.

[0074] Specifically, the introduced enhancement methods include:

[0075] A. Temporal Augmentation:

[0076]

[0077]

[0078] B. Frequency domain enhancement:

[0079]

[0080] According to one embodiment of this application, based on the second problem, the random enhancement module uses a method of adding dynamic weighting to the time-frequency domain enhancement method to enhance the source domain / target domain data.

[0081] According to one embodiment of this application, the random enhancement module also includes a method of not enhancing the source domain / target domain data.

[0082] This application adapts the augmentation method based on the consistency generated by random augmentation, aiming to utilize a method as consistent as possible with the one used in the task. In each iteration, at iteration time, based on the normalized matrix... Perform enhanced sampling.

[0083] like Figure 2 As shown in (a) of this application, the system further includes a feature extraction module F, which is used to extract features from the enhanced source / target domain data.

[0084] According to one embodiment of this application, the system further includes a multi-expert committee module, which performs classification prediction on the enhanced source domain / target domain data respectively. In the source domain, cross-entropy is calculated based on the true label and the prediction result. In the target domain, consensus entropy is calculated by multi-expert voting based on the prediction results of multiple classifiers.

[0085] It includes multiple (K) expert classification sub-modules. The data enhanced by the random enhancement module is then processed by the feature extraction module (shared feature extractor) to extract features before being input into the K expert classification sub-modules.

[0086] exist Figure 2 In example (a), three expert classification submodules are shown: "Source Distribution Expert (E...)" S "Equilibrium Distribution Expert (E)" E ")" and "Random Distribution Expert (E)" R )".

[0087] Among them, the source distribution expert (E S ) is an expert classification submodule generated based on the source distribution, such as Figure 1 As illustrated in the prior art regarding imbalanced datasets, the data at the decision boundary may be skewed due to the influence of the source distribution. The source distribution expert classification submodule is designed to address this situation.

[0088] Equilibrium Distribution Expert (E E The expert classification submodule considers balanced distributions. Based on the same class distribution, i.e., when the current dataset is imbalanced, this classification submodule is used to balance the dataset at the decision boundary.

[0089] Random distribution expert (E R () refers to randomly selecting a method to classify data.

[0090] Based on the multi-expert committee module that includes multiple expert classification sub-modules, logical compensation is used to increase the learning difficulty of instances with fewer categories, i.e., to increase the learning difficulty of minority classes, thereby enhancing the system model's ability to predict rare events (minority classes).

[0091] According to one embodiment of this application, the multi-expert committee module consists of at least one E S And 1 E E Composition. Different expert classification submodules will lead to different adjustments to the decision boundary (e.g. Figure 2 (See the schematic diagram of the decision boundary to the right of each expert classification submodule in (a)).

[0092] According to one embodiment of the present invention, the definitions of the three specific expert classification sub-modules are described as follows:

[0093] Source Distribution Expert Classification Submodule

[0094] Benchmark: Classification is based on the original cross-entropy loss, where the prior probability π is set as the zero vector.

[0095] Objective: The goal of source distribution experts is to obtain the classification boundary when all samples are selected uniformly in the source domain.

[0096] Working method: Using the original cross-entropy loss, by... i Let P be the source label distribution. s (y i This ensures uniform selection of source domain samples within the source domain.

[0097] Loss function:

[0098]

[0099] Equilibrium Distribution Expert Classification Submodule

[0100] Benchmark: Balanced distribution experts address the class imbalance problem in the source domain by modifying predicted probabilities using the label distribution as a prior.

[0101] Objective: The goal is to address the class imbalance problem in the source domain by adjusting the predicted probabilities to better mitigate minority class discrimination.

[0102] Working method: Optimize cross-entropy loss using optimal adjustment. Adjust the prediction probabilities to ensure that equilibrium experts give priority to minority classes in the source domain compared to source domain experts.

[0103] The formula is:

[0104]

[0105] π i =P s (y i ( ) represents the source label distribution. By adjusting the prediction probabilities, the equilibrium expert prioritizes the minority class in the source domain compared to the source domain expert.

[0106] Randomized distribution expert classification submodule (Expert_ER):

[0107] Benchmark: When K>3, the π of the random expert _ER i Let it be π i =μP s (y i)+ε, where μ represents random weights and ε represents random noise.

[0108] Objective: The goal of stochastic experts is to introduce randomness into the adjustment of decision boundaries.

[0109] Working method: Using random weights and noise, by adjusting π i Let π be the weighted source label distribution. i =μP s (y i )+ε, to introduce randomness to adjust the expert's decision boundary.

[0110] Loss function:

[0111]

[0112] In order to make E E The expert classification submodule prioritizes minority classes, which increases the difficulty of supervised learning for minority classes in the source domain.

[0113] When K>3, the random expert E R π i Let it be π i =μP s (y i )+ε, where μ represents random weights and ε represents random noise.

[0114] In the embodiments of this application, all K are the same: K = number of branches in different expert classification submodules = number of enhancement functions = number of expert classification submodules, K ≥ 2.

[0115] II. Training Methods

[0116] Figure 2 (b) shows the basis Figure 2 The method for training the model structure shown in (a) is as follows:

[0117] The entire training process can be divided into source domain training and target domain training. After each training iteration, the cross-entropy of the source domain training and the uniform entropy of the target domain training are obtained. These two are added together and used to backpropagate the model (backpropagate the gradient function) for the next batch of training. The resulting function is minimized through multiple iterations of training. The training methods for the source domain and target domain are explained below:

[0118] Regardless of the source or target domain, during each training iteration, the augmentation function Aug() for each instance (each instance in the source and target domains of the current batch) is based on W... norm The normalized probabilities are selected from the time-frequency augmentation library. At the end of the iteration, if the same prediction results are produced, the weight of augmentation W(i) is increased by 1, and then augmentation is performed separately and input into the training model.

[0119] In one embodiment of this application, the initial weights of the model are set to 1000.

[0120] In a preferred embodiment of this application, the initial weights of the model are 10-10000.

[0121] Source domain: Data in the source domain has labels.

[0122] Step A1: Use the time-frequency enhancement module to randomly enhance the source domain data.

[0123] Step A2: The randomly augmented data is then used for feature extraction.

[0124] Step A3: Input the features obtained in step A2 into multiple expert classification submodules to obtain different prediction outputs, and minimize the cross-entropy between the prediction output and the true label.

[0125] That is, during the training of the source domain, the cross-entropy of each expert classification submodule can be calculated based on the real label and the predicted output, and the multiple results are added together and backpropagated to the model.

[0126] Target domain: The target domain data is unlabeled. The consistent entropy is calculated based on the system of this application during training.

[0127] Step B1: Use the time-frequency enhancement module to randomly enhance the acquired target domain data.

[0128] Step B2: The randomly augmented data enters the feature extraction module for feature extraction.

[0129] Step B3: Input the features obtained in step B2 into multiple expert classification submodules to obtain different prediction outputs. Calculate the consistency entropy of the different prediction outputs and use δ... 1-K Represents the generalized Kronecker function, which minimizes the uniform entropy of target domain samples with the same predicted value. To enhance the model's confidence in the sample, samples with different predicted values ​​maximize the consistency entropy. This reduces the model's confidence in the sample.

[0130] In practical use, we use a simple majority voting scheme: for each target sample, after different enhancements and different expert classification submodules, a softmax layer is obtained (the expert classification submodule includes a linear layer (such as one or more fully connected layers) for linearly transforming the sample features to obtain classification features, and a softmax layer for calculating the confidence probability of each category based on the classification features. The category with the highest confidence probability in the final output of the classification submodule is used as the predicted classification result). The category corresponding to the largest softmax value is the predicted category of the target sample. The prediction result of the classification submodule uses one-hot encoding (only one category is 1 and the rest are 0). If all three categories are the same, they are considered consistent, at which point the consistency entropy is minimized and the model's confidence in the sample is maximized. If they are different, they are considered inconsistent, at which point the consistency entropy is maximized and the model's confidence in the sample is minimized.

[0131] In summary, the method of this application uses the enhancement function Aug() and the unspecified expert classification submodule E() for composite processing (i.e., concatenating the two modules). For example, taking target domain data as an example, target domain data x t The output is represented as:

[0132] ME(x t ) = E i (F(Aug i (x t )))

[0133] The entire training function is represented as

[0134]

[0135] in, Let be the overall training function, where

[0136]

[0137] Denotes uniform entropy, δ 1-K This represents the generalized Kronecker function, used to control the sign of the uniform entropy, i.e., if the predicted class (y) j If all branches of the expert classification submodule are equal, return 1; otherwise, return 0. The above formula represents the result for sample x. t It is y j The predicted probability, P θ This represents the predicted probability.

[0138] Based on the above training method, the cross-entropy of all expert classification submodules in the source domain can be minimized, and the decision boundary can be adjusted according to the imbalanced distribution of the source domain to avoid bias towards the minority class. On the other hand, the model optimizes the consistency entropy in the target domain based on the different results of different expert classification submodules. This application minimizes the sum of the obtained cross-entropy function and the consistency entropy function through iterative training to make the model have higher robustness and stronger generalization ability.

[0139] This application conducted experiments based on the above methods, and the relevant details are as follows:

[0140] Experimental Setup: This application uses a pre-trained ResNet34-1D model for all methods (methods 1-5 in the prior art and the method of this application). The batch size is set to 32, and the Adam optimizer is used for optimization. The learning rate is set to 0.003, and the weight decay parameter is set to 0.0005. The maximum epoch is configured to be 70. All experiments are conducted in 3 different randomized trials. The initial value of each W(i) is 1000. In the experiments described below, μ ~ Bernoulli(1, p = 0.5) follows a Bernoulli distribution (the Bernoulli distribution is a discrete probability distribution in probability theory that describes a random variable with only two values ​​(0 or 1). The random variable X can take the value 0 or 1, where p is the probability of 1 and 1-p is the probability of 0.) and ε = 0. In practical applications, this distribution can be adjusted to a uniform distribution or other suitable forms.

[0141] Experimental metrics: High overall accuracy can be achieved using trivial solution schemes on highly imbalanced datasets. Therefore, average accuracy (AvgAcc) is used for evaluation.

[0142] Experimental Datasets: Experiments were conducted on activity recognition and Parkinson's tremor recognition, comparing the proposed method with existing techniques. The behavior recognition dataset includes four behavior categories (walking / getting up / getting down / sitting). The sample counts for these categories across the three datasets are as follows: WISDM(W) (8258 / 2341 / 1901 / 1158), PAMAP2(P) (1584 / 905 / 808 / 1430), and UCIHAR(U) (893 / 809 / 746 / 926). For Parkinson's tremor recognition, levels from 0 to 3 represent the severity of the tremor. Data with accelerometer values ​​greater than 25 and significant label errors were removed. In the four Parkinson's tremor time-series datasets, the sample counts for these categories were: Tim-Tremor (TI) (1180 / 761 / 696 / 455), PdAssist (PA) (5939 / 270 / 257 / 38), IMU-Wild (IW) (8916 / 2215 / 1145 / 241), and PD-BioStamp (BI) (1791 / 47 / 87 / 22). Due to the nature of the disease, the Parkinson's datasets exhibited a more pronounced class imbalance.

[0143] Table 1. Average Precision AvgAcc Scoring Results on the Behavioral Dataset

[0144]

[0145]

[0146]

[0147] Table 2. Macro F1 scores on the Parkinson's dataset

[0148] Note: K represents the number of expert classification submodules.

[0149] Explanation of existing technical methods 1-5 used in the experiment:

[0150] Method 1: Empirical Risk Minimization. This method combines data from all source domains to minimize the cross-entropy loss of the classification. Empirical risk minimization typically favors a larger number of classes because, without considering class balance, the model may more easily achieve low loss on the majority of classes. This can lead to the model prioritizing performance on the majority of classes while neglecting the performance on the minority classes in cases of class imbalance.

[0151] Method 2: Domain Adversarial Neural Networks (DANNs) employ a domain discriminator and use adversarial learning to align the feature distributions of the source and target domains. The core idea is to achieve adaptation by minimizing the domain disparity between the source and target domains. However, when class imbalance exists, domain adaptation may not be effective in reducing the differences between different classes in the target domain because the model is more likely to be influenced by the class distribution in the source domain, ignoring the class distribution in the target domain.

[0152] Method 3: Maximizing Classification Module Difference. This method finds two classification modules that maximize the difference between target samples and then generates features that minimize this difference. Maximizing classification module difference typically focuses more on the dominant class, while exhibiting relatively lower discrimination performance for the minority classes. This can lead to insufficient model accuracy for minority classes in imbalanced datasets.

[0153] Method 4: Maximizing the deterministic nature of the binary classification module. Building upon the binary classification module method, this method replaces the L1 distance with deterministic differences between classification modules for calculating differences. This modification increases distinguishability, but simultaneously exacerbates discrimination against minority classes when dealing with class imbalances, as minority classes are more likely to be discarded.

[0154] Method 5: Selective Entropy Optimization. This method selectively optimizes the target domain instance by minimizing the prediction entropy (increasing confidence) for target instances where predictions are highly consistent across multiple image augmentations, and maximizing the prediction entropy (decreasing confidence) for highly inconsistent target instances. Essentially, Selective Entropy Optimization still relies on the pseudo-labels of the target domain provided by the augmentation committee multiple times, and the committee itself becomes biased as the degree of bias increases.

[0155] Based on the experimental results above, it is evident that these methods perform poorly when handling source and target domains with varying distributions, even worse than empirical risk minimization. In contrast, the method presented in this application achieves superior results in most tasks, particularly on highly imbalanced tremor datasets.

[0156] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0157] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0158] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can be, for example, including but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0159] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A domain-adaptive method for imbalanced time series based on multi-expert consensus entropy, the method comprising: Step S1: Obtain time series data of the source domain and the target domain respectively. The time series data is collected from human activity behavior by sensors on the wearable device. The time series data of the source domain is labeled, while the time series data of the target domain is unlabeled. Step S2 involves applying various types of random augmentation to the time series data in the source / target domain; Step S3: Extract features from the enhanced source / target time series data respectively; Step S4: The features extracted from the time series data of the source domain are input into multiple expert classification sub-modules to obtain different prediction outputs. The corresponding cross-entropy is calculated based on the prediction output of each expert classification sub-module and the label of the time series data of the source domain, and the multiple cross-entropies are added together. Step S5 involves inputting the features extracted from the time series data of the target domain into multiple expert classification submodules to obtain different prediction outputs, and calculating the consistency entropy of the different prediction outputs based on the obtained multiple prediction outputs. Step S6: Adjust the parameters of multiple expert classification submodules in reverse by summing the cross-entropy of the source domain and the uniform entropy of the target domain. Repeat steps S4, S5, and S6 to minimize the sum of the cross-entropy and the uniform entropy.

2. The method according to claim 1, wherein, The sum of the cross-entropy and the uniform entropy is expressed as: in, For the overall training function, Represents uniform entropy, This represents the generalized Kronecker function, used to control the sign of the uniform entropy. and y represents the data in the source and destination domains, respectively. s Here, represents the label of the source domain, Aug() is the enhancement function, K represents the number of expert classification submodules, and i represents the i-th expert classification submodule among the K expert classification submodules. Cross-entropy representing different expert classification submodules.

3. The method according to claim 1, wherein, The various types of random enhancements include no enhancement, time-domain enhancement, frequency-domain enhancement, and dynamic weighted enhancement in both the time and frequency domains.

4. The method according to claim 1, wherein, The plurality of expert classification submodules include at least one source distribution expert classification submodule and one balanced distribution expert classification submodule.

5. The method according to claim 4, wherein, The multiple expert classification submodules also include a random distribution expert submodule.

6. The method according to claim 2, wherein, The expert classification submodule makes different adjustments to the decision boundary, as follows: in, Indicates the first One expert classification submodule, Indicates to Logit adjustments were performed.

7. A storage medium storing a computer program that, when executed by a processor, can be used to implement the method of any one of claims 1-6.

8. An electronic device comprising a processor and a memory, the memory storing a computer program that, when executed by the processor, is capable of implementing the method of any one of claims 1-6.

9. A quasi-imbalanced time series domain adaptive system based on multi-expert consensus entropy, the system comprising: The data acquisition module is used to acquire time-series data of the source domain and the target domain respectively. The time-series data is collected from human activity behavior by sensors on the wearable device. The random augmentation module is used to perform various types of random augmentation on time series data in the source / target domain; The feature extraction module is used to extract features from the enhanced source / target time series data, respectively. The multi-expert committee module is used for: Features extracted from the time series data of the source domain are input into multiple expert classification sub-modules to obtain different prediction outputs. The corresponding cross-entropy is calculated based on the prediction output of each expert classification sub-module and the label of the source domain data, and the multiple cross-entropies are added together. Features extracted from time-series data of the target domain are input into multiple expert classification submodules to obtain different prediction outputs. The consistency entropy of the different prediction outputs is calculated based on the multiple prediction outputs obtained. The parameters of multiple expert classification submodules are adjusted in reverse by summing the cross-entropy of the source domain and the uniform entropy of the target domain. Repeat the above steps to minimize the sum of the cross entropy and the uniform entropy.

Images