Cross-user wearable activity recognition method based on group-specific concept-aware representation learning

Abstract

The application discloses a cross-user wearable activity recognition method based on group-specific concept-aware representation learning, which comprises the following steps: collecting multi-user sensor data and preprocessing; measuring the concept drift degree from the time sequence perspective and the semantic perspective respectively, fusing the multi-perspective measurement results to perform user clustering, and generating group-specific concept labels; constructing a perception model comprising an activity encoder, a user encoder and multiple classifiers, and performing supervised learning by minimizing the joint loss of activity, user and group-specific concept classification; introducing a conditional discriminator to construct a representation pair of joint distribution and marginal distribution, minimizing the conditional mutual information of activity representation and user representation under the group-specific concept through adversarial training, and obtaining a trained model; and inputting test data into the trained model for activity recognition. The application explicitly models the group-specific concept and decouples the activity and user features, eliminates the cross-user concept drift, and significantly improves the activity recognition generalization ability of the model on new users.

Description

Technical Field

[0001] This invention belongs to the field of wearable human activity recognition, specifically relating to a cross-user wearable activity recognition method based on group-specific concept perception representation learning. Background Technology

[0002] With the rapid development of wearable sensing technology and the increasing prevalence of smart devices, Wearable Activity Recognition (WHAR) has become one of the key technologies in ubiquitous computing. WHAR aims to identify user activities using time-series data collected by wearable sensors, showing broad application prospects in areas such as health monitoring, fitness management, human-computer interaction, and smart homes. Wearable activity recognition faces the critical challenge of cross-user generalization: individual differences among users may lead to distribution shifts in activity data between different users, resulting in a significant decrease in the activity recognition performance of a trained model on new users.

[0003] Domain generalization (DG) methods can be used to generalize activity recognition models across users. DG treats each user's data as a domain, aiming to learn a generalizable model from data from multiple training users, thereby achieving good activity recognition performance on test data from new users. Existing domain generalization methods can be summarized into three main categories: data manipulation-based methods improve generalization ability through data augmentation, representation learning-based methods improve generalization ability by imposing domain-invariant constraints on activity labels, and learning strategy-based methods utilize strategies such as ensemble learning, meta-learning, and self-supervised learning to promote model generalization.

[0004] Representation learning-based methods are currently the most widely used approach. These methods eliminate individual differences by learning representations that are invariant across all users, thus achieving cross-user generalization. However, distribution shifts between activity data of different users can be categorized into two types: changes in the distribution of input sensor data (covariate shift) and changes in the underlying mapping function from sensor data to activity categories (concept shift). The aforementioned representation learning-based methods are only applicable to cases where covariate shift occurs, not concept shift. Under concept shift, the discrimination rules for activity categories change, i.e., the definitions of various activities change. Existing methods force all users to learn a uniform, invariant representation, which prevents the model from capturing changing class definitions, thereby limiting the model's ability to identify activities of different users.

[0005] In the WHAR scenario, concept shift may manifest as a systematic change between different user groups, rather than differences between individual users. For example, young people generally exhibit greater activity intensity than older people; therefore, the definitions and classification boundaries of activities like "walking" and "jogging" for young users show greater activity intensity than for older users, creating similarity in the discrimination rules between different users within these two groups, as well as differences in the discrimination rules between the two groups. Furthermore, user group structures may differ across activity categories. For instance, young people and older people may show significant differences in the dynamic activity of "walking," but similarities in the static activity of "lying down." To detect group-level concept shift, the degree of concept shift between pairs of users can be measured, and then clustering can be performed to obtain the user group structure. However, existing concept shift metrics can only measure the overall degree of concept shift between different domains, not the degree of concept shift within each class, and therefore cannot be used to detect user group structures that differ across different activities.

[0006] To capture class definitions that vary under concept shifts, some recent work has achieved generalization by learning to distinguish data from different domains and performing domain-aware representation learning. However, while capturing class definitions that vary under group-level concept shifts allows the model to learn the differences between different user groups, it cannot remove individualized features of users within a group. Individualized features are only applicable to a single user and are not transferable to other users, which may affect the model's ability to generalize to new users. Summary of the Invention

[0007] The technical problem this invention aims to solve is how to effectively detect and model group-level concept shifts between users in cross-user activity recognition tasks, thereby learning representations with discriminative and generalization capabilities. To address this problem, this invention provides a cross-user wearable activity recognition method based on group-specific concept-aware representation learning. This method uses temporal and semantic perspectives to measure and cluster concept shifts to obtain group-specific concept labels. Based on user classification, activity classification, and group-specific concept labels, a group-specific concept-aware model is constructed to predict activity categories, users, and group-specific concepts. The group-specific concept-aware model is updated using adversarial training that minimizes conditional mutual information, thus enabling activity recognition representation learning and effectively solving the concept shift problem in cross-user activity recognition.

[0008] To achieve the above-mentioned objectives, this invention provides a cross-user wearable activity recognition method based on group-specific concept-aware representation learning, comprising the following steps: Time-series data from wearable sensors and their activity tags were collected from multiple users, and after preprocessing, a dataset with user tags and activity categories was obtained. Based on the dataset, for each pair of users and each activity category, the degree of concept shift between users is measured from both temporal and semantic perspectives. The results of multi-perspective measurement are fused to perform user clustering and generate group-specific concept labels for each data sample. A group-specific concept-aware model is constructed to extract activity representations and user representations from a dataset as input. Then, an activity classifier and a group-specific concept detector are used to map the activity representations to activity category prediction probabilities and group-specific concept prediction probabilities, respectively. At the same time, a user classifier is used to map the user representations to user prediction probabilities. During training, the parameters of the group-specific concept-aware model are updated by minimizing the joint loss that includes activity classification, user classification, and group-specific concept detection. At the same time, a conditional discriminator is introduced, and the parameters of the conditional discriminator and various encoders are updated alternately through adversarial training to obtain the trained group-specific concept-aware model. When applied, the wearable sensor data of the test user is used as input, and the activity category prediction is obtained through the trained group-specific concept perception model to realize the activity result recognition.

[0009] Preferably, the dataset obtained after preprocessing includes: Outlier processing and normalization are performed on the time series data of wearable sensors collected from multiple users to obtain the initial time series data; The initial time series data is divided by a sliding window and labeled with corresponding activity tags to obtain a dataset with user tags and activity categories; where each user's time series data corresponds to a domain.

[0010] Preferably, for each pair of users and each activity category, the degree of conceptual shift between users is measured from both a temporal and semantic perspective, including: Construct an activity center sample based on users and activity categories; Based on the activity center samples, the degree of conceptual shift between users in each activity category is measured from a temporal perspective. The dynamic time warping method is used to calculate the inconsistency distance and the separability distance between users of the same category and the different categories, respectively, to obtain the same category distance matrix and the different category distance matrix. Simultaneously, the semantic shift between users in each activity category is measured from a semantic perspective. Text prompts are generated using the physiological attributes of each user, and activity representations are obtained through a text encoder. The semantic distance between each pair of users is calculated to obtain a semantic distance matrix.

[0011] More preferably, based on each pair of users and each activity, an activity center sample is constructed using a dynamic time warping centroid averaging method, specifically including: Using the entire sample set of a specific user and a specific activity category as input, calculate the point-by-point mean of all samples on the original time axis as the initial center sequence; The following steps are executed iteratively until convergence: Align each sample with the current center sequence using the dynamic time warping method to obtain the correspondence between each time point in the sample and each time point in the center sequence; Calculate the average value of all sample points mapped to the same position in the center sequence based on the correspondence, update the value at that position, and obtain a new center sequence; The converged center sequence is used as the active center sample.

[0012] Preferably, the process of fusing multi-perspective measurement results to perform user clustering and generate group-specific concept labels for each data sample includes: Under each activity category, the similarity distance matrix, dissimilarity distance matrix, and semantic distance matrix are converted into the corresponding similarity matrix; For each activity category, the three similarity matrices are fused using a similarity network to obtain a fused similarity matrix. Then, spectral clustering is performed on the fused similarity matrix to automatically determine the number of user groups and obtain the user groups under that activity category. Users across all activity categories are grouped and uniformly numbered, and group-specific concept labels are generated for each data sample.

[0013] More preferably, the step of converting the similarity distance matrix, the dissimilarity distance matrix, and the semantic distance matrix into corresponding similarity matrices includes: The similarity matrix, the dissimilarity matrix, and the semantic distance matrix are normalized respectively, and then mapped to three similarity matrices using a Gaussian kernel. The similarity matrix corresponding to the heterogeneous distance matrix also needs to be symmetricized and inverted.

[0014] Since the heterogeneous distance matrix is ​​an asymmetric matrix, and the larger the element value, the smaller the concept offset, the heterogeneous distance matrix is ​​first symmetricized and then inverted so that smaller element values ​​are mapped to higher similarity.

[0015] Preferably, before constructing a group-specific concept-aware model, the dataset needs to be divided into batches with a fixed batch size.

[0016] Preferably, the active encoder and the user encoder are composed of multiple convolutional neural networks. Each convolutional neural network includes a convolutional layer, a batch normalization layer and an activation function. The output layer of the encoder is an average pooling layer, and the convolutional neural networks do not share parameters. The activity encoder specifically takes activity sensor data from the dataset as input, and performs one-dimensional convolution in the time dimension through convolutional layers, batch normalization layers, and activation functions to extract dynamic information of activity categories that change over time; then it obtains the activity representation through an average pooling layer. The user encoder takes sensor data from the dataset as input and performs one-dimensional convolution in the time dimension through convolutional layers, batch normalization layers, and activation functions to extract dynamic information of the user that changes over time; then it obtains the user representation through an average pooling layer.

[0017] Preferably, the activity classifier, group-specific concept detector, and user classifier all include fully connected layers and activation functions; The activity classifier takes the activity representation as input, outputs the activity category prediction value through a fully connected layer, and then maps it to the activity category prediction probability through an activation function. The group-specific concept detector takes activity representation as input, outputs group-specific concept prediction values ​​through a fully connected layer, and then maps them to group-specific concept prediction probabilities through an activation function. The user classifier takes user representations as input, outputs user prediction values ​​through a fully connected layer, and then maps them to user prediction probabilities through an activation function.

[0018] Preferably, the cross-entropy loss is calculated based on the predicted probability of the activity category output by the activity classifier and the actual activity label to obtain the activity classification loss; The user classification loss is obtained by calculating the cross-entropy loss based on the user predicted probability output by the user classifier and the user label. The cross-entropy loss is calculated based on the group-specific concept prediction probability output by the group-specific concept detector and the generated group-specific concept label to obtain the group-specific concept loss.

[0019] More preferably, the joint loss is a weighted sum of the activity classification loss, user classification loss, and group-specific concept loss, expressed as: ,in, Classify the losses for the activity. Categorize losses for users, For group-specific concept loss, Weighted hyperparameters for group-specific concept loss.

[0020] Preferably, a conditional discriminator is introduced, and its parameters are updated through adversarial training, including: Constructing representation pairs for joint distribution and marginal distribution specifically includes: under the same set of specific concepts, concatenating the activity representation and user representation of the same data sample to obtain representation pairs from the joint distribution; simultaneously, under the same set of specific concepts, concatenating the activity representation and randomly shuffled user representation to obtain representation pairs from the marginal distribution, and labeling them with binary classification labels. The one-hot encoding of the pairs and their corresponding group-specific concepts is input into the conditional discriminator to obtain the discrimination probability; The discrimination loss is calculated based on the discrimination probability and the binary classification label, and the parameters of the conditional discriminator are updated with the goal of minimizing the discrimination loss.

[0021] More preferably, a conditional discriminator is introduced to update various encoder parameters through adversarial training, including: With the discriminator parameters fixed, the discriminant loss is recalculated, and the parameters of the active encoder and user encoder are updated with the goal of maximizing the recalculated discriminant loss.

[0022] This step enables the encoder to "trick" the discriminator, pushing the joint distribution to align with the marginal distribution, thereby minimizing conditional mutual information and completing the parameter updates for the active encoder and the user encoder.

[0023] Compared with the prior art, the beneficial effects of the present invention include at least the following: (1) This invention integrates the temporal perspective (dynamic time warping measurement based on sensor waveforms) and the semantic perspective (text encoding measurement based on physiological attributes). Through similarity network fusion and spectral clustering, it automatically estimates the number of groups and generates groups, avoiding the blindness of subjectively setting the number of groups. It no longer regards users as a single whole or completely independent individuals, but discovers user groups with similar behavioral patterns (corresponding to specific group concepts) within each activity category. The grouping results have clear semantic interpretability.

[0024] (2) In the concept offset measurement stage, the present invention introduces dynamic time warping and centroid averaging to effectively measure the differences between time series samples and aggregate time series samples of the same user and the same activity; in the representation learning stage, it adaptively eliminates the differences between individual users within the group through adversarial training, rather than simply aligning the feature distribution of all users, and can handle complex scenarios with multiple users and multiple activity categories at the same time.

[0025] (3) The present invention constructs a multi-task learning framework (activity classification + user classification + group-specific concept detection). Joint optimization enables the activity representation learned by the encoder to have both activity discriminativeness and group-specific concept discriminativeness, and the user representation to have user discriminativeness. Furthermore, through explicit modeling and quantification of concept shift, and by using adversarial training, the activity representation is made conditionally independent of the user representation under group-specific concepts, thus removing user-related interference information and retaining discriminative features related to the essence of the activity. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0027] Figure 1 This is a flowchart illustrating the cross-user wearable activity recognition method based on group-specific concept-aware representation learning provided by the present invention.

[0028] Figure 2 A flowchart for the construction and training of a group-specific concept perception model.

[0029] Figure 3 A dual-view group-level concept offset detection framework diagram provided for an embodiment.

[0030] Figure 4 This is a diagram of the group-specific concept perception representation learning framework provided in the embodiment.

[0031] Figure 5 A comparison of the activity representation distribution of the present invention with the conventional domain adversarial neural network method provided in the embodiments; wherein, Figure 5 Figure A in the diagram represents the activity representation distribution of a traditional domain adversarial neural network method. Figure 5 Figure B in the figure represents the activity representation distribution of the method of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and given in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0033] The inventive concept of this invention is as follows: Addressing the generalization bottleneck caused by cross-user concept shifts in existing technologies, the challenges of quantifying and discovering user differences, the blindness and subjectivity of user grouping, and the high coupling between activity information and user identity information, this invention provides a cross-user wearable activity recognition method based on group-specific concept perception representation learning. It obtains group-specific concept labels through concept shift measurement and clustering from temporal and semantic perspectives. A group-specific concept perception model is constructed based on user classification, activity classification, and group-specific concept labels to predict activity categories, users, and group-specific concepts. The group-specific concept perception model is updated by combining adversarial training that minimizes conditional mutual information, and the activity recognition is performed through representation learning. This effectively solves the problems of group-level concept shifts and personalized shifts in cross-user activity recognition.

[0034] like Figure 1 and Figure 2 As shown in the embodiment, a cross-user wearable activity recognition method based on group-specific concept-aware representation learning includes the following steps: S1. Collect time-series data and activity tags from wearable sensors from multiple users, and obtain a dataset with user tags and activity categories after preprocessing.

[0035] In this embodiment, wearable sensor time-series data and their activity tags are collected from multiple training users. The raw sensor sequences are preprocessed, including outlier handling and normalization. Continuous sequences are divided into samples using a sliding window, and corresponding activity tags are labeled. Each user... The data is treated as a domain and labeled with user tags. This forms the training dataset. Specifically: Outlier detection is performed on the sensor time series data, and missing and invalid values ​​are filled using interpolation. The data after outlier elimination is then normalized to the range [-1, 1], calculated using the following formula: , in, After eliminating outliers, the time step Data, These represent the maximum and minimum values ​​of the time series data, respectively. This is the normalized data.

[0036] The size of the time window is set manually based on experience. The training dataset is obtained by dividing the normalized data using a fixed-length sliding step. Labeled data of training users , among which, the The data for each user is as follows: , in, For the number of channels Sensor samples, For event tags, User tags, This represents the total number of activity categories.

[0037] S2. Based on the dataset, for each pair of users and each activity category, measure the degree of concept shift between users from both temporal and semantic perspectives. Integrate the multi-perspective measurement results to perform user clustering and generate group-specific concept labels for each data sample.

[0038] In the embodiments, such as Figure 2 As shown, for each training user and each activity category The activity center sample is obtained by using the sample of this activity on this user based on the Dynamic Time Warping Barycenter Averaging (DBA). It is used to characterize the discriminative pattern of the activity for that user.

[0039] The sample set of this user in this activity is: Using the dynamic time-warped centroid average function The activity center sample is calculated using the following formula: .

[0040] In this embodiment, based on each user and each activity, an activity center sample is constructed using a dynamic time warping centroid averaging method, specifically including: Using the entire sample set of a specific user and a specific activity category as input, calculate the point-by-point mean of all samples on the original time axis as the initial center sequence; The following steps are executed iteratively until convergence: Align each sample with the current center sequence using the dynamic time warping method to obtain the correspondence between each time point in the sample and each time point in the center sequence; Calculate the average value of all sample points mapped to the same position in the center sequence based on the correspondence, update the value at that position, and obtain a new center sequence; The converged center sequence is used as the active center sample.

[0041] Next, each activity class is measured from a temporal perspective. The concept offset between users is calculated. For each activity, a concept offset metric between each pair of users is calculated based on Dynamic Time Warping (DTW), including the distance between activity center samples of the same class across users, and the distance between activity center samples of the same class across users and activity center samples of other classes, thus obtaining two user distance matrices corresponding to the activity. and It is used to measure cross-user inconsistency within the same category and divisibility between different categories. For the same type of distance matrix, This is the distance matrix between different classes.

[0042] In each activity category Next, using the DTW function For each training user Calculate the distance of inconsistency within the same category, for each activity category. and each activity category Next, using the DTW function For each training user Calculate the distance between different classes and form distance matrices respectively. and The calculation formula is as follows: , , in, The larger the element value, the greater the cross-user difference of that type of activity, i.e., the greater the degree of concept deviation. The larger the element value, the easier it is to distinguish this type of activity from other activities across users, that is, the smaller the degree of concept deviation.

[0043] Construct activity representations. For each training user and each activity category The system uses the user's physiological attributes (age, gender, height, weight, etc.) and the activity input template to generate a text prompt, such as "An elderly person is walking." A pre-trained model's text encoder then encodes the prompt into an activity representation. .

[0044] Indicates the first A training user's physiological attribute vector, for activity category Using template functions Generate prompts and input them into the text encoder. Obtain activity representation The calculation formula is as follows: , in, Indicates a prompt template.

[0045] In the embodiments, each activity category is measured from a semantic perspective. The degree of conceptual shift between users. For each activity category. Based on cosine similarity, the activity representation distance between each pair of users is calculated, thereby obtaining the user distance matrix corresponding to the activity. This is used to measure the semantic differences in the activities of different users.

[0046] In each activity category Below, using the cosine distance function For each training user Calculate the distance between their activity representations to form an activity semantic difference matrix. The calculation formula is as follows: , in, The larger the element value, the greater the semantic difference of that type of activity, that is, the greater the degree of conceptual deviation.

[0047] Transform the user distance matrix into a user similarity matrix. For each activity category... Next, normalization and kernel function mapping are performed on the three distance matrices respectively. For the heterogeneity matrix, symmetry and inversion are further performed to obtain the corresponding three similarity matrices.

[0048] For each distance matrix Using the Min-Max function Normalization is performed, and then a Gaussian kernel function is used. The normalized distance matrix is ​​converted into a similarity matrix, and the calculation formula is as follows: , , , in, It is a median function. It is a constant term used to maintain numerical stability.

[0049] because It is an asymmetric matrix, where larger element values ​​represent smaller concept shifts. We first symmetrize it, then invert it so that smaller element values ​​are mapped to higher similarity. The calculation formula is as follows: , in, This represents the similarity matrix of the heterogeneous distance matrix after symmetrization and inversion.

[0050] Similarity network fusion and user grouping. Separately for each activity category. Next, the Similarity Network Fusion (SNF) algorithm is performed on the three similarity matrices, and the result is based on the fused similarity matrix. The spectral clustering algorithm was used to obtain user groups, and the number of user groups was [not specified]. From the similarity matrix The spectral gap algorithm (eigengap) automatically estimates that each group corresponds to a group-specific concept under that activity.

[0051] Using a similarity network fusion algorithm Obtain the fused similarity matrix The calculation formula is as follows: .

[0052] Based on the fused similarity matrix Using spectral clustering algorithm Get user groups The calculation formula is as follows: , in, It is from user tags To user group number The mapping function.

[0053] Generate group-specific concept tags : Uniformly number all group-specific concepts of activities as Label the training samples with multiple classification labels. This indicates the group-specific concept to which the sample belongs, used for subsequent representation learning. Specifically: In order categorize the activities Below The specific concept number of each group is: categorize the activities The K_1 groups of specific concepts are numbered as follows: And so on. For any sample Its group-specific concept tags The calculation formula is as follows: .

[0054] S3. Construct a group-specific concept-aware model, which takes the dataset as input, first extracts activity representations and user representations through an activity encoder and a user encoder, respectively; then, through an activity classifier and a group-specific concept detector, the activity representations are mapped to activity category prediction probabilities and group-specific concept prediction probabilities, respectively, and at the same time, the user representations are mapped to user prediction probabilities through a user classifier.

[0055] In this embodiment, the training dataset is divided into batches according to a fixed batch size, with a total number of batches of [number missing]. .

[0056] Based on the manually set batch size Divide the training dataset into batches, with a total number of batches. The calculation formula is as follows: , in, This represents the total number of samples from all training users.

[0057] Select indices sequentially from the training dataset. A batch of training samples, among which Construct a group-specific concept perception model. Specifically: The batch of training samples is input into the active encoder. With user encoder Extracting activity representations With user representation .

[0058] The active encoder and the user encoder both consist of three-layer convolutional neural networks with identical network structures and no shared parameters. The encoder input is a channel number... Sensor samples First, the input samples are fed into three convolutional layers for one-dimensional convolution operations in the time dimension to extract dynamic information that changes over time in the sensor data. The (output channel number, kernel size, stride) of the first, second, and third convolutional layers are (128, 8, 2), (256, 5, 2), and (128, 3, 1), respectively. Each convolutional layer is followed by a batch normalization layer to stabilize training and improve convergence, and a non-linear ReLU activation function. Second, the features calculated by the convolutional layers are fed into an average pooling layer of length 1, outputting a vector representation of length 128. , This serves as the output of the encoder. The calculation formulas for activity representation and user representation are as follows: .

[0059] Then, through an activity classifier and a group-specific concept detector, the activity representation is mapped to the activity category prediction probability and the group-specific concept prediction probability, respectively. At the same time, through a user classifier, the user representation is mapped to the user prediction probability.

[0060] The activity classifier, user classifier, and group-specific concept detector each consist of a single fully connected layer, with the corresponding representation vector as input. or The output dimensions are the number of activity categories. Number of training users and the number of specific concepts in a group The softmax activation function outputs the corresponding classification probability vector.

[0061] S4. During training, the parameters of the group-specific concept-aware model are updated by minimizing the joint loss that includes activity classification, user classification, and group-specific concept detection. At the same time, a conditional discriminator is introduced, and the parameters of the conditional discriminator and various encoders are updated alternately through adversarial training to obtain the trained group-specific concept-aware model.

[0062] In this embodiment, supervised learning training is performed on the constructed group-specific concept perception model, such as... Figure 4 As shown, the details are as follows: Activity representation Input to activity classifier Group-specific concept detectors In the middle, user representation Input to user classifier In this process, supervised loss is calculated based on the predicted classification probability of the output and the corresponding sample label. The relevant network parameters are updated with the goal of minimizing the supervised loss.

[0063] Constructing an activity identification loss based on cross-entropy User identification loss Group-specific concept detection loss The supervised loss is established as the joint loss, and the calculation formula is as follows: , , , , in, Weighted hyperparameters for group-specific concept loss.

[0064] Next, a conditional discriminator is introduced to distinguish activity-user representation pairs from joint and marginal distributions under group-specific concepts. For each group-specific concept... Filter out tags samples and its activity characteristics With user representation Construct from the joint distribution and marginal distribution The representation of And label them with binary classification labels (1 / 0 indicates from joint / marginal distribution). This will characterize pair- and group-specific concept labels. The input is fed into the conditional discriminator, and the discriminant loss is calculated based on the output discriminant probability and the binary classification label. The conditional discriminator parameters are updated with the goal of minimizing the discriminant loss.

[0065] The conditional discriminator consists of two fully connected layers, with the input being the representation pair. as well as Group-specific concept tags The corresponding one-hot encoding (length is) The first fully connected layer has an output dimension of 512 and uses the LeakyReLU activation function. The second fully connected layer has an output dimension of 1 and uses the Sigmoid activation function. The output is the binary classification probability.

[0066] To ensure that activity representations remain conditionally invariant under group-specific concepts, the conditional mutual information between activity representations and user representations should be reduced during representation learning. .because It is valid if and only if Alignment joint distribution With marginal distribution This can reduce the conditional mutual information. To minimize the Jenson-Shannon divergence (JSD) between the two distributions, adversarial training is performed. The discriminant loss is calculated as follows: , in, Given a binary cross-entropy function, the representation of the pair under the joint distribution. By splicing corresponding samples of The characterization of marginal distribution is obtained. By splicing the samples in this batch in sequence Compared to random shuffling under a specific group concept get.

[0067] The encoder is trained to confuse the discriminator. The discriminator parameters are fixed, and the discrimination loss is recalculated. The encoder parameters are updated with the goal of maximizing the discriminative loss. Adversarial training is used to "fool" the discriminator, pushing the joint distribution to align with the marginal distribution. This is equivalent to minimizing the conditional mutual information between representations, i.e. This allows the representation of activities to remain conditionally unchanged based on group-specific concepts.

[0068] Repeat the training process until all batches of the training dataset have participated in model training, and then iterate the training until the specified number of iterations is reached.

[0069] S5. When applied, the wearable sensor data of the test user is used as input, and the activity category prediction is obtained through the trained group-specific concept perception model to realize the activity result recognition.

[0070] To further verify the feasibility and effectiveness of the cross-user wearable activity recognition method based on group-specific concept-aware representation learning proposed in this invention, this embodiment was validated on four publicly available wearable human activity datasets. Table 1 shows the comparison results between this scheme and traditional methods on four publicly available wearable human activity datasets (UCI-HAR, WISDM, UniMiB, HARTH+). The activity classification performance metrics for two new users are accuracy and macro F1.

[0071] Table 1

[0072] Among them, the maximum mean difference method achieves generalization by reducing the maximum mean difference index of the activity representation distribution between pairs of users; the association alignment method minimizes the second-order statistic of the activity representation distribution between pairs of users; the domain adversarial neural network method achieves domain invariance by adversarially training the activity representation against the user classifier; the generative adversarial network method simulates the distribution of sensor data to generate more training data for data augmentation; the latent independence incentive method performs adversarial training on the activity representation against the user classifier and adversarial training on the user representation against the activity classifier to achieve representation untangling; and the class concept invariant learning method simultaneously achieves cross-user activity representation invariance and classification probability invariance.

[0073] Figure 5 The distribution of activity representations on the UniMiB dataset was compared between traditional domain adversarial neural network (DN) methods and the proposed method. The dimensionality of the representation distribution was reduced using t-Distributed Stochastic Neighbor Embedding (t-SNE), where each color represents an activity category, and the intensity of each color represents different group-specific concepts within that activity category. Results show that in DN methods, samples from different activity categories are mixed together, and samples within the same category are scattered and lack clear local structure. This may be because the method enforces cross-user representation invariance, thus weakening user-specific discrimination patterns under concept drift. In the proposed method, samples for each group-specific concept tend to be concentrated. This phenomenon is due to the conditional invariance constraint removing personalized patterns within the group. More importantly, different group-specific concepts within the same activity category are easily distinguishable, as are different activity categories. This indicates that the proposed method can model the changing category definitions corresponding to each activity, thereby obtaining a classifier capable of modeling the potential changes in activity discrimination rules.

[0074] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for cross-user wearable activity recognition based on group-specific concept-aware representation learning, characterized in that, Includes the following steps: Time-series data from wearable sensors and their activity tags were collected from multiple users, and after preprocessing, a dataset with user tags and activity categories was obtained. Based on the dataset, for each pair of users and each activity category, the degree of concept shift between users is measured from both temporal and semantic perspectives. The results of multi-perspective measurements are fused to perform user clustering, generating group-specific concept labels for each data sample. This includes: under each activity category, converting the similarity distance matrix, dissimilarity distance matrix, and semantic distance matrix into corresponding similarity matrices; performing similarity network fusion on the three similarity matrices under each activity category to obtain a fused similarity matrix, and then performing spectral clustering on the fused similarity matrix to automatically determine the number of user groups and obtain user groups under that activity category; uniformly numbering the user groups of all activity categories, and generating group-specific concept labels for each data sample. A group-specific concept-aware model is constructed to extract activity representations and user representations from a dataset as input. Then, an activity classifier and a group-specific concept detector are used to map the activity representations to activity category prediction probabilities and group-specific concept prediction probabilities, respectively. At the same time, a user classifier is used to map the user representations to user prediction probabilities. During training, the parameters of the group-specific concept-aware model are updated by minimizing the joint loss that includes activity classification, user classification, and group-specific concept detection. At the same time, a conditional discriminator is introduced, and the parameters of the conditional discriminator and various encoders are updated alternately through adversarial training to obtain the trained group-specific concept-aware model. When applied, the wearable sensor data of the test user is used as input, and the activity category prediction is obtained through the trained group-specific concept perception model to realize the activity result recognition.

2. The cross-user wearable activity recognition method based on group-specific concept-aware representation learning according to claim 1, characterized in that, The preprocessed dataset, which includes user-labeled data, comprises: Outlier processing and normalization are performed on the time series data of wearable sensors collected from multiple users to obtain the initial time series data; The initial time series data is divided by a sliding window and labeled with corresponding activity tags to obtain a dataset with user tags and activity categories; where each user's time series data corresponds to a domain.

3. The cross-user wearable activity recognition method based on group-specific concept-aware representation learning according to claim 2, characterized in that, For each pair of users and each activity category, the degree of conceptual shift between users is measured from both temporal and semantic perspectives, including: Construct an activity center sample based on users and activity categories; Based on the activity center samples, the degree of conceptual shift between users in each activity category is measured from a temporal perspective. The dynamic time warping method is used to calculate the inconsistency distance and the separability distance between users of the same category and the different categories, respectively, to obtain the same category distance matrix and the different category distance matrix. Simultaneously, the semantic shift between users in each activity category is measured from a semantic perspective. Text prompts are generated using the physiological attributes of each user, and activity representations are obtained through a text encoder. The semantic distance between each pair of users is calculated to obtain a semantic distance matrix.

4. The cross-user wearable activity recognition method based on group-specific concept-aware representation learning according to claim 3, characterized in that, Based on each pair of users and each activity, an activity center sample is constructed using a dynamic time warping centroid averaging method, specifically including: Using the entire sample set of a specific user and a specific activity category as input, calculate the point-by-point mean of all samples on the original time axis as the initial center sequence; The following steps are executed iteratively until convergence: Align each sample with the current center sequence using the dynamic time warping method to obtain the correspondence between each time point in the sample and each time point in the center sequence; Calculate the average value of all sample points mapped to the same position in the center sequence based on the correspondence, update the value at that position, and obtain a new center sequence; The converged center sequence is used as the active center sample.

5. The cross-user wearable activity recognition method based on group-specific concept-aware representation learning according to claim 1, characterized in that, The active encoder and the user encoder are composed of multiple layers of convolutional neural networks. Each layer of the convolutional neural network includes a convolutional layer, a batch normalization layer and an activation function. The output layer of the encoder is an average pooling layer, and the convolutional neural networks do not share parameters. The activity encoder specifically takes activity sensor data from the dataset as input, and performs one-dimensional convolution in the time dimension through convolutional layers, batch normalization layers, and activation functions to extract dynamic information of activity categories that change over time; then it obtains the activity representation through an average pooling layer. The user encoder takes sensor data from the dataset as input and performs one-dimensional convolution in the time dimension through convolutional layers, batch normalization layers, and activation functions to extract dynamic information of the user that changes over time; then it obtains the user representation through an average pooling layer.

6. The cross-user wearable activity recognition method based on group-specific concept-aware representation learning according to claim 1, characterized in that, The activity classifier, group-specific concept detector, and user classifier all include fully connected layers and activation functions; The activity classifier takes the activity representation as input, outputs the activity category prediction value through a fully connected layer, and then maps it to the activity category prediction probability through an activation function. The group-specific concept detector takes activity representation as input, outputs group-specific concept prediction values ​​through a fully connected layer, and then maps them to group-specific concept prediction probabilities through an activation function. The user classifier takes user representations as input, outputs user prediction values ​​through a fully connected layer, and then maps them to user prediction probabilities through an activation function.

7. The cross-user wearable activity recognition method based on group-specific concept-aware representation learning according to claim 1, characterized in that, The cross-entropy loss is calculated based on the predicted probability of the activity category output by the activity classifier and the actual activity label to obtain the activity classification loss. The user classification loss is obtained by calculating the cross-entropy loss based on the user predicted probability output by the user classifier and the user label. The cross-entropy loss is calculated based on the group-specific concept prediction probability output by the group-specific concept detector and the generated group-specific concept label to obtain the group-specific concept loss.

8. The cross-user wearable activity recognition method based on group-specific concept-aware representation learning according to claim 1, characterized in that, A conditional discriminator is introduced, and its parameters are updated through adversarial training, including: Constructing representation pairs for joint distribution and marginal distribution specifically includes: under the same set of specific concepts, concatenating the activity representation and user representation of the same data sample to obtain representation pairs from the joint distribution; simultaneously, under the same set of specific concepts, concatenating the activity representation and randomly shuffled user representation to obtain representation pairs from the marginal distribution, and labeling them with binary classification labels. The one-hot encoding of the pairs and their corresponding group-specific concepts is input into the conditional discriminator to obtain the discrimination probability; The discrimination loss is calculated based on the discrimination probability and the binary classification label, and the parameters of the conditional discriminator are updated with the goal of minimizing the discrimination loss.

9. The cross-user wearable activity recognition method based on group-specific concept-aware representation learning according to claim 8, characterized in that, A conditional discriminator is introduced, and various encoder parameters are updated through adversarial training, including: With the discriminator parameters fixed, the discriminant loss is recalculated, and the parameters of the active encoder and user encoder are updated with the goal of maximizing the recalculated discriminant loss.

Images