Adversarial Domain Adaptive Human Behavior Recognition Method and System for IMU Data

Abstract

This invention discloses an adversarial domain-adaptive human behavior recognition method and system applicable to IMU data. The method includes: acquiring a training dataset; filtering user-dimensional and device-dimensional source domain data based on the target user's specific device; extracting features based on sensor characteristics; using an adversarial training module to simultaneously perform multi-dimensional domain-invariant feature extraction training and task classification training while ensuring dimensional differences, and validating and testing the model's performance; and performing human feature recognition and classification on the target domain data. This invention utilizes the correlation between source domain devices and target domain devices in the user and device dimensions to perform domain adaptation for newly added specific devices, thereby meeting the user's needs for human behavior recognition in the context of new devices.

Description

Technical Field

[0001] This application relates to the field of transfer learning, and in particular to an adversarial domain-adaptive human behavior recognition method and system suitable for IMU data. This domain-adaptive human behavior recognition method considers the domain transfer problem involving a new device with multiple source domains. The aim is to fully utilize information related to both the user and device dimensions to train a high-performing human behavior recognition model for the user's new device, thereby meeting the user's needs for human behavior recognition in scenarios with few samples when using the new device. Background Technology

[0002] With the rapid development of AIoT (Artificial Intelligence of Things) technologies, the application scale of various sensors is constantly expanding, and mobile terminal devices are becoming increasingly widespread. Human Activity Recognition (HAR) is a common application on mobile devices, typically using data from the inertial measurement unit (IMU) within the device for processing and analysis to perform human activity recognition tasks. However, using deep learning for human activity recognition requires a large amount of high-quality labeled data. Labeling large amounts of sensor data is extremely time-consuming and labor-intensive, making it challenging to perform human activity recognition tasks using small sample device data.

[0003] To address the above issues, some scholars have proposed using Domain Adaptation (DA) methods in transfer learning to solve the problem of insufficient labeled data for target devices. This type of method studies the issue of how to label source domain data with target domain data when labeled source domain data and unlabeled target domain data share the same features and categories but have different feature distributions.

[0004] Methods for human behavior recognition adapted to mobile device data domains have not been fully explored. Some works suggest that using existing historical device data on a user can be helpful for new devices. This work utilizes historical device data on a user and extracts uncertain features using a CNN-based feature extractor to complete the transfer between different devices on the same user (Akbari A, Jafari R. Transferring activity recognition models for new wearable sensors with deep generative domain adaptation[C] / / Proceedings of the 18th International Conference on Information Processing in Sensor Networks.2019:85-96.). Other works enhance the feature extraction of all devices on each source domain user, taking the user as the source domain unit, and performing feature extraction and modeling of sensor devices at different locations on the user's body. Domain adaptive transfer is then performed between users to train classification models for new, unknown user devices (Kang H, Huang Q, Zhang Q. Augmented Adversarial Learning for Human Activity Recognition with PartialSensor Sets[J].Proceedings of the ACM on Interactive, Mobile, Wearable). and Ubiquitous Technologies, 2022, 6(3): 1-30.). The Codats method uses data from other users' devices as source domain data and introduces the proportion of the user's own action categories as a weak supervision term (Wilson G, Doppa JR, Cook D J. Multi-source deep domain adaptation with weak supervision for time-series sensor data[C] / / Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 2020: 1768-1778.) to address the above challenges.However, few existing technologies simultaneously utilize historical data from other devices on a user and data from the same devices of other users, because the relevance between these two dimensions of source domain data and the target domain is heterogeneous. Ignoring this heterogeneity and using domain-adaptive methods can lead to negative transfer, meaning that the performance of transfer learning models deteriorates as more diverse source domain data is added. Therefore, finding ways to simultaneously and fully utilize both user-dimensional and device-dimensional information to aid in human behavior recognition for specific mobile devices is of significant research importance. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide an adversarial domain-adaptive human behavior recognition method suitable for IMU data.

[0006] To achieve the above technical objectives, the present invention adopts the following solution:

[0007] Adversarial domain-adaptive human behavior recognition methods applicable to IMU data include:

[0008] IMU data from each mobile device is acquired. The newly added devices of a specific user are taken as the target domain, and the different devices of the specific user and the same devices of different users are taken as the source domain. The acquired data samples are divided to construct training sample data. Among them, the data of different devices of a specific user are defined as user-dimensional source domain data, and the data of the same devices of different users are defined as device-dimensional source domain data.

[0009] Adversarial training is performed on source domain data (both device and user dimensions) against target domain data, including:

[0010] Feature extraction and action classification and recognition are performed on the training sample data; an intra-dimensional confusion module consisting of a gradient inversion layer and an intra-dimensional classifier is established to conduct adversarial training on user dimension source domain data and device dimension source domain data against target domain data respectively; a dimensional classifier is established to perform dimensional discrimination and is trained based on the overall optimization objective;

[0011] The target domain data to be classified is input into the trained model to perform human behavior recognition.

[0012] As a preferred implementation, the IMU data of each mobile device is preprocessed and then divided into samples, including:

[0013] Using the device with the lowest sampling rate as a benchmark, all collected data are downsampled to the same frequency.

[0014] As a preferred implementation, for the collected data, when the time series data is univariate, the time series data consists of several ordered sensor recording data; when the time series data is multivariate, the time series data consists of several univariate time series, and each univariate time series data contains several ordered sensor recording data.

[0015] In one preferred embodiment, a feature extractor is used to extract features from the training sample data, and an action classifier is used to classify and recognize the extracted features.

[0016] The intra-dimensional obfuscation module includes a device classifier that performs adversarial training on the user dimension source domain and target domain, and a user classifier that performs adversarial training on the device dimension source domain and target domain.

[0017] In a preferred embodiment, when the classifiers of different dimensions in the intra-dimensional confusion module require the same gradient inversion weights, they share a common gradient inversion layer; otherwise, they each use an independent gradient inversion layer.

[0018] As a preferred implementation, the loss of each classifier is calculated based on the cross-entropy loss function.

[0019] In a preferred embodiment, the user dimension source domain data is used to train the feature extractor, action classifier, device classifier, and dimension classifier.

[0020] The device dimensional source domain data is used to train the feature extractor, action classifier, user classifier, and dimensional classifier.

[0021] The target domain data is used to train the feature extractor, device classifier, user classifier, and dimension classifier.

[0022] As a preferred implementation, the objective formula for the dimensional classifier is as follows:

[0023]

[0024] Where, θ g The parameter g represents the dimension classifier. A g represents the true dimension label of the user dimension source domain data. B g represents the true dimension label of the source domain data in the device dimension. T This represents the true dimensional labels of the training data in the target domain. This represents the classification loss of the source domain data in the user dimension participating in the three-dimensional classification. This represents the classification loss of the source domain data from the device dimension participating in the three-dimensional classification. This represents the classification loss for the target domain training data participating in the three-dimensional classification. Source Domain Device Ai The marginal distribution that the samples follow; Source Domain Device B i The marginal distribution it follows; The training data T in the target domain follows a marginal distribution.

[0025] As a preferred implementation method, the overall optimization objective of adversarial training is as follows:

[0026]

[0027] In the formula, θ f and θ c These represent the parameters of the feature extractor and the action classifier, respectively. The classification loss of the action classifier. This represents the actual device label of the source domain data in the user dimension. This indicates the use of user-dimensional source domain data A. i The loss is used to classify m+1 device types for a specific user based on a sample, d T This represents the real device label of the training data T in the target domain. This indicates that the target domain training data is used to perform a classification loss for m+1 device types on a specific user. This represents the actual user tags in source domain data B at the device dimension. This indicates the use of device-level source domain data B. i Using samples to classify the n+1 users belonging to a specific device, u T This represents the real user labels in the target domain training data. This indicates that the target domain training data is used to perform classification loss for n+1 users on a specific device.

[0028] Another object of the present invention is to provide an adversarial domain-adaptive human behavior recognition system suitable for IMU data, comprising:

[0029] The data processing module acquires IMU data from various mobile devices. Taking a newly added device of a specific user as the target domain and different devices of the specific user and the same devices of different users as the source domain, the acquired data samples are divided to construct training sample data. Among them, the data of different devices of a specific user is defined as user-dimensional source domain data, and the data of the same devices of different users is defined as device-dimensional source domain data.

[0030] The adversarial training module includes a feature extractor, an action classifier, an intra-dimensional obfuscation module, and a dimensional classifier; the intra-dimensional obfuscation module consists of a gradient inversion layer, a user classifier, and a device classifier.

[0031] The feature extractor extracts features from the training sample data and then inputs them into the action classifier, user classifier, device classifier, and dimension classifier according to the data dimensions for training.

[0032] The human behavior recognition module uses a trained classifier to classify the target domain data and outputs the human action recognition classification results.

[0033] This invention provides an adversarial domain-adaptive human behavior recognition method and system suitable for IMU data. By fully utilizing the correlation between source domain devices and target domain devices in the user dimension and device dimension, it performs domain adaptation for users adding specific devices. While preserving dimensional differences, it minimizes inter-domain differences of samples in the same dimension. Through adversarial training, it trains a more suitable model for the user's new devices to meet the user's needs for human behavior recognition in the context of adding devices. Attached Figure Description

[0034] Figure 1 An adversarial domain-adaptive human behavior recognition framework for IMU data.

[0035] Figure 2 This is a diagram illustrating the dimensional division.

[0036] Figure 3 Flowchart for training source domain data A for the user dimension.

[0037] Figure 4 Flowchart for training source domain data B at the device dimension.

[0038] Figure 5 Training flowchart for target domain training data T.

[0039] Figure 6 The flowchart for the target domain human behavior recognition module. Detailed Implementation

[0040] This invention aims to design a domain-adaptive human behavior recognition method and system suitable for small sample sizes. By fully utilizing the correlation between source domain data and target domain data in two dimensions—user dimension and device dimension—a domain-adaptive method is used to comprehensively extract domain-invariant features that differ between the two dimensions, effectively providing an accurate human behavior recognition model for target domain devices.

[0041] To achieve this objective, the specific technical solution adopted by the present invention is as follows:

[0042] Adversarial Domain Adaptive Human Behavior Recognition System for IMU Data (see) Figure 1 ),include:

[0043] The system comprises three modules: a data processing module, an adversarial training module, and a target domain human behavior recognition module. The data processing module preprocesses various types of collected IMU data, aligning the data from different devices for model training and partitioning dimensions based on the heterogeneity between the source and target domains. The adversarial training module leverages the heterogeneous correlations between multi-dimensional source and target domain data, treating source domain data of different dimensions differently to train feature extractors and classifiers capable of extracting cross-dimensional domain-invariant features. The target domain recognition module utilizes the model trained in the adversarial training module to provide accurate human behavior recognition results for unlabeled target domain data.

[0044] The operational steps of each module in the adversarial domain-adaptive human behavior recognition method and system framework of the present invention are as follows:

[0045] (1) The data processing module preprocesses all collected mobile device IMU data. Then, based on specific user and device IDs, it divides and initially filters the target domain and source domain sets. In addition, based on the heterogeneous correlation between the target domain and the source domain, the source domain data set is further divided into two dimensions: user and device, and then training set, validation set and test set are constructed.

[0046] (2) The adversarial training module uses labeled source domain data and unlabeled target domain training data for adversarial training. Given a time series sample, the module first extracts time series features from the original IMU data to obtain the extracted features. Then, based on the dimension to which the sample belongs, the corresponding data stream is selected for adversarial training, so that the feature extractor can extract domain-invariant features with good action recognition capabilities in each dimension while maintaining dimensional differences.

[0047] (3) The target domain human behavior recognition module performs human action recognition and classification on the data of the target domain device. Given the target domain device data, the feature extractor extracts cross-dimensional domain invariant features as input to the classifier, and the classifier outputs the corresponding human action recognition and classification result for the time series sample.

[0048] The specific implementation steps of the above data processing module include:

[0049] 1) Data preprocessing, including:

[0050] The data we used included 3D accelerometer and 3D gyroscope data. Since data collection involved different devices, we needed to preprocess the collected data. We downsampled the collected data to the same frequency, using the device with the lowest sampling rate as the benchmark. Considering the need to run on mobile devices and to ensure the validity of the sampled data, we set the downsampling frequency to 20Hz and divided the data into sliding windows with a window length of 120. In actual use, this can be adjusted according to the specific device.

[0051] In this application, we use X to represent time series data. We consider both univariate and multivariate cases. In the univariate case, the time series data X = {X1, X2, ..., X...} H The time series data X = {X_t} consists of data recorded by H ordered sensors; in the case of multiple variables, the time series data X = {X_t} 1 ,X 2 ,…,X K The dataset contains K univariate time series data points, each containing H ordered sensor data points. These multivariate feature data could be accelerometer x, y, and z axis values, with each variable representing actual sensor data. Simultaneously, the data samples are labeled, recording the action classification label Y = {Y1, Y2, ..., Y...}. L}, where L is the number of human action categories; the user labels that generated this data sample. Where n+1 represents the total number of users in the experiment; device (or location) label Where m+1 represents the total number of equipment types (locations) involved in the experiment.

[0052] 2) Determining the target domain devices and source domain devices, including:

[0053] In our scenario, the data we collect can be categorized by the devices that generated it. The specific ID of each device can be determined by the m+1 types of devices involved and the n+1 users. Our target domain is the newly added devices of a specific user; therefore, we can specify a specific user ID. Specific device number Determine our target domain The samples collected from the target domain follow a distribution. This also causes other known device data samples to be classified as native source domain data S.

[0054] 3) Source domain data filtering, including:

[0055] We initially screened the source domain data based on the relationship between the source and target domain data. In our problem scenario, the relationship between target domain devices and source domain data can be categorized into three types: different devices of the same user, the same devices of different users, and different devices of different users. Based on the migration assessment, different devices of different users show weak migration to the target domain and have significant heterogeneous influencing factors; therefore, we removed them from our source domain data set. Obtain the filtered source domain data set S′.

[0056] 4) Source domain dimension partitioning, (see...) Figure 2 This includes:

[0057] For the existing source domain data, we further segment it dimensionally. A portion of the source domain data comes from different devices belonging to the same users as the target domain. This type of data shares the same user attributes as the target domain data, but differs in device sampling rate and deployment location. We then define the users in the dataset as... Yes, but the device number is not... Data The selected data consists of m different source domain devices belonging to a specific user, which serve as the user-dimensional source domain data A. User-level source domain device A i The samples collected in the process, Source Domain Device A i The marginal distribution that the samples follow;

[0058]

[0059] Similarly, another portion of the source domain data in the source domain set comes from devices with the same device ID as the target domain data but different user IDs. This type of data has the same device attributes as the target domain data, but there are differences in user-dimensional aspects such as user natural attributes and behavioral habits. We will define the data in the set with device IDs as... But the user number is not... Data The selected data consists of n devices of the same type belonging to different users, serving as the source domain data B for the device dimension. It is device dimension source domain device B i The samples collected in the process, Source Domain Device B i The marginal distribution it follows.

[0060]

[0061] 5) Construction of training, validation, and test sets, including:

[0062] a. Training Set: The training set data consists of three categories: user-dimensional source domain data A, device-dimensional source domain data B, and target domain training data. The action classification labels in the target domain training data are inaccessible during the training phase. It is the marginal distribution of the training data T in the target domain.

[0063] b. Both the test set and validation set consist of target domain data and have action classification labels to verify the classification performance of the test model. It is the marginal distribution of the target domain data.

[0064] The specific implementation steps of the above adversarial training module include:

[0065] 1) Feature processing, including

[0066] We use a feature extractor to extract features from the training sample data. This extractor can extract high-dimensional temporal features. This is a replaceable module, and this application is not limited to any specific feature extraction method; it is applicable to any effective temporal feature extraction method. The main function of the temporal feature extractor is to extract features from the temporal data, which serve as input to the downstream multi-task adversarial module. For a temporal data sample X, F(x) represents the high-dimensional feature extracted by the feature extractor F. This feature will be used as input to the downstream adversarial module during training and to the classifier during testing.

[0067] 2) Action classification and recognition, including

[0068] Action classification and recognition is trained using source domain data A and B (data containing human action classification labels). The aim is to train a better classifier through feature extractors and classifiers. Action classifier C is used to perform the action recognition task. Action classifier C takes high-dimensional features F(x) extracted from feature extractor F as its input and obtains the final predicted action category C(F(x)), such as walking, running, standing, cycling, climbing stairs, etc. The true label of the sample is y. Therefore, we can calculate the classification loss of the action classifier. In this application, we compute the classification loss of the action classifier using cross-entropy loss.

[0069] 3) Intra-dimensional obfuscation, including

[0070] For data within a single dimension, the user dimension A∪T consists of source domain user dimension data A and target domain training data T; the device dimension B∪T consists of source domain device dimension data B and target domain training data T. We designed an intra-dimensional obfuscation module for intra-dimensional domain invariant feature extraction. This module consists of a gradient inversion layer and an intra-dimensional classifier, which improves the classifier's performance while making the features extracted by the feature extractor more difficult to distinguish within that dimension, thus achieving stronger dimensionality characteristics in the features. User dimension obfuscation is trained using user dimension data, and device dimension obfuscation is trained using device dimension data. This includes:

[0071] a. Gradient reversal layer: This layer allows the forward propagation to be consistent with normal network operations, but the gradient is reversed during the backward propagation phase. This allows two modules with opposite training objectives to be trained simultaneously in one propagation process.

[0072]

[0073] In equation (3), It can be represented by different forward and backward propagation behaviors, where λ is a specified constant and I is the identity matrix.

[0074] When the gradient inversion weights of classifiers of different dimensions are consistent, they can share a single gradient inversion layer; otherwise, each classifier should use its own independent gradient inversion layer.

[0075] b. User-dimensional intra-dimensional obfuscation: The target domain training data T and the user-dimensional source domain data A are used as training data. The goal is to enable the device discriminator to better identify the true device d of the sample within the same user, but to blur the differences between devices within the same user in the features extracted by the feature extractor, thereby extracting the intra-dimensional invariant features of the user dimension.

[0076]

[0077] In equation (4), θ d The parameters representing the device classifier D, This represents the actual device label of source domain data A in the user dimension. This indicates the use of user-dimensional source domain data A. i The loss is used to classify m+1 device types for a specific user based on a sample, d T This represents the real device label of the training data T in the target domain. This represents the loss for classifying m+1 device types for a specific user using training data T in the target domain, where the device classifier loss is... Cross-entropy loss is used for calculation.

[0078] c. Intra-dimensional obfuscation in the device dimension: The target domain training data T and the device dimension source domain data B are used as training data to update the user classifier. The goal is to enable the user discriminator to better identify the true user u of the sample, but to blur the differences between users in the features extracted by the feature extractor, thereby extracting the intra-dimensional invariant features of the device dimension.

[0079]

[0080] In equation (5), θ u The parameters representing the user classifier U, This represents the actual user tags in source domain data B at the device dimension. This indicates the use of device-level source domain data B. i Using samples to classify the n+1 users belonging to a specific device, u T This represents the true user labels in the target domain training data T. This represents the loss for classifying n+1 users on a specific device using training data T in the target domain, where the user classifier loss is... Cross-entropy loss is used for calculation.

[0081] 4) Dimensional identification, including

[0082] For the target domain device training data T, user dimension source domain data A, and device dimension source domain data B, since the correlation between different dimensions and the target domain varies, we use a dimensional classifier to perform inter-dimensional classification, thereby amplifying the differences between sample dimensions. Specifically, the dimensions here are represented as user dimension source domain data A, device dimension source domain data B, and target domain data T. Therefore, the dimension label of each sample has three possible values.

[0083]

[0084] In equation (6), θ g The parameters of the dimensional classifier G, g A g represents the true dimension label of the source domain data A for the user dimension. B g represents the actual dimension label in the source domain data B of the device dimension. T This represents the true dimension label of the training data T in the target domain.

[0085] This represents the classification loss of source domain data A in the user dimension when it participates in the classification of the three dimensions. This represents the classification loss of source domain data B (device dimension) participating in the three-dimensional classification. This represents the classification loss of the target domain training data T in three dimensions. For the above three types of data, the classification loss of the dimensional classifier is... Calculated using cross-entropy loss.

[0086] 5) Overall optimization objective of multi-data-stream adversarial training

[0087]

[0088] Our original optimization objective is given by equation (7), where θ f and θ c Let F and C represent the parameters of the feature extractor and action classifier, respectively. The overall optimization objective of the adversarial training proposed in this application can be viewed as extracting suitable temporal features, given well-trained device classifier D, user classifier U, and dimensional classifier G, to achieve the best possible performance on action and dimensional classification tasks (achieving good activity recognition and inter-dimensional discrimination capabilities), and the worst possible performance on device and user classification tasks (reducing sample differences within the same dimension). Based on dimensional differences, we employ multi-datastream training to update specific modules, including:

[0089] a. User-dimensional source domain data A training stream (see...) Figure 3 )

[0090] The user dimension source domain data A participates in the training of four sub-modules: feature extractor F, action classifier C, device classifier D, and dimension classifier G.

[0091] b. Device-dimensional source domain data B training stream (see...) Figure 4 )

[0092] The source domain data B of the device dimension participates in the training of four sub-modules: feature extractor F, action classifier C, user classifier U, and dimension classifier G.

[0093] c. Target domain training data T training stream (see...) Figure 5 )

[0094] The target domain training data T participates in the training of four sub-modules: feature extractor F, device classifier D, user classifier U, and dimension classifier G.

[0095] By using gradient reversal (Equation 3) and training with different data streams, we can update all modules instead of training each module step by step. Our final adversarial optimization objective can be expressed as the following formula.

[0096]

[0097] The aforementioned target domain recognition module utilizes a trained model for human behavior recognition. Specific implementation steps include (see flowchart). Figure 6 ):

[0098] For the target domain device data, we process a small amount of unlabeled target domain training data T using the data processing module described in (1), and obtain all possible source domain data S. By filtering the source domain data, we obtain dimension-related source domain data S′, and further divide the source domain data to obtain the corresponding user dimension source domain data A and device dimension source domain data B. We input the training data into the data processing module described in (2), and through multi-data-stream adversarial training, we obtain the trained feature extractor F and action classifier C. We use the data processing module described in (1) to process a small amount of labeled target domain test data T. test After passing through feature extractor F and action classifier C, the final T is obtained. test The corresponding classification result C(F(x)) is used to evaluate the classification performance of the model.

Claims

1. An adversarial domain-adaptive human behavior recognition method applicable to IMU data, characterized in that, include: IMU data from each mobile device is acquired. The newly added devices of a specific user are taken as the target domain, and the different devices of the specific user and the same devices of different users are taken as the source domain. The acquired data samples are divided to construct training sample data. Among them, the data of different devices of a specific user are defined as user-dimensional source domain data, and the data of the same devices of different users are defined as device-dimensional source domain data. Adversarial training is performed on source domain data (both device and user dimensions) against target domain data, including: Feature extraction and action classification and recognition are performed on the training sample data; an intra-dimensional confusion module consisting of a gradient inversion layer and an intra-dimensional classifier is established to conduct adversarial training on user dimension source domain data and device dimension source domain data against target domain data respectively; a dimensional classifier is established to perform dimensional discrimination and is trained based on the overall optimization objective; The target domain data to be classified is input into the trained model to perform human behavior recognition.

2. The method according to claim 1, characterized in that, After preprocessing the IMU data from each mobile device, samples are divided, including: Using the device with the lowest sampling rate as a benchmark, all collected data are downsampled to the same frequency.

3. The method according to claim 1, characterized in that, For the collected data, when the time series data is univariate, the time series data consists of several ordered sensor records; when the time series data is multivariate, the time series data consists of several univariate time series, and each univariate time series data contains several ordered sensor records.

4. The method according to claim 1, characterized in that, The feature extractor is used to extract features from the training sample data, and the action classifier is used to classify and recognize the extracted features. The intra-dimensional obfuscation module includes a device classifier that performs adversarial training on the user dimension source domain and target domain, and a user classifier that performs adversarial training on the device dimension source domain and target domain.

5. The method according to claim 1 or 4, characterized in that, In the intra-dimensional confusion module, if the classifiers of different dimensions require the same gradient inversion weights, they share a common gradient inversion layer; otherwise, they each use independent gradient inversion layers.

6. The method according to claim 1, characterized in that, The loss of each classifier is calculated based on the cross-entropy loss function.

7. The method according to claim 4, characterized in that, The device dimensional source domain data is used to train the feature extractor, action classifier, user classifier, and dimensional classifier. The device-dimensional source domain data is used to train the feature extractor, action classifier, user classifier, and dimension classifier. The target domain data is used to train the feature extractor, device classifier, user classifier, and dimension classifier.

8. The method according to claim 4, characterized in that, The objective formula for the dimensional classifier is as follows: Where, θ g The parameter g represents the dimension classifier. A g represents the true dimension label of the user dimension source domain data. B g represents the true dimension label of the source domain data in the device dimension. T This represents the true dimensional labels of the training data in the target domain. This represents the classification loss of the source domain data in the user dimension participating in the three-dimensional classification. This represents the classification loss of the source domain data from the device dimension participating in the three-dimensional classification. This represents the classification loss for the target domain training data participating in the three-dimensional classification. Source Domain Device A i The marginal distribution that the samples follow; Source Domain Device B i The marginal distribution it follows; The training data T in the target domain follows a marginal distribution.

9. The method according to claim 8, characterized in that, The overall optimization objective of adversarial training is as follows: In the formula, θ f and θ c These represent the parameters of the feature extractor and the action classifier, respectively. The classification loss of the action classifier. This represents the actual device label of the source domain data in the user dimension. This indicates the use of user-dimensional source domain data A. i The loss is used to classify m+1 device types for a specific user based on a sample, d T This represents the real device label of the training data T in the target domain. This indicates that the target domain training data is used to perform a classification loss for m+1 device types on a specific user. This represents the actual user tags in source domain data B at the device dimension. This indicates the use of device-level source domain data B. i Using samples to classify the n+1 users belonging to a specific device, u T This represents the real user labels in the target domain training data. This indicates that the target domain training data is used to perform classification loss for n+1 users on a specific device.

10. An adversarial domain-adaptive human behavior recognition system suitable for IMU data, characterized in that, include: The data processing module acquires IMU data from various mobile devices. Taking a newly added device of a specific user as the target domain and different devices of the specific user and the same devices of different users as the source domain, the acquired data samples are divided to construct training sample data. Among them, the data of different devices of a specific user is defined as user-dimensional source domain data, and the data of the same devices of different users is defined as device-dimensional source domain data. The adversarial training module includes a feature extractor, an action classifier, an intra-dimensional obfuscation module, and a dimensional classifier; the intra-dimensional obfuscation module consists of a gradient inversion layer, a user classifier, and a device classifier. The feature extractor extracts features from the training sample data and then inputs them into the action classifier, user classifier, device classifier, and dimension classifier according to the data dimensions for training. The human behavior recognition module uses a trained classifier to classify the target domain data and outputs the human action recognition classification results.

Images