Fall behavior recognition method and system based on feature adaptive fusion domain generalization

Abstract

The application provides a behavior recognition method and system based on feature adaptive fusion domain generalization, and specifically, the algorithm includes a domain-specific representation learning module and a domain-invariant representation learning module; through learning of domain-specific features, specific characteristics of each source domain are extracted, so that more diverse feature representations are obtained; through the domain-invariant representation learning module, domain-independent features are learned, common and invariant characteristics of all domains are extracted, and the generalization performance is improved. The two parts of features are adaptively fused, so that a model with good generalization performance can be trained, thereby realizing good classification performance on an unknown target domain and completing the human behavior recognition task of the target domain.

Description

Technical Field

[0001] This invention relates to the fields of machine learning, domain generalization, and behavior recognition, and specifically to a fall behavior recognition method and system based on feature adaptive fusion domain generalization. Background Technology

[0002] Human behavior recognition is an active research topic, aiming to train machine learning models using collected behavioral data to identify human actions. Specifically, sensor-based behavior recognition utilizes collected sensor data to identify activities, offering universality and non-invasiveness, thus playing a crucial role. Behavior recognition has been widely applied in smart homes, fatigue detection, and fall detection for the elderly. In recent years, behavior recognition has demonstrated some capability in the auxiliary diagnosis and prediction of diseases such as Parkinson's disease, attention deficit hyperactivity disorder, and stroke, providing health monitoring for patients' daily lives. Therefore, accurate behavior recognition is essential in practical applications.

[0003] Many machine learning methods have been used to improve the performance of behavior recognition, such as Support Vector Machines (SVM), k-Nearest Neighbors (KNN), Random Forest (RF), and deep learning models including Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM). Despite some success, a key challenge remains the generalization ability of behavior recognition models—that is, the performance of applying a model trained on existing data to a new, unseen dataset. In practical applications, sensor signals are easily influenced by the individual characteristics of end users, such as their behavioral styles, habits, or body types. When tested on a new end user (whose activity data has never been seen in the training set), the model's performance may degrade, often due to differences in data distribution among different users.

[0004] Transfer learning and domain adaptation can be used to solve the domain transfer problem and have been applied to action recognition in recent years. Transfer learning methods first pre-train a model on the source domain dataset and then fine-tune it on new test data. Domain adaptation methods learn a domain-invariant feature space by re-weighting instances or transforming features between the training and test datasets, minimizing their distributional differences. However, both transfer learning and domain adaptation require access to data in the target domain during training, which is often impractical in real-world applications, as the ultimate goal is to train once and apply it to new scenarios. Furthermore, retraining the model on end-user devices is infeasible due to limited computing power. Domain generalization is an emerging research topic in recent years. It focuses on leveraging knowledge from several different domains to build a model that can be well generalized to any unknown domain, i.e., generalized to new, unseen test datasets. However, we cannot directly apply existing domain generalization methods to solve our problem. First, unlike existing data manipulation methods for computer vision tasks, there is a lack of quantitative evaluation of sensor data augmentation. Directly applying such augmentation to action recognition may harm model performance. Secondly, since different people have different behavioral patterns, it is necessary to retain their specific characteristics to better generalize the target domain, while existing representation learning domain generalization methods only focus on learning domain-invariant features. Summary of the Invention

[0005] To address the domain generalization problem in human behavior recognition, this invention proposes a behavior recognition method based on adaptive feature fusion domain generalization, which includes:

[0006] Step 1: Obtain behavioral datasets from multiple data domains. Each behavioral data in these datasets has a behavioral identification label and a domain label. Use one domain from these datasets as the target domain and the domains outside the target domain as the source domains. Construct a behavioral identification model that includes a behavioral classification module, a common feature extraction module, a specific feature extraction module, and a domain classification module.

[0007] Step 2: Use the common feature extraction module to extract the common features among the source domain behavioral data in this behavioral dataset;

[0008] Step 3: Train the specific feature extraction module by calculating the domain-specific loss of the source domain, and extract the domain-specific features of each source domain;

[0009] Step 4: Perform distribution adaptation on domain-specific features and calculate the adaptation loss to perform distribution adaptation on all domain-specific features;

[0010] Step 5: Input the common feature into the domain classifier, use the output of the domain classifier as weights, and weight and fuse the domain-specific features to obtain the fused feature. Input the fused feature into the behavior classification module to obtain the classification result, and combine the behavior recognition label to construct the classification loss of the classification module.

[0011] Step 6: Based on the classification loss, the adaptation loss, and the domain-specific loss, train the behavior recognition model, and input the behavior data to be tested as the target domain into the trained behavior recognition model to obtain the behavior recognition result of the behavior data to be tested.

[0012] The aforementioned behavior recognition method based on feature adaptive fusion and domain generalization, wherein...

[0013] Step 3 includes:

[0014] Step 31, the loss specific to the k-th source domain is:

[0015]

[0016] The k-th source domain has n k Data samples, f d For the classifier of this domain, f e For this shared feature extraction module, x i Let d be the i-th data sample in the current source domain. k For x i Domain tags;

[0017] Step 32: Averaging the losses across all source domains yields the domain-specific loss. Where K is the total number of source domains;

[0018] Step 4 includes:

[0019] Step 41: Obtain the adaptation loss using the following formula:

[0020]

[0021] Where i and j are the domain labels, and φ(x) is the mapping function that maps the original feature space to the regenerated Hilbert space. Indicates the training domain;

[0022] Step 5 includes:

[0023] Cross-entropy loss is used as the classification loss for this classification module:

[0024]

[0025] Where N is the number of training samples.

[0026] The aforementioned behavior recognition method based on feature adaptive fusion and domain generalization, wherein step 6 includes:

[0027]

[0028] Where λ and β are adaptive hyperparameters, obtained through gradient descent. Optimization is performed to complete the training of the behavior recognition model.

[0029] The aforementioned behavior recognition method based on feature adaptive fusion and domain generalization, wherein step 6 includes:

[0030] The features of the behavior data to be tested are extracted by all common feature extractors and then passed through the domain classifier to obtain the similarity between the target domain data and each source domain, which is used as the weight. The behavior data to be tested is extracted sequentially by all domain feature extractors to obtain multiple domain-specific features. These domain-specific features are weighted and summed using the weights, and the summation result is input into the behavior classification module to obtain the behavior recognition result.

[0031] This invention also proposes a behavior recognition system based on feature adaptive fusion and domain generalization, comprising:

[0032] Module 1 is used to acquire behavioral datasets from multiple data domains. Each behavioral data in the dataset has a behavioral identification label and a domain label. One domain in the dataset is used as the target domain, and the domains outside the target domain are used as the source domains. A behavioral identification model is constructed, which includes a behavioral classification module, a common feature extraction module, a specific feature extraction module, and a domain classification module.

[0033] Module 2 is used to extract common features among source domain behavioral data in the behavioral dataset using the common feature extraction module;

[0034] Module 3 is used to train a specific feature extraction module by calculating the domain-specific loss of the source domain, and to extract the domain-specific features of each source domain;

[0035] Module 4 is used to perform distributional adaptation of domain-specific features, learn domain-invariant features, and calculate the adaptation loss to perform distributional adaptation on all domain-specific features.

[0036] Module 5 is used to input the common features into the domain classifier, use the output of the domain classifier as weights, and weight and fuse the domain-specific features to obtain the fused features. The fused features are then input into the behavior classification module to obtain the classification result. The classification loss of the classification module is constructed by combining the behavior recognition label.

[0037] Module 6 is used to train the behavior recognition model based on the classification loss, the adaptation loss, and the domain-specific loss, and to input the behavior data to be tested as the target domain into the trained behavior recognition model to obtain the behavior recognition result of the behavior data to be tested.

[0038] The aforementioned behavior recognition system based on feature adaptive fusion and domain generalization, wherein

[0039] Module 3 includes:

[0040] The loss calculation module is used to calculate the loss specific to the k-th source domain using the following formula:

[0041]

[0042] The k-th source domain has n k Data samples, f d For the classifier of this domain, f e For this shared feature extraction module, x i Let d be the i-th data sample in the current source domain. k For x i Domain tags;

[0043] The averaging module is used to average the losses across all source domains to obtain the domain-specific loss. Where K is the total number of source domains;

[0044] Module 4 is used to obtain the adaptation loss using the following formula:

[0045]

[0046] Where i and j are the domain labels, and φ(x) is the mapping function that maps the original feature space to the regenerated Hilbert space. Indicates the training domain;

[0047] Module 5 uses cross-entropy loss as the classification loss for this classification module.

[0048]

[0049] Where N is the number of training samples.

[0050] The aforementioned behavior recognition system based on feature adaptive fusion and domain generalization, wherein module 6 is used to perform gradient descent on the following formula... Optimize the training to complete the behavior recognition model:

[0051] Where λ and β are adaptive hyperparameters.

[0052] The aforementioned behavior recognition system based on feature adaptive fusion and domain generalization includes module 6, which is used to extract features of the behavior data to be tested through a common feature extractor, and then pass them through the domain classifier to obtain the similarity between the target domain data and each source domain, which is used as weight; the behavior data to be tested is extracted sequentially through all domain feature extractors to obtain multiple domain-specific features, and the weights are used to perform a weighted summation of all these domain-specific features, and the summation result is input into the behavior classification module to obtain the behavior recognition result.

[0053] The present invention also proposes a storage medium for storing a program that executes any of the aforementioned behavior recognition methods based on feature adaptive fusion domain generalization.

[0054] The present invention also proposes a client for any of the aforementioned behavior recognition systems based on feature adaptive fusion domain generalization.

[0055] As can be seen from the above solutions, the advantages of the present invention are:

[0056] The method proposed in this invention can train a model with good generalization performance using source domain data when the target domain data is unknown, and then test it directly on the target domain data. This solves the problem in human behavior recognition where a model is trained only once and then applied to new unlabeled data, enabling rapid behavior recognition.

[0057] Specifically, the proposed algorithm includes a domain-specific representation learning module and a domain-invariant representation learning module. By learning domain-specific features, it extracts characteristics specific to each source domain, resulting in more diverse feature representations. The domain-invariant representation learning module learns domain-independent features, extracting common, invariant characteristics across all domains, thus improving generalization performance. The two feature sets are adaptively fused, enabling the training of a module with good generalization performance. This allows for good classification performance in unknown target domains, completing the human behavior recognition task within the target domain. Attached Figure Description

[0058] Figure 1 This is a logic block diagram of the method system of the present invention. Detailed Implementation

[0059] We propose a domain-specific generalization method based on adaptive feature fusion to improve the generalization ability of behavior recognition models. The key to this method is learning domain-specific and domain-invariant representations and adaptively fusing them into a unified deep neural network. Specifically, domain-invariant representation learning refers to acquiring general, transferable knowledge from the training domain, while domain-specific representation learning refers to learning specific features of each domain, maintaining feature diversity and improving generalization ability. This invention is based on the fact that although we cannot access test data and different people's sensor readings differ, they still have some similarities that can be used to learn domain-invariant representations. Therefore, the goal of this invention is to generalize while maintaining diversity in learning transferable knowledge. This invention can be optimized in an end-to-end neural network, ultimately improving the model's generalization ability on new data.

[0060] This invention proposes a domain generalization method and system based on feature adaptive fusion. The method mainly consists of two modules: a domain-specific representation learning module and a domain-invariant representation learning module.

[0061] 1) Domain-Specific Representation Learning Module

[0062] The purpose of the domain-specific representation learning module is to learn domain-specific features for each source, and then fuse features from multiple sources to represent the invisible target features, aggregating them into a unified feature representation. The domain can be data from different people / individuals or different groups, such as different gender groups or different age groups.

[0063] 2) Domain-invariant representation learning module

[0064] While domain-specific feature learning encourages modules to learn domain-specific information, the distribution of features can vary significantly due to different representations. Therefore, to improve generalization ability, we further designed a domain-invariant representation learning module to seek a balance with domain-specific representation learning.

[0065] The workflow of a domain generalization method and system based on feature adaptive fusion is as follows: Figure 1 As shown, the main steps include:

[0066] During training:

[0067] 1) Preprocess the human behavior dataset by dividing it into K+1 data domains, selecting one domain as the target domain, and the other K domains as the source domains. The target domain is only used during testing, that is, after training the model, it is directly applied to the target domain for testing. Preprocessing includes slicing the data through a sliding window, normalization, etc.

[0068] 2) The feature extractor extracts low-level features;

[0069] 3) K domain-specific feature extractors extract domain-specific features;

[0070] 4) Calculate the domain-specific loss and output the features for each source domain branch;

[0071] 5) Perform distribution adaptation for domain-specific features and calculate the adaptation loss;

[0072] 6) Use the output of the domain classifier as weights to weight and fuse domain-specific features;

[0073] 7) Classifier classification;

[0074] 8) Optimize using gradient descent until convergence.

[0075] During testing:

[0076] The trained model is then tested on target domain data. During testing, only target domain data is input, and features are extracted simultaneously through multiple domain feature extractors. Assuming there are n domain feature extractors, each extracts features, resulting in n features. The low-level features are then processed by a domain classifier, which outputs weights. These weights are then used to perform a weighted summation of the n features.

[0077] This invention includes the following key technical points:

[0078] Key Point 1: This invention addresses the problem of target domain data being invisible during training in human behavior recognition by employing domain generalization. Unlike traditional machine learning methods that train models on existing data before predicting new data, and unlike previous transfer learning and domain adaptation methods, this invention utilizes both source and target domain data during training. Specifically, it proposes a feature adaptive fusion method to improve the model's generalization performance, addressing the issue that target domain data cannot participate in training under the domain generalization setting.

[0079] Key point 2: By learning domain-specific features through domain-specific representation modules, the learned modules possess the characteristics of various domains, resulting in more diverse feature representations;

[0080] Key point 3: By learning domain-invariant features through the domain-invariant representation module, the learned features are generalizable and can extract common and invariant features of each domain, thereby improving the model's generalization performance.

[0081] To make the above features and effects of the present invention clearer and easier to understand, specific embodiments are described below, and detailed descriptions are provided in conjunction with the accompanying drawings.

[0082] 1. Problem Definition

[0083] This invention addresses behavior recognition tasks under a domain generalization setting. We assume that multiple training (source domains) are available, i.e., K different but related training domains. This is obtainable during the training process. Among them, This represents the k-th training domain, which has n elements. k The goal of this invention is to train a model h with generalization ability on K training domains, enabling it to perform training in unseen target domains. The error is minimized on the target domain. This problem is particularly challenging because the test domain (target domain) is unavailable during training. All domain data for sensors and behaviors are isomorphic, meaning they share the same feature space and label space. In real-world applications, different domains have different data probability distributions, i.e., P i (x)≠P j (x)≠P te (x), 1≤i≤j≤K.

[0084] 2. Domain-Specific Representation Learning Module

[0085] The purpose of the domain representation learning module is to learn domain-specific features and then fuse features from multiple source domains into an invisible target feature, thus creating a unified feature representation. More formally, given new test data x, its feature z can be represented as:

[0086]

[0087] Among them, w k It is the source domain The weights represent the difference between the target domain data and the source domain data. Data similarity. k It is a field Feature learning function, f e It is a shared feature extraction function.

[0088] This process can be viewed as a form of ensemble learning, where multiple trained base learners are integrated to obtain a stable model that performs well in all aspects. The lower layers of deep neural networks tend to extract common features at lower levels, while the higher layers tend to learn task-relevant features. Therefore, in our problem, the base learner consists of two parts: a feature extractor f shared by all source domains. e and each source domain-specific feature extractor f k .

[0089] In order to learn the weights w of each source domain k We constructed a domain classifier f d, will f e The output features are used as input. During the training phase, the neighborhood label d for each sample... k ≡k can be used as prior knowledge; therefore, the loss for each source domain is given by , which is used to train the domain classifier:

[0090]

[0091] In the formula, l represents the classification loss. Then, the domain-specific loss is obtained by averaging the losses of all source domains.

[0092] During testing, since we lack prior knowledge of domain labels for our target domain, we use the output of the domain classifier as the weight of each source domain-specific feature (using the softmax technique to make the sum of the weights for each source domain feature equal to 1) for weighted fusion.

[0093] 3. Domain-invariant representation learning module

[0094] While domain-specific feature learning encourages models to learn specific information for each domain, the feature distribution gap can also widen due to different feature representations. Therefore, to improve generalization ability, we further designed a domain-invariant representation learning module (e.g., Figure 1 We employ distribution adaptation to seek a balance between learning domain-specific representations and source domain (SDN) data. We perform feature adaptation to reduce the distributional differences between the source and target domains at specific layers. The domain adaptation strategy aims to reduce the distributional differences between the source and target domains, making the model robust to the target. In domain generalization scenarios, we cannot access target domain data during training; therefore, learning domain-invariant feature representations is crucial for representing unknown targets. We reduce the pairwise source domain sample pairs... and Domain adaptation is performed to minimize the distribution differences between them. Where dist(·,·) is a measure of distributional difference.

[0095] Specifically, we use the maximum mean difference as a distribution metric to reduce domain disparity. This can be formalized as:

[0096]

[0097] Where i and j are the domain labels, and φ(x) is the mapping function that maps the original feature space to the regenerated Hilbert space. The overall domain-invariant loss (adaptation loss) is calculated as follows:

[0098]

[0099] We use cross-entropy loss as the loss function for our classifier, as shown below:

[0100]

[0101] Where N is the number of training samples, y i x is the true label (i.e., category) of the i-th sample. i Let be the i-th sample, and P be the predicted probability value of the output. The learning and classification losses are represented by the two modules above. And β are adaptive hyperparameters, and the overall objective function of our method is:

[0102]

[0103] 4. Training and reasoning

[0104] For training, our method can be easily optimized using gradient descent. Describing the model parameters as θ, the gradient of the overall loss function above can be calculated for a mini-batch of data sampled from K domains using the following formula:

[0105]

[0106] Then, the model parameters can be optimized based on the learning rate and gradient:

[0107]

[0108] When performing inference on target domain data, we fix the model parameters and directly test to obtain the behavior classification on the target domain data.

[0109] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0110] This invention also proposes a behavior recognition system based on feature adaptive fusion and domain generalization, comprising:

[0111] Module 1 is used to acquire behavioral datasets from multiple data domains. Each behavioral data in the dataset has a behavioral identification label and a domain label. One domain in the dataset is used as the target domain, and the domains outside the target domain are used as the source domains. A behavioral identification model is constructed, which includes a behavioral classification module, a common feature extraction module, a specific feature extraction module, and a domain classification module.

[0112] Module 2 is used to extract common features among source domain behavioral data in the behavioral dataset using the common feature extraction module;

[0113] Module 3 is used to train a specific feature extraction module by calculating the domain-specific loss of the source domain, and to extract the domain-specific features of each source domain;

[0114] Module 4 is used to perform distributional adaptation of domain-specific features, learn domain-invariant features, and calculate the adaptation loss to perform distributional adaptation on all domain-specific features.

[0115] Module 5 is used to input the common features into the domain classifier, use the output of the domain classifier as weights, and weight and fuse the domain-specific features to obtain the fused features. The fused features are then input into the behavior classification module to obtain the classification result. The classification loss of the classification module is constructed by combining the behavior recognition label.

[0116] Module 6 is used to train the behavior recognition model based on the classification loss, the adaptation loss, and the domain-specific loss, and to input the behavior data to be tested as the target domain into the trained behavior recognition model to obtain the behavior recognition result of the behavior data to be tested.

[0117] The aforementioned behavior recognition system based on feature adaptive fusion and domain generalization, wherein

[0118] Module 3 includes:

[0119] The loss calculation module is used to calculate the loss specific to the k-th source domain using the following formula:

[0120]

[0121] The k-th source domain has n k Data samples, f d For the classifier of this domain, f e For this shared feature extraction module, x i Let d be the i-th data sample in the current source domain. k For x i Domain tags;

[0122] The averaging module is used to average the losses across all source domains to obtain the domain-specific loss. Where K is the total number of source domains;

[0123] Module 4 is used to obtain the adaptation loss using the following formula:

[0124]

[0125] Where i and j are the domain labels, and φ(x) is the mapping function that maps the original feature space to the regenerated Hilbert space. Indicates the training domain;

[0126] Module 5 uses cross-entropy loss as the classification loss for this classification module.

[0127]

[0128] Where N is the number of training samples.

[0129] The aforementioned behavior recognition system based on feature adaptive fusion and domain generalization, wherein module 6 is used to perform gradient descent on the following formula... Optimize the training to complete the behavior recognition model:

[0130] Where λ and β are adaptive hyperparameters.

[0131] The aforementioned behavior recognition system based on feature adaptive fusion and domain generalization includes module 6, which is used to extract features of the behavior data to be tested through a common feature extractor, and then pass them through the domain classifier to obtain the similarity between the target domain data and each source domain, which is used as weight; the behavior data to be tested is extracted sequentially through all domain feature extractors to obtain multiple domain-specific features, and the weights are used to perform a weighted summation of all these domain-specific features, and the summation result is input into the behavior classification module to obtain the behavior recognition result.

[0132] The present invention also proposes a storage medium for storing a program that executes any of the aforementioned behavior recognition methods based on feature adaptive fusion domain generalization.

[0133] The present invention also proposes a client for any of the aforementioned behavior recognition systems based on feature adaptive fusion domain generalization.

Claims

1. A behavior recognition method based on feature adaptive fusion and domain generalization, characterized in that, include: Step 1: Obtain human behavior datasets from multiple data domains. Each human behavior data in these datasets has a behavior recognition label and a domain label. One domain in the human behavior dataset is taken as the target domain, and the domains outside the target domain are taken as the source domain; a behavior recognition model is constructed, which includes a behavior classification module, a common feature extraction module, a specific feature extraction module, and a domain classifier. Step 2: Using the common feature extraction module, extract the common features among the source domain human behavior data in this human behavior dataset; Step 3: Train the specific feature extraction module by calculating the domain-specific loss of the source domain, and extract the domain-specific features of each source domain; Step 4: Perform distributional adaptation on domain-specific features. Learn domain-invariant features and calculate the adaptation loss to perform distributional adaptation on all domain-specific features. Step 5: Input the common features into the domain classifier, use the output of the domain classifier as weights, and weight and fuse the domain-specific features to obtain the fused features. Input the fused features into the behavior classification module to obtain the classification result, and combine the behavior recognition label to construct the classification loss of the classification module. Step 6: Based on the classification loss, the adaptation loss, and the domain-specific loss, train the behavior recognition model, and input the human behavior data collected by the sensor into the trained behavior recognition model as the target domain to obtain the behavior recognition result of the human behavior data. These data areas include: smart home data, fatigue detection data, and fall detection data for the elderly. Step 3 includes: Step 31, the loss specific to the k-th source domain is: The k-th source domain has One data sample, For this domain classifier, For this shared feature extraction module, x i For the i-th data sample in the current source domain, For x i Domain tags; Step 32: Averaging the losses across all source domains yields the domain-specific loss. , where K is the total number of source domains; Step 4 includes: Step 41: Obtain the adaptation loss using the following formula: Where i and j are the domain labels, It is a mapping function that maps the original feature space to the regenerated Hilbert space. Indicates the training area. and These represent the i-th and j-th training domains, respectively. Step 5 includes: Cross-entropy loss is used as the classification loss for this classification module: ， Where N is the number of training samples, y i It is the true label of the i-th sample.

2. The behavior recognition method based on feature adaptive fusion and domain generalization as described in claim 1, characterized in that, Step 6 includes: in and To adapt hyperparameters, gradient descent is used to... Optimization is performed to complete the training of the behavior recognition model.

3. The behavior recognition method based on feature adaptive fusion and domain generalization as described in claim 1, characterized in that, Step 6 includes: During testing, all common feature extractors extract features from the human behavior data to be tested, which are then passed through the domain classifier to obtain the similarity between the target domain data and each source domain, and used as weights. The human behavior data to be tested is then extracted sequentially by all domain feature extractors to obtain multiple domain-specific features. These domain-specific features are then weighted and summed using the weights, and the summation result is input into the behavior classification module to obtain the behavior recognition result.

4. A behavior recognition system based on feature adaptive fusion and domain generalization, characterized in that, include: Module 1 is used to acquire human behavior datasets from multiple data domains. The human behavior data in this dataset all have behavior recognition labels and domain labels. One domain in the human behavior dataset is taken as the target domain, and the domains outside the target domain are taken as the source domain; a behavior recognition model is constructed, which includes a behavior classification module, a common feature extraction module, a specific feature extraction module, and a domain classifier. Module 2 is used to extract common features among the source domain human behavior data in the human behavior dataset using the common feature extraction module; Module 3 is used to train a specific feature extraction module by calculating the domain-specific loss of the source domain, and to extract the domain-specific features of each source domain; Module 4 is used to perform distributional adaptation of domain-specific features, learn domain-invariant features, and calculate the adaptation loss to perform distributional adaptation on all domain-specific features. Module 5 is used to input the common features into the domain classifier, use the output of the domain classifier as weights, and weight and fuse the domain-specific features to obtain the fused features. The fused features are then input into the behavior classification module to obtain the classification result. The classification loss of the classification module is constructed by combining the behavior recognition label. Module 6 is used to train the behavior recognition model based on the classification loss, the adaptation loss, and the domain-specific loss, and to input the human behavior data to be tested collected by the sensor as the target domain into the trained behavior recognition model to obtain the behavior recognition result of the human behavior data to be tested. These data areas include: smart home data, fatigue detection data, and fall detection data for the elderly. Module 3 includes: The loss calculation module is used to calculate the loss specific to the k-th source domain using the following formula: The k-th source domain has One data sample, For this domain classifier, For this shared feature extraction module, x i For the i-th data sample in the current source domain, For x i Domain tags; The averaging module is used to average the losses across all source domains to obtain the domain-specific loss. , where K is the total number of source domains; Module 4 is used to obtain the adaptation loss using the following formula: Where i and j are the domain labels, It is a mapping function that maps the original feature space to the regenerated Hilbert space. Indicates the training area. and These represent the i-th and j-th training domains, respectively. Module 5 uses cross-entropy loss as the classification loss for this classification module. ， Where N is the number of training samples, y i It is the true label of the i-th sample.

5. The behavior recognition system based on feature adaptive fusion and domain generalization as described in claim 4, characterized in that, This module 6 is used to perform gradient descent using the following formula. Optimize the training to complete the behavior recognition model: in and For adaptive hyperparameters.

6. The behavior recognition system based on feature adaptive fusion and domain generalization as described in claim 4, characterized in that, Module 6 is used to extract features from the human behavior data to be tested through a common feature extractor, and then pass them through the domain classifier to obtain the similarity between the target domain data and each source domain, which is used as a weight. The human behavior data to be tested is extracted sequentially through all domain feature extractors to obtain multiple domain-specific features. The weights are used to perform a weighted summation of all these domain-specific features, and the summation result is input into the behavior classification module to obtain the behavior recognition result.

7. A storage medium for storing a program that executes the behavior recognition method based on feature adaptive fusion domain generalization as described in any one of claims 1 to 3.

8. A client for the behavior recognition system based on feature adaptive fusion domain generalization as described in any one of claims 4 to 6.

Images