A federated personalized human activity recognition training method based on a super network

Abstract

The application discloses a kind of federal individualized human activity identification training methods based on super network, comprising: server randomly selects multiple clients to participate in training, and broadcast embedding network parameters to these clients.Clients receive embedding network parameters, generate embedding description vector in combination with local data set, and upload to server.The server generates corresponding individualized model parameters according to the embedding description vector uploaded by each client, and then issues it to the client for local fine-tuning.After fine-tuning is completed, the client uploads the individualized model parameter update amount to the server, and the server updates the super network accordingly and generates the embedding description vector update amount and returns to the client.The client generates the update gradient of the guide embedding network in combination with the similarity consistency constraint between embedding space-individualized model parameter space and according to embedding description vector update amount.The server collects all the update gradient uploaded by the client and aggregates, and completes the update of embedding network 。

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence and federated learning, and particularly relates to a federated personalized human activity recognition training method based on a hypernetwork. BACKGROUND

[0002] With the wide promotion of federated learning in privacy protection applications, more and more researches introduce it into the human activity recognition task to realize collaborative modeling between multiple clients under the premise of protecting user privacy. In the face of the challenges of non-independent and identically distributed (non-IID) data, label skew and sample sparsity commonly existing in HAR (Human Activity Recognition, HAR) applications, existing researches mainly explore from the directions of regularization constraint, model structure design, meta-learning mechanism, etc.

[0003] Although the existing methods improve the adaptability and robustness of the local model to a certain extent and alleviate the performance degradation caused by non-IID data, they still face the following limitations in actual HAR applications:

[0004] Most methods rely on relatively sufficient local labeled data, while in many real scenarios, high-quality labeled data is usually obtained at a high cost, especially in scenarios where data is scarce and non-IID characteristics are significant, which will seriously weaken the generalization ability and fairness of the global model, resulting in a significant decrease in the recognition performance of traditional models. SUMMARY

[0005] In order to solve the above problems existing in the prior art, the application provides a federated personalized human activity recognition training method based on a hypernetwork. The technical problems to be solved by the application are realized by the following technical solutions:

[0006] The embodiment of the application provides a federated personalized human activity recognition training method based on a hypernetwork, which is applied to a human activity recognition personalized modeling system including a server and multiple clients. The server is deployed with an embedding network and a hypernetwork, and each client is deployed with an embedding network. The corresponding training method comprises:

[0007] The server randomly selects a client and broadcasts the current embedding network parameters to the selected client;

[0008] Each client collects and pre-processes local human activity data, constructs a local data set, generates its own embedding description vector based on the local data set and the embedding network parameters, and sends the embedding description vector to the server;

[0009] The server generates corresponding personalized model parameters by using a hypernetwork according to the embedding description vectors sent by all clients, and sends the personalized model parameters to each client;

[0010] Each client fine-tunes the personalized model parameters sent by the server by using the local data set to obtain personalized model parameter updates, and sends the personalized model parameter updates to the server;

[0011] The server calculates the hypernetwork gradient and the embedding description vector updates by using a back propagation algorithm and according to the personalized model parameter updates, and sends the embedding description vector updates to each client;

[0012] Each client calculates corresponding first embedding network gradients according to the embedding description vector updates, generates corresponding second embedding network gradients according to the similarity consistency constraint between the embedding space and the personalized model parameter space, calculates the optimized embedding network gradient of the corresponding client according to the first embedding network gradient and the second embedding network gradient, and sends the optimized embedding network gradients of all clients to the server;

[0013] The server aggregates all the optimized embedding network gradients, updates the parameters of the embedding network and the hypernetwork according to the hypernetwork gradient and the aggregated embedding network gradient, and performs test of personalized human activity recognition according to the updated embedding network and hypernetwork.

[0014] The beneficial effects of the present application are as follows:

[0015] The federated personalized human activity recognition training method based on the super network is a federated meta-learning method which can realize generalization modeling and personalized inference without relying on a large amount of local labeled data, and specifically: by designing an embedding-generation integrated super network architecture, the embedding description vectors uploaded by the clients are uniformly learned on the server, and the personalized model parameters are directly generated, thereby realizing efficient adaptation and generalization of multi-source non-independent and identically distributed data. Meanwhile, the embedding network is combined to realize collaborative optimization of personalized modeling and global knowledge transfer, thereby enhancing the robustness of the system to the non-IID characteristics and sample sparsity of the data, so that the system can adapt to the data distribution under the premise of only obtaining a small amount of client uploaded information, thereby effectively addressing the modeling degradation problem caused by data sparsity and improving the generalization ability and fairness of the model under the non-IID condition. In addition, the invention supports a lightweight deployment mechanism, which flexibly adjusts the local fine-tuning process according to the computing resources of the edge device, and under the condition of limited resources, the personalized model parameters generated by the server can be directly used for efficient inference, the client does not need to perform an additional local fine-tuning process, and the super network on the server can directly generate personalized model parameters matching the data distribution of the client by uploading the pre-extracted embedding description vectors, thereby reducing the communication and computing burden of the system, improving the deployment efficiency and applicability, and when a certain computing capability is available, the performance can be further improved through a small amount of local fine-tuning of the client, thereby realizing dynamic balance between model precision and system efficiency, and being applicable to various edge computing scenarios. The super network is centrally trained and personalized model parameters are generated, and the client only needs to transmit lightweight embedding description vectors, thereby effectively reducing the communication bandwidth requirement, reducing the computing load of the client, greatly improving the scalability and applicability of the system, and significantly improving the practicality and rapid adaptation ability of the system in a resource-limited terminal environment; the invention introduces a similarity consistency constraint between the embedding space and the personalized model parameter space during the training process, thereby constraining the potential representation space of the personalized modeling of the client, effectively suppressing the fluctuation range of the model performance during the training process, thereby improving the stability and accuracy of the personalized model parameters generated by the super network, effectively alleviating the training interference caused by data skew or noise, and improving the model convergence and training stability, and the mechanism effectively alleviates the model oscillation and performance degradation problem caused by the data distribution difference of the client, thereby enhancing the discriminability and generalization ability of the model.

[0016] The invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a flowchart of a federated personalized human activity recognition training method based on a super network provided by an embodiment of the invention;

[0018] Figure 2 is a schematic diagram of a human activity recognition personalized modeling system provided by an embodiment of the invention;

[0019] Figure 3 is a schematic diagram of a similarity consistency constraint mechanism of an embedding space and a personalized model parameter space provided by an embodiment of the present application;

[0020] Figure 4 is a schematic diagram of loss function curves of a model before and after adding a similarity consistency constraint provided by an embodiment of the present application;

[0021] Figure 5 is a schematic diagram of client classification accuracy in a training stage before and after adding a similarity consistency constraint provided by an embodiment of the present application. DETAILED DESCRIPTION

[0022] The present application will be further described in detail below with reference to specific embodiments, but the embodiments of the present application are not limited thereto.

[0023] For the recognition performance degradation problem caused by the data non-independent and identically distributed characteristics and sample sparsity in the multi-client scenario, please refer to Figure 1 An embodiment of the present application provides a federated personalized human activity recognition training method based on a super network, which is applied to a human activity recognition personalized modeling system including a server and multiple clients as shown in Figure 2 The server is deployed with an embedding network and a super network, and each client is deployed with an embedding network. By constructing a shared embedding network and a super network structure, the server supports unified modeling and the clients support efficient personalized generation, thereby improving the adaptability and generalization ability of the recognition model while protecting data privacy. After the training starts, first, a communication connection between the server and each client is established, and then the corresponding training method includes:

[0024] S10, the server randomly selects a client and broadcasts the current embedding network parameters to the selected client;

[0025] S20, each client collects and pre-processes local human activity data, constructs a local data set, generates an embedding description vector based on the local data set and the embedding network parameters, and sends the embedding description vector to the server;

[0026] S30, the server generates corresponding personalized model parameters using the super network according to the embedding description vectors sent by all clients, and sends the personalized model parameters to each client;

[0027] S40, each client fine-tunes the personalized model parameters sent by the server using the local data set to obtain a personalized model parameter update amount, and sends the personalized model parameter update amount to the server;

[0028] S50, the server utilizes a back propagation algorithm and calculates a super network gradient and an embedding description vector update amount according to the personalized model parameter update amount, and sends the embedding description vector update amount to each client;

[0029] S60, each client calculates a corresponding first embedding network gradient according to the embedding description vector update amount, generates a corresponding second embedding network gradient according to a similarity consistency constraint between an embedding space and a personalized model parameter space, calculates an optimized embedding network gradient of the corresponding client according to the first embedding network gradient and the second embedding network gradient, and sends the optimized embedding network gradients of all clients to the server;

[0030] S70, the server performs aggregation processing on all the optimized embedding network gradients, completes parameter updating of the embedding network and the super network according to the super network gradient and the aggregated embedding network gradient, and performs test of personalized human activity recognition according to the embedding network and the super network after parameter updating.

[0031] S10~S70 are described in detail as follows.

[0032] As shown in the human activity recognition personalized modeling system of the application Figure 2 Relying on a federal learning framework, the human activity recognition personalized modeling system is composed of a plurality of clients (such as user smart wearable devices) and a server. Each client collects local data through a wearable device (such as an accelerometer, a gyroscope, etc.) for a local human activity recognition model. Under the privacy protection principle of federal learning, the client only shares necessary model-related information with the server and does not transmit original data. The purpose is to improve the personalized ability and generalization ability of the human activity recognition model under the premise of protecting data privacy. In the human activity recognition personalized modeling system, it is assumed that there are clients (users), the client has a local data set It is assumed that the data is non-independent and identically distributed (non-IID), that is, the data distribution is different between different clients, that is (when ). Wherein, represents the local data of the client, represents the corresponding activity label, represents the number of local data owned by the client.

[0033] For each client , the client objective optimization function is defined as:

[0034] (1);

[0035] in, Indicates the first The client-side objective optimization function for each client. Indicates the first Personalized model parameters for each client, i.e., local model parameters. For loss function, Indicates the first The number of training samples used for training in each client's local dataset. , Indicates the first The training sample set of the client is the first One training sample (local data), express Corresponding activity tags, express The corresponding loss function, such as cross-entropy loss or mean squared error loss, represents the personalized model parameters. Down, The loss.

[0036] This invention teaches each client a unique, personalized model parameter, denoted as . The goal is to train a local model for each user, ensuring good performance on local data, while leveraging the collaborative knowledge of other users to improve generalization ability. Ultimately, the modeling objective can be formalized as the following joint optimization problem:

[0037] (2);

[0038] in, For the first The client-side objective optimization function of each client represents a form of shared structure or knowledge carrier (such as meta-initialization, feature mapping function, generator, etc.); Represents personalized model parameters With shared structure Coordination constraints between clients are used to mitigate distribution biases among clients; For regularization terms in sparse sample scenarios (such as structural constraints, robustness against perturbations, or prior guidance); This describes a method that balances collaborative constraints and sparse regularization terms, which can be implemented using existing technologies. This is the regularization term strength control parameter; The number of clients selected; Let be the total number of clients in the system. The joint optimization problem shown in Equation (2) takes each client model as the core, uses shared knowledge for indirect collaboration, and improves the modeling stability and generalization ability under data sparsity and non-IID conditions through a regularization mechanism.

[0039] Based on the above analysis, this invention deploys a meta-learning network on a server, which consists of an embedded network. Hechao Network It consists of two parts:

[0040] (3);

[0041] They work together to achieve the meta-learning goal of "learning how to generate models." The embedding network integrates convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to jointly model multi-axis sensor data. The network first extracts features from preprocessed local data (such as gyroscope and accelerometer signals) through multi-layer convolutional structures, aiming to capture spatial dependencies between different axes within a single sensor, while also uncovering potential correlations between multiple sensors. Specifically, for each input sequence sample consisting of multiple time slices, the embedding network first performs convolution operations on the frequency and amplitude pairs of each local data along different axes, then integrates all axes of the same local data, and finally completes the fusion of cross-sensor data in the last convolutional layer. After the convolutional stage, the embedding network further processes the extracted feature sequences using two layers of Long Short-Term Memory (LSTM) units to capture the temporal characteristics between different time slices, ultimately outputting a fixed-length embedding description vector representing the behavioral pattern of the entire input sample.

[0042] The hypernetwork's structure primarily consists of two parts: a feature extractor and a parameter generator. First, the client's embedding description vector serves as input to the hypernetwork, representing the client's feature information. This embedding description vector is then fed into a multilayer perceptron (feature extractor) containing three fully connected layers to extract high-level feature vectors, mapping the low-dimensional representation of the input to a more expressive feature space. Next, the extracted feature vectors are fed into the parameter generator, which generates personalized model parameters for constructing the client's model based on these vectors. Throughout this process, the hypernetwork achieves an end-to-end mapping from client embeddings to personalized model parameters, enabling the personalized model to adaptively adapt to the data distribution of different clients. This hypernetwork-based design avoids the overhead of storing model parameters separately for each client, while also providing customized models for each client, enhancing the personalization effect.

[0043] For each client, the client's embedding description vector After the super network Obtain personalized model parameters Among them, the client's embedded description vector It is due to its embedded network The output maps each training sample to an embedding description vector and averages them:

[0044] (4);

[0045] in, Indicates the first Embedded description vectors for each client, Represents local dataset The amount of local data in China, i.e. , Indicates sample After being embedded through the network, an embedding description vector is generated.

[0046] Since the local dataset collected and preprocessed by the client is non-IID, the goal is to improve the performance of personalized modeling by extracting the client's embedding description vectors and constructing a model generation mechanism without disclosing the local data. Therefore, the method of this invention can be defined as a model generator composed of a shared embedding network and a supernetwork, wherein... =( , ) represents the meta-model parameters.

[0047] This invention aims to learn personalized model parameters for each client based on the principles of federated learning. This improves the model's generalization ability and stability. Therefore, during each client-server interaction, the objective function can be defined in two parts: each client constructs its own client-side objective function, and the server constructs its own server-side objective function. Based on these two functions, a comprehensive objective function is constructed. Then, based on this comprehensive objective function, multiple iterations are performed until the iteration stopping condition is met.

[0048] One part is the client-side objective optimization function for each client:

[0049] (5);

[0050] The other part is the server's objective optimization function:

[0051] (6);

[0052] Finally, the overall objective function is:

[0053] (7);

[0054] in, Indicates the first The client-side objective optimization function for each client. This represents the server's objective optimization function. This indicates the number of clients selected. Indicates the first Personalized model parameters for each client, Indicates client The local dataset that we have.

[0055] Substituting formulas (5) and (6) into formula (7), we get:

[0056] (8);

[0057] in, express The embedding description vector is obtained after passing through the client's embedding network. This embedding description vector is then input into the server's hypernetwork to generate the corresponding personalized model parameters. This represents the regularization parameter corresponding to the personalized model parameters. Indicates hypernetic parameters. Indicates embedded network parameters, This represents the regularization parameter corresponding to the hypernetic parameters. This represents the regularization parameter corresponding to the embedded network parameters. This represents the L2 norm.

[0058] During the training phase, to protect user privacy, the client does not directly upload local data. Instead, it only shares necessary model-related information with the server during the training process. Specifically, the server first uploads the current embedded network parameters... The data is distributed to each client. Upon receiving the data, the client collects and preprocesses local human activity data, constructs a local dataset, and then uses this local dataset along with the received embedded network parameters. Generate its own embedding description vector Then embed the description vector Uploaded to the server. After receiving all the embedded description vectors uploaded by the clients, the server uses the hypernetwork to generate corresponding personalized model parameters. And send it back to their respective clients. The clients receive the personalized model parameters. Then, it can be used for inference or further fine-tuning of local tasks, and can also participate in subsequent iterative optimization processes as needed. Preprocessing involves transforming local data into a format usable by the model, such as data cleaning, time window partitioning, and data partitioning, using existing technologies.

[0059] During the reverse update phase of the hypernetwork, each client performs multiple rounds of fine-tuning on its personalized model parameters and local dataset, and updates the resulting personalized model parameters. Send to the server. Can be regarded as The proxy value. According to the chain rule, we have:

[0060] (9);

[0061] (10);

[0062] in, Indicates the first The client-side objective optimization function for each client. express exist The partial derivatives, which are the gradients of the supernetwork. , express exist The partial derivatives, express exist The partial derivatives, express exist The partial derivatives, express exist The partial derivatives of .

[0063] The server can use the backpropagation algorithm to calculate the above expression, thereby obtaining the hypernetwork gradient for each client. and Embedded description vector update amount The embedded description vector update amount Sent back to the client as The proxy value. The hypernetwork gradients corresponding to all clients. Perform aggregation: To determine the gradient of the aggregated supernetwork Update the super network.

[0064] Based on the chain rule of formula (11), the client can calculate the gradient of the first embedded network. And send it back to the server.

[0065] (11);

[0066] in, Indicates the first The first embedding network gradient of each client, Indicates the first The client-side objective optimization function for each client. Indicates embedded network parameters, express exist The partial derivatives, Indicates the first Embedded description vectors for each client, express exist The partial derivative of also represents the amount of the embedding description vector update. express exist The partial derivatives of .

[0067] Finally, the server updates all network parameters based on the average contribution of each client and the gradient information of its own objective function.

[0068] In highly heterogeneous federated learning scenarios for human activity recognition, some clients, due to extremely scarce samples or a single category, may contain only one activity category. This causes their embedding representations to deviate significantly from the backbone distribution, resulting in unstable or misleading mappings during the generation of personalized models in the supernetwork. This bias not only weakens the generalization ability of the client's own model but may also negatively impact the supernetwork during training, contaminating the generation paths of other clients.

[0069] To alleviate this problem, this invention introduces a similarity consistency constraint between the embedding space and the personalized model space. This means that the similarity of the embedded description vectors between clients remains consistent with the similarity of their personalized model parameters. This mechanism effectively improves the performance and stability of the model in non-IID and sparse data environments in HAR tasks. Furthermore, addressing the issue of unstable training processes caused by common data characteristics in HAR tasks, such as non-independent and identically distributed data, label skew, and noise interference, this invention proposes an improved training strategy.

[0070] In federated learning, the differences in client-specific model parameters can, to some extent, reflect the differences in their local data distribution. Therefore, this invention introduces structural consistency constraints to enhance the expressive power of embedding vectors on client data distribution. Each client generates a corresponding second embedding network gradient based on the similarity consistency constraint between the embedding space and the personalized model parameter space, including: downloading a historical set of embedding description vectors and a historical set of personalized model parameters from the server; calculating the embedding similarity matrix among all clients based on the historical embedding description vector set, and calculating the model parameter similarity matrix among all clients based on the historical personalized model parameter set; constructing an auxiliary loss term based on the embedding similarity matrix and the model parameter similarity matrix, and calculating the second embedding network gradient for the corresponding client based on the auxiliary loss term. Specifically:

[0071] like Figure 3 As shown, two sets of similarity matrices were designed to characterize the structural correspondence between the embedding space and the personalized model parameter space: On the one hand, cosine similarity was used to measure the similarity between the client-embedded descriptive vectors, resulting in the embedding similarity matrix. According to Figure 3 middle calculate On the other hand, by calculating and normalizing the Euclidean distance between personalized model parameters, a similarity matrix reflecting the differences in model structure is constructed. According to Figure 3 middle calculate Subsequently, an auxiliary loss term is constructed by minimizing the mean squared error between the two sets of similarity matrices:

[0072] (12);

[0073] in, Indicates auxiliary loss, This indicates the number of clients selected. Indicates the first The client and the first Embedding similarity between clients Indicates the first The client and the first Model parameter similarity between clients.

[0074] Introducing this mechanism into the training process guides the embedding space to maintain consistency with the personalized model parameter space. This mechanism encourages clients with similar data distributions to have closer embedding vectors, while clients with significantly different data maintain greater discriminative power in the embedding space, thereby improving the discriminativeness of the embedding representation and the ability to personalize modeling. Simultaneously, this strategy effectively enhances the consistency between the embedding space and the model space by guiding the hypernetwork to learn structure-preserving mappings during the training phase, thus improving the overall model stability and robustness. Particularly when dealing with isolated clients, this consistency constraint limits the extreme model outputs corresponding to their anomalous embeddings, reducing gradient shifts and generalization degradation risks during global training. Furthermore, this mechanism also possesses implicit transfer learning capabilities, allowing low-resource clients to obtain higher-quality model generation by leveraging other clients with similar embeddings, contributing to improved robustness and personalization performance under realistic distributions.

[0075] The update of the embedded network consists of two parts:

[0076] Part of it is the current number Embedded description vectors of each client The amount of embedded description vector updates sent by the server The gradient of the first embedded network obtained by joint computation , as in formula (11);

[0077] The other part is calculated from the auxiliary loss term. The gradient of the second embedded network for the corresponding client is calculated based on the auxiliary loss term, as expressed by the formula:

[0078] (13);

[0079] in, Indicates the first The second embedding network gradient of each client, Indicates auxiliary loss, Indicates embedded network parameters, express exist The partial derivatives, This represents the embedding vector similarity matrix for all clients. express exist The partial derivatives, express exist The partial derivatives of .

[0080] By weighted fusion of the two gradient components, the optimized embedding network gradient for the corresponding client is obtained, which serves as the update direction. The formula is expressed as:

[0081] (14);

[0082] in, Indicates the first Optimization of embedded network gradients for each client, Indicates the first The first embedding network gradient of each client, Indicates the first The second embedding network gradient of each client, This represents the weight parameters. Finally, the optimizations from all clients are embedded into the network gradients and uploaded to the server for aggregation. To determine the gradient of the aggregated embedded network Update the embedded network.

[0083] The above only describes a single round of training. After each round of training, it is determined whether the preset number of iterations has been reached. If not, the next round of training continues. If the conditions are met, the training process is terminated and the model training is completed. The trained model is then used to identify personalized human activities during testing, such as using a new client that did not participate in the training process for testing.

[0084] The training process of this invention mainly consists of two parts: server training and aggregation process and client-side embedded computing and personalized model parameter update.

[0085] (1) Server-side training and aggregation process

[0086] This invention designs an efficient federated meta-learning model training and aggregation process on a server. The entire training process executes T rounds, in which the server randomly selects data from the system during each round. N Select from clients n Each client participates in modeling. At the start of each round, the server broadcasts the current embedding network parameters to the selected clients, and then receives the embedding description vectors returned by each client. Based on the collected embedding description vectors, the server generates corresponding personalized model parameters using the hypernetwork and distributes them to the respective clients. After completing several rounds of fine-tuning training on their local dataset using the personalized model parameters, the clients upload the updated personalized model parameters to the server. Based on the updated personalized model parameters reported by the clients, the server calculates the hypernetwork gradient and embedding network gradient using the backpropagation algorithm, and updates the hypernetwork and embedding network parameters accordingly. This updated hypernetwork and embedding network are then used for subsequent testing. This process iterates until the iterative convergence condition is met, thus completing the global optimization of the embedding network and hypernetwork.

[0087] (2) Client-side embedded computing and personalized model parameter update process

[0088] On the client side, this invention proposes a process for calculating embedding description vectors and updating personalized model parameters to achieve local data modeling and collaborative training. In each round of federated training, the selected client first constructs its own embedding description vector based on its local dataset and uploads it to the server. After the server returns personalized model parameters using the hypernetwork, the client performs several rounds of fine-tuning training locally based on the received personalized model parameters to adapt to personalized needs, and then returns the updated personalized model parameters to the server. Simultaneously, to enhance the model's structural consistency and training stability, the client also downloads the historical embedding description vector set and personalized model parameter set maintained by the server, and calculates the embedding similarity matrix for each. Similarity matrix with model parameters The update direction of the embedded network is obtained through a similarity constraint-assisted mechanism, and the final calculated gradient of the embedded network is uploaded to the server. This mechanism effectively improves the performance of personalized modeling and enhances the training robustness of the system in non-IID and data sparse environments.

[0089] This invention uses the trained overall model (including the embedding network and the supernetwork) during the testing phase to generate personalized model parameters based on the new client's local dataset, thereby effectively improving personalized modeling performance. Typically, during testing, the generated personalized model parameters can be directly used to perform local human activity recognition tasks on the client, achieving high recognition accuracy. To further improve recognition performance, a few rounds of fine-tuning can be performed locally, allowing for the use of the fine-tuned personalized model parameters to achieve even higher accuracy in local personalized human activity recognition tasks.

[0090] To verify the effectiveness of the federated personalized human activity recognition training method based on hypernetwork provided in this embodiment of the invention, the following experiments were conducted.

[0091] This invention selects three typical federated learning human activity recognition task datasets for evaluation: the real-world multi-device, multi-user HHAR dataset, the classic sensor action recognition benchmark UCI-HAR dataset, and the WISDM multi-user acceleration dataset with non-independent and identically distributed characteristics. All three can be used to simulate typical multi-client heterogeneous environments in federated learning, and are suitable for validating personalized model building.

[0092] To simulate the personalized and heterogeneous distribution of client data, this invention randomly deletes 0-2 action categories of data from each client. On all three datasets, a unified basic network structure is used: a two-layer gated recurrent unit (GRU) network with 128 hidden units per layer is constructed as the base network for time series inference tasks. The results are output through a fully connected layer and compared with the ground truth to obtain the prediction accuracy. The optimizer is Adam, with an initial learning rate of 0.001 and a cross-entropy loss function. The number of epochs for local training in each communication round is set to 5, and the batch size is set to 32, 32, and 8 samples on the three datasets, respectively. All federated methods have a maximum iteration count of 300 per communication round.

[0093] This invention uses classification accuracy and macro-average F1 score as evaluation metrics. All clients participating in the training were tested on their local test datasets, and the average score was taken as the final result. All experiments were conducted using the PyTorch 2.3.1 framework, running on a Windows 11 system with a 13th Gen Intel(R) Core(TM) i5-13500H processor and 32GB of memory. Experimental results verify the effectiveness of the proposed method in multi-user heterogeneous and data-sparse scenarios.

[0094] To comprehensively evaluate the effectiveness of the proposed method, this invention selects several representative federated learning methods for comparison, including classic global sharing models, early personalized federated methods, and structured personalized strategies proposed in recent years: FedAvg's classic federated averaging algorithm weights all client model parameters, lacks a personalized mechanism, and is susceptible to non-IID data; Meta-HAR is based on a meta-learning framework, learning initialization parameters during training to improve rapid adaptation to new tasks; Knn-Per utilizes the feature similarity between clients to perform K-nearest neighbor aggregation, thereby achieving personalized prediction, suitable for scenarios with small samples and significant local personalized features; pFedHN introduces a hypernetwork structure to generate personalized model parameters for each client, improving adaptability and generalization ability under heterogeneous data; ProtoHAR is based on the idea of ​​prototype learning, constructing a global prototype representation to assist local classifier optimization, and exhibiting good transfer capabilities in heterogeneous environments. This invention constructs a consistent mapping mechanism between personalized embedding representations and model parameters, integrating a hypernetwork-based model generation structure, client-side embedding description computation methods, and a similarity-constrained loss strategy. This achieves superior personalized modeling results and higher system robustness in complex scenarios with multiple clients, non-independent and identically distributed data, and sparse data. The method of this invention is denoted as the Hyper-HAR method. Specific effects are as follows:

[0095] (1) Improve the robustness of task modeling and recognition accuracy in multi-client environments.

[0096] The experiments are shown in Tables 1-3. On the HHAR dataset, there are significant differences in data distribution among clients. The FedAvg method performs the worst in this context, with low accuracy and F1 score, reflecting its insufficient adaptability to non-IID environments. The Meta-HAR and Knn-Per methods, as early personalized methods, show improved overall performance. The Knn-Per method achieves high accuracy of 90.74% and 83.22% on clients 2 and 6, respectively, but its average F1 score is still lower than more complex methods. The pFedHN and ProtoHAR methods, by introducing hypernetworks and prototype representations, demonstrate stable performance across multiple clients, exhibiting good transferability and robustness. The Hyper-HAR method proposed in this invention achieves leading or near-best performance on all clients, demonstrating stronger generalization ability and consistency, and exhibiting the best overall performance. In the UCI-HAR dataset: due to the relatively regular data distribution, all methods generally performed well. In particular, the Hyper-HAR method proposed in this invention achieved an accuracy of over 90% across all clients, reaching 97.65% and 92.27% on clients 1 and 3 respectively, with an average accuracy of 94.18% and an F1 score of 0.9325, making it the best performing method. On the WISDM dataset: limited by uneven sample distribution and class skewness, the FedAvg method showed a significant performance decline, with accuracy below 80% on some clients. The Meta-HAR and Knn-Per methods performed slightly better but lacked stability. The pFedHN and ProtoHAR methods demonstrated stronger structural adaptability. The Hyper-HAR method proposed in this invention also maintained the highest average accuracy on this dataset, verifying its strong robustness and generalization ability in highly heterogeneous and sparse data environments.

[0097] Table 1. Classification accuracy (Acc, %) and F1 score of different methods on the HHAR dataset.

[0098]

[0099] Table 2. Classification accuracy (Acc, %) and F1 score of different methods on the UCI HAR dataset.

[0100]

[0101] Table 3. Classification accuracy (Acc, %) and F1 score of different methods on the WISDM dataset.

[0102]

[0103] In summary, the strategy of directly generating personalized model parameters based on hypernetworks demonstrates stronger adaptability and robust performance in typical federated learning challenges such as multiple data distributions, sparse samples, and uneven class distributions, and its overall performance is superior to existing personalized federated methods.

[0104] (2) Improve the flexibility of model deployment and performance response efficiency on edge devices

[0105] The experiments are shown in Tables 4-6. This invention compares the performance of various methods on three datasets after 0 rounds (direct inference), 1 round, 3 rounds, and 5 rounds of fine-tuning. The results show that the proposed Hyper-HAR method can effectively balance model performance and system efficiency in edge computing environments through an on-demand fine-tuning mechanism. Without local fine-tuning (0 rounds), the Hyper-HAR method significantly outperforms existing methods in classification accuracy and F1 score on the three datasets. Specifically, the direct inference accuracy on the UCI HAR dataset reaches 87.13%, a 4.04% improvement over the second-best existing pFedHN method. This fully verifies its excellent personalization capabilities; the personalized model generated on the server side already possesses strong user feature adaptation capabilities and can be directly applied to edge devices with extremely limited computing resources. When edge devices have a certain computing margin, only one round of fine-tuning is needed to bring significant performance improvements. The Hyper-HAR method stabilizes and continues to outperform existing methods after 3-5 rounds of fine-tuning. For example, on the UCI HAR dataset, after 5 rounds of fine-tuning, the accuracy reaches 94.18% and the F1 score is 0.9325, both of which are best-in-class. Hyper-HAR also performs robustly on the challenging WISDM dataset. Experimental data shows that without any local fine-tuning (0 rounds), Hyper-HAR's recognition accuracy (62.81%) is close to the performance of other personalized federated learning methods after 5 rounds of fine-tuning.

[0106] Table 4. Classification accuracy (Acc) and F1 score of different methods at different rounds of fine-tuning on the HHAR dataset.

[0107]

[0108] Table 5. Classification accuracy (Acc) and F1 score of different methods at different rounds of fine-tuning on the UCI HAR dataset.

[0109]

[0110] Table 6 shows the classification accuracy (Acc, %) and F1 score of different methods at different rounds of fine-tuning on the WISDM dataset.

[0111]

[0112] These results demonstrate that the on-demand fine-tuning mechanism of this invention can flexibly adjust the optimization intensity according to the actual computing power of edge devices: for resource-constrained devices, pre-trained models can be used directly for efficient inference, while for devices with sufficient computing power, performance can be quickly improved through a small number of fine-tuning rounds, thereby maximizing model accuracy while ensuring system response efficiency, and providing a practical and efficient solution for lightweight human activity recognition systems in edge computing environments.

[0113] (3) Enhance model training stability and improve cross-client representation consistency.

[0114] The client classification accuracy results during the training phase of this invention are shown below for three datasets, with and without auxiliary loss terms added to the client. Figure 4 As shown, from Figure 4 It can be observed that, compared to the case without similarity constraints, the model with this auxiliary loss term exhibits a more stable and continuously increasing accuracy trend during training. Although the difference in accuracy improvement speed is not significant in the initial stage, as the number of training epochs increases, the model without similarity constraints generally experiences increased accuracy fluctuations, slower convergence speeds, and even performance degradation. In contrast, with the introduction of similarity constraints, the model accuracy remains relatively stable with smaller fluctuations, ultimately achieving higher classification performance. This indicates that the auxiliary loss term effectively guides the hypernetwork to learn more consistent and discriminative representations in the structural space during training, thereby improving model training stability and convergence efficiency. Figure 4 The yellow curve represents the classification accuracy without the auxiliary loss term, while the blue curve represents the classification accuracy with the auxiliary loss term.

[0115] The present invention provides loss change curves for the overall objective optimization function during training on three datasets with and without auxiliary loss terms added to the client, as shown in the figure. Figure 5 As shown. From Figure 5 It can be observed that without similarity constraints, although the overall loss value decreases rapidly in the early stages of training, it begins to fluctuate significantly in later stages, with the amplitude of the oscillations gradually increasing. This phenomenon indicates that in the absence of structural constraints, the parameter space of the personalized model exhibits strong instability, making the training process prone to oscillations or even degradation, resulting in poor convergence. In contrast, with the addition of similarity constraints, the training curve is smoother overall, the overall loss value decreases continuously and remains stable in the later stages, without significant oscillations. Figure 5 The yellow curve represents the classification accuracy without the auxiliary loss term, while the blue curve represents the classification accuracy with the auxiliary loss term.

[0116] (4) Improve the accuracy of personalized modeling and mitigate the impact of data sparsity and class imbalance.

[0117] As shown in Table 7, the overall performance of the Hyper-HAR method significantly decreases without the introduction of similarity constraints. Taking the HHAR dataset as an example, the average accuracy drops from 64.52% to 63.01%, and the F1 score decreases from 0.6305 to 0.6179. Similar performance degradation is observed on the UCIHAR and WISDM datasets. This indicates that structural consistency constraints play a positive role in improving the discriminativeness of client-side embedding representations and enhancing the model's personalization capabilities. Particularly on the WISDM dataset, the accuracy improves from 65.40% to 68.79% and the F1 score from 0.5342 to 0.5537 after introducing similarity constraints, demonstrating a more significant improvement. This indicates that the mechanism is particularly effective in addressing uneven data distribution and class imbalance. This demonstrates that structural consistency constraints not only guide embedding vectors to better align with the personalized modeling needs of each client but also effectively improve the stability of model training and the final generalization effect, validating the effectiveness and practical value of this module in federated personalized learning scenarios.

[0118] Table 7. Classification accuracy (Acc, %) and F1 score on the three datasets before and after adding similarity constraints.

[0119]

[0120] As can be seen from the above, the Hyper-HAR method proposed in this invention can be better applied to various human activity data. For example:

[0121] (1) Smart medical and health monitoring scenarios supported by wearable devices

[0122] This invention is deployed on a smart healthcare platform for monitoring patients' daily behavior and assessing their rehabilitation. The hospital assigns each patient a smart wearable device (such as a smart bracelet or motion-sensing garment), and the client performs human activity recognition by collecting accelerometer and gyroscope data. A personalized model based on a hypernetwork automatically adapts to the different lifestyles and behavioral patterns of different patients; for example, elderly patients tend to engage in low-intensity activities, while patients in rehabilitation require a focus on identifying gait abnormalities during exercise. The system models individual differences by embedding vectors, avoiding misjudgments of behavioral features by a uniform model. The embedding-model consistency constraint mechanism effectively improves the stability of model generation among different users, ensuring high sensitivity and low false alarm rate for abnormal activities such as sudden falls, providing highly reliable support for personalized telemedicine.

[0123] (2) Application of personalized training feedback in intelligent fitness assistance systems

[0124] This invention is integrated into a smart gym system, where users wear wearable devices for physical training. The server-side generates personalized models based on users' historical data and physical condition using a hypernetwork. These models identify and evaluate the training movements of different users; for example, for the same set of squat movements, the system can adjust the model sensitivity according to individual strength differences. When multiple users train simultaneously, the system learns the different movement styles of users through an embedding network and improves embedding discrimination ability with similarity constraints, avoiding misidentification due to data distribution discrepancies between individuals. The client provides training guidance based on the received personalized models and can further optimize model performance through local fine-tuning when resources permit, achieving accurate training movement evaluation and correction suggestions.

[0125] (3) Campus security patrol system under edge computing conditions

[0126] This invention is deployed in a university security patrol system. Multiple cameras or mobile robots act as clients, collecting video data of human behavior and performing local recognition tasks. Different deployment areas exhibit different human behavior characteristics; for example, dormitories show a large amount of stationary behavior at night, while teaching buildings primarily show mobile behavior. This invention utilizes embedded networks to generate feature descriptions for each camera location. The server infers personalized model parameters based on the hypernetwork, achieving accurate recognition of behaviors in different scenarios. For example, a dormitory camera location might mistakenly identify sleeping postures as falls under the original model; after introducing personalized modeling, the false alarm rate significantly decreases. Even without local fine-tuning (0 rounds), the server-generated model exhibits good recognition performance, meeting the real-time and accuracy requirements of campus edge devices with limited computing power.

[0127] (4) Application of personalized activity recognition in health monitoring of the elderly

[0128] This invention is applied to a home-based health monitoring system for elderly people in a community. Addressing the challenges of significant differences in daily behavior patterns, limited mobility, and limited movement types among different elderly individuals, it constructs a distributed modeling scheme that supports privacy protection and personalized identification. Each user's mobile device collects daily activity data, such as walking, sitting / lying down, and falls. A shared embedding network is trained on the server side using a federated learning framework, and a personalized model is automatically generated based on client-side feature descriptions. The system utilizes a hypernetwork to model each user's movement patterns, generating stable and reliable identification models even with limited activity types and sparse sample sizes. For example, for users with limited mobility, their activity data is more concentrated, and personalized modeling avoids misjudgments; while for users who are habitually active, the system distinguishes their active behavior patterns using descriptive vectors and generates more discriminative model parameters. Furthermore, when the system detects a user maintaining a similar posture or repetitive trajectory for an extended period (such as prolonged sitting or lying still), it identifies potential risk states through a laziness dilemma detection mechanism and adjusts the model strategy using a reward mechanism to guide attention to potential health abnormalities.

[0129] This invention has significant advantages in handling data heterogeneity among users, ensuring personalized accuracy, and enhancing anomaly identification capabilities, providing a flexible and highly adaptable model generation method for intelligent health monitoring.

[0130] In summary, the federated personalized human activity recognition training method based on hypernetworks proposed in this invention is a federated meta-learning method that can achieve generalized modeling and personalized inference without relying on a large amount of locally labeled data. Specifically, by designing an embedding-generation integrated hypernetwork architecture, the server uniformly learns the embedding description vectors uploaded by the client to directly generate personalized model parameters, achieving efficient adaptation and generalization to multi-source, non-independent, and identically distributed data. Simultaneously, by combining the embedding network to achieve synergistic optimization of personalized modeling and global knowledge transfer, the system's robustness to non-IID characteristics and sample sparsity is enhanced. This allows the system to adapt to the data distribution with only a small amount of client-uploaded information, effectively addressing the modeling degradation problem caused by data sparsity and improving the model's generalization ability and fairness under non-IID conditions. Furthermore, this invention supports a lightweight deployment mechanism, flexibly adjusting the local fine-tuning process based on the computing resources of edge devices. Under resource-constrained conditions, personalized model parameters generated by the server can be directly used for efficient inference. The client does not need to perform an additional local fine-tuning process; it only needs to upload the pre-extracted embedding description vector, and the hypernetic network on the server can directly generate personalized model parameters that match its data distribution. This reduces the communication and computing burden of the system, improves deployment efficiency and applicability. When there is sufficient computing power, performance can be further improved through a small amount of local fine-tuning on the client side, achieving a dynamic balance between model accuracy and system efficiency, and is suitable for various edge computing scenarios. By employing centralized training of the hypernetwork and personalized model parameter generation, the client only needs to transmit lightweight embedding description vectors, effectively reducing communication bandwidth requirements and client computational load. This significantly improves system scalability and applicability, and substantially enhances the system's practicality and rapid adaptability in resource-constrained terminal environments. During training, this invention introduces a similarity consistency constraint between the embedding space and the personalized model parameter space, constraining the potential representation space of the client's personalized modeling. This effectively suppresses fluctuations in model performance during training, thereby improving the stability and accuracy of personalized model parameters generated by the hypernetwork. It effectively alleviates training interference caused by data skew or noise, improving model convergence and training stability. Furthermore, this mechanism effectively mitigates model oscillations and performance degradation caused by differences in client data distribution, enhancing the model's discriminative and generalization abilities.

[0131] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0132] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the specification and accompanying drawings, will understand and implement other variations of the disclosed embodiments in carrying out the claimed invention. In the specification, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. While certain measures are described in different embodiments, this does not mean that these measures cannot be combined to produce good results.

[0133] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1.A method for training a federated personalized human activity recognition based on a super network, characterized in that, Applied to a personalized modeling system for human activity recognition including a server and a plurality of clients, the server is deployed with an embedding network and a super network, and each client is deployed with an embedding network; The corresponding training method comprises: The server randomly selects a client and broadcasts the current embedding network parameters to the selected client; Each client collects and pre-processes local human activity data, constructs a local dataset, generates its own embedding description vector based on the local dataset and the embedding network parameters, and sends the embedding description vector to the server; The server generates corresponding personalized model parameters using the super network according to the embedding description vectors sent by all clients, and sends the personalized model parameters to each client; Each client fine-tunes the personalized model parameters sent by the server using the local dataset to obtain a personalized model parameter update, and sends the personalized model parameter update to the server; The server calculates the super network gradient and the embedding description vector update using the back propagation algorithm and according to the personalized model parameter update, and sends the embedding description vector update to each client; Each client calculates a corresponding first embedding network gradient according to the embedding description vector update, generates a corresponding second embedding network gradient according to the similarity consistency constraint between the embedding space and the personalized model parameter space, calculates the optimized embedding network gradient of the corresponding client according to the first embedding network gradient and the second embedding network gradient, and sends the optimized embedding network gradients of all clients to the server; The server aggregates all the optimized embedding network gradients, updates the parameters of the embedding network and the super network according to the super network gradient and the aggregated embedding network gradient, and tests the personalized human activity recognition according to the updated embedding network and super network. 2.The super-network based federated personalized human activity recognition training method of claim 1, wherein, The local human activity data collected and pre-processed by each client has a non-independent and identically distributed characteristic. 3.The super-network based federated personalized human activity recognition training method of claim 1, wherein, During the interaction between each client and the server, it includes: Each client constructs a client target optimization function, the server constructs a server target optimization function, and an overall target optimization function is constructed according to the client target optimization function and the server target optimization function; Based on the overall target optimization function, multiple iterations are performed until the iteration stopping condition is met. 4.The super-network based federated personalized human activity recognition training method of claim 3, wherein, The overall target optimization function is constructed, and the formula is: ； wherein, represents a client objective optimization function of the i-th client, represents a server objective optimization function, represents the number of selected clients, represents a personalized model parameter of the i-th client, represents a local dataset owned by the client, , represents an embedding description vector obtained after processing by the embedding network of the client, and then input to the super network of the server to generate the corresponding personalized model parameter, represents a regularization parameter corresponding to the personalized model parameter, represents a super network parameter, represents an embedding network parameter, , represents a regularization parameter corresponding to the super network parameter, represents a regularization parameter corresponding to the embedding network parameter, represents an L2 norm.​​​ 5.The super-network based federated personalized human activity recognition training method of claim 3, wherein, Each client calculates a corresponding first embedding network gradient according to the embedding description vector update, and the formula is: ； wherein, denotes a first embedding network gradient for the denotes a client target optimization function for the denotes an embedding network parameter, denotes denotes a partial derivative of denotes an embedding description vector for the denotes denotes a partial derivative of denotes denotes a partial derivative of .​​​​​ 6.The super-network based federated personalized human activity recognition training method of claim 1, wherein, Each client generates a corresponding second embedding network gradient according to the similarity consistency constraint between the embedding space and the personalized model parameter space, which includes: Downloading a set of historical embedding description vectors and a set of historical personalized model parameters from the server; According to the set of historical embedding description vectors, an embedding similarity matrix between all clients is calculated, and according to the set of historical personalized model parameters, a model parameter similarity matrix between all clients is calculated; According to the embedding similarity matrix and the model parameter similarity matrix, an auxiliary loss term is constructed, and a second embedding network gradient of the corresponding client is calculated according to the auxiliary loss term. 7.The super-network based federated personalized human activity recognition training method of claim 6, wherein, The constructed auxiliary loss term is represented by a formula: ； wherein, denotes an auxiliary loss, denotes the number of selected clients, denotes the embedding similarity between the th client and the th client, denotes the model parameter similarity between the th client and the th client. 8.The super-network based federated personalized human activity recognition training method of claim 6, wherein, The second embedding network gradient of the corresponding client is calculated according to the auxiliary loss term, and the formula is represented as: ； wherein, denotes the second embedding network gradient for the denotes the auxiliary loss, denotes the embedding network parameters, denotes denotes the partial derivative of denotes the partial derivative of denotes the embedding similarity matrix for all clients, denotes denotes the partial derivative of denotes the partial derivative of denotes denotes the partial derivative of denotes the partial derivative of​ 9.The super-network based federated personalized human activity recognition training method of claim 1, wherein, The optimization embedding network gradient of each client is calculated, and the formula is represented as: ； wherein, denotes the optimized embedding network gradient of the i-th client, denotes the first embedding network gradient of the i-th client, denotes the second embedding network gradient of the i-th client, denotes the first embedding network gradient of the i-th client, denotes the second embedding network gradient of the i-th client, denotes the second embedding network gradient of the i-th client, denotes the weight parameter.

Images