A model training method for human behavior recognition application based on federated learning

Abstract

The application discloses a model training method for human behavior recognition application based on federal learning, and the method comprises the following steps: acquiring parameters of a global mixed model and distributing the parameters to strong computing power equipment and weak computing power equipment; setting training parameters and estimating the maximum number of samples that can be cooperatively trained by each strong computing power equipment; acquiring a local sample loss list uploaded by a user, combining a preset threshold, and generating the number of samples selected by each weak computing power equipment to participate in cooperative training; taking the maximization of the total number of samples participating in cooperative training as a target, matching weak computing power equipment for strong computing power equipment to perform cooperative training, and obtaining a matching result; training according to the matching result to generate local model parameters; and aggregating the local model parameters to obtain final global mixed model parameters. Through the use of the application, heterogeneous equipment can adaptively train a suitable deep learning model according to the computing capacity of the equipment, and the computing resources of the equipment can be more fully utilized. The application relates to the field of model application.

Description

Technical Field

[0001] This invention relates to the field of model applications, and more particularly to a model training method for human behavior recognition applications based on federated learning. Background Technology

[0002] Human behavior recognition based on smart sensors is a key technology with wide applications in healthcare, smart homes, and augmented / virtual reality. For example, fitness enthusiasts can use human behavior recognition technology to identify and record fitness-related activities and further monitor their daily fitness progress. However, training high-quality human behavior recognition models using machine learning methods typically requires large amounts of user data. Traditional centralized training methods, which upload user data to remote cloud servers for training and storage, pose significant privacy risks. Federated learning, as a distributed machine learning framework, can effectively mitigate privacy concerns by uploading locally trained model parameters instead of raw data. Although federated learning has shown great potential in human behavior recognition applications, the heterogeneity of user devices still presents challenges in training high-precision human behavior recognition models in real-world scenarios. The first challenge is the heterogeneity of computing power across user devices participating in federated learning. For example, traditional federated learning frameworks require users to train models with the same structure; generally, training deeper and larger deep learning models can achieve more accurate predictions.

[0003] In human behavior recognition applications, smartphones are typically used as data collectors for user behavior recognition. However, the computing power of different smartphones varies significantly, and some less powerful devices lack the computational resources to train large models. Secondly, the heterogeneity in the number of devices participating in federated learning presents a challenge. In human behavior recognition scenarios, mid-to-low-end smartphones constitute the majority, sometimes several times the number of high-end or flagship smartphones. Coordinating these devices, which are significantly heterogeneous in both quantity and computing power, further complicates federated learning training. Traditional methods for addressing the heterogeneity in user device computing power and the number of device types mainly focus on how to select user devices to participate in federated learning. However, these methods fail to fully utilize the computing power of the devices and the user data. Especially in human behavior recognition applications, selecting only devices with high computing power for federated learning can lead to decreased model accuracy due to a lack of training data. Summary of the Invention

[0004] In view of this, in order to solve the technical problem that existing methods, which only select high-performance computing devices for federated learning, lead to a decrease in model accuracy due to a lack of training data, this invention proposes a model training method for human behavior recognition applications based on federated learning. The method includes the following steps:

[0005] Obtain the parameters of the global hybrid model and distribute them to high-performance and low-performance computing devices;

[0006] Set training parameters and estimate the maximum number of collaborative training samples that each high-performance computing device can handle.

[0007] Obtain the list of local sample losses uploaded by the user, and combine it with a preset threshold to generate the number of samples selected by each low-computing-power device to participate in collaborative training;

[0008] Based on the maximum number of collaborative training samples that each high-performance computing device can provide and the number of samples selected by each low-performance computing device to participate in collaborative training, with the goal of maximizing the total number of samples participating in collaborative training, high-performance computing devices are matched with low-performance computing devices for collaborative training, and the matching results are obtained.

[0009] Train the model based on the matching results to generate local model parameters;

[0010] The local model parameters are aggregated to obtain the final global hybrid model parameters.

[0011] In some embodiments, the training parameters include the training batch size of each high-performance computing device in this round of federated learning, the training latency of each batch of samples, the number of local training rounds, the training deadline of this round, the size of intermediate sample data, and the average communication bandwidth between the low-performance computing device and the high-performance computing device.

[0012] In some embodiments, the maximum number of collaborative training samples that each high-performance computing device can handle is calculated as follows:

[0013]

[0014]

[0015]

[0016]

[0017]

[0018] This represents the total training latency of the powerful computing device m during the e-th round of federated learning iterations. This represents the estimated total transmission latency of intermediate data from all weak computing devices matched by the powerful computing device m during the collaborative training process. This represents the training latency of a high-performance computing device m training local data. L represents the training latency of intermediate data from weak computing devices matched to the high-performance computing device m. mLet m represent the maximum number of training samples that a high-performance computing device can collaboratively train during this round of federated learning iterations; D represent the size of the intermediate data; and B represent the average communication bandwidth between the low-performance and high-performance computing devices. This represents the latency required for a high-performance computing device m to train each batch of samples. This represents the number of local training epochs, and S represents the size of the training batch. m This represents the number of local samples for the high-performance computing device m. This indicates the training deadline for the high-performance computing device m during this round of federated learning iterations.

[0019] In some embodiments, the step of obtaining the list of local sample losses uploaded by the user and generating the number of samples selected by each weak computing power device to participate in collaborative training, in combination with a preset threshold, specifically includes:

[0020] The second iteration of federated learning begins. Based on the list of local sample losses uploaded by users, the upper bound of the loss threshold for weak computing devices in this round of sample selection is defined according to a preset ratio.

[0021] Based on the maximum total number of collaborative training samples that all high-performance computing devices can handle, a binary search method is used to determine the lower bound of the loss threshold.

[0022] Based on the upper and lower bounds of the loss threshold, output the number of samples selected by each weak computing device to participate in collaborative training.

[0023] In some embodiments, the step of matching strong computing devices with weak computing devices for collaborative training, with the goal of maximizing the total number of samples participating in collaborative training, based on the maximum number of collaborative training samples that each strong computing device can use and the number of samples selected by each weak computing device to participate in collaborative training, and obtaining the matching result, specifically includes:

[0024] Sort the high-performance computing devices;

[0025] For each high-performance computing device, the transmission latency of samples from each low-performance computing device participating in collaborative training to the high-performance computing device and the training latency on the high-performance computing device are calculated sequentially, and the larger of the two values ​​is taken as the weight of the low-performance computing device.

[0026] The number of samples that weak computing devices participate in collaborative training is defined as value, and the cost-effectiveness of each weak computing device relative to the strong computing device is calculated in combination with the weight of the weak computing device.

[0027] Sorting low-computing-power devices based on cost-effectiveness;

[0028] For each weak computing device, a matching judgment is performed in sequence. If the total training latency of the strong computing device does not exceed its training deadline, then the weak computing device is matched with the strong computing device for collaborative training.

[0029] In some embodiments, the step of training based on the matching results specifically includes:

[0030] Based on the upper and lower bounds of the loss threshold for sample selection, the weak computing power device selects samples to participate in collaborative training and transmits the intermediate data of the samples to the corresponding strong computing power device for training.

[0031] High-performance computing devices train local samples and receive intermediate data from all matched low-performance computing devices for training.

[0032] After training, the device uploads the parameters of the local model to the cloud server.

[0033] In some embodiments, the step of aggregating local model parameters to obtain the final global hybrid model parameters specifically includes:

[0034] Based on the preset model aggregation weights, the backbone is aggregated by combining the local model parameters of all devices;

[0035] Based on the preset model aggregation weights, the first branch is aggregated using the local model parameters of the weak computing power device.

[0036] The second branch is aggregated based on the preset model aggregation weights and the local model parameters of the high-performance computing device.

[0037] In some embodiments, the formula for calculating the aggregation of the backbone is as follows:

[0038]

[0039] In the above formula, This indicates the aggregated weight of the backbone of high-performance computing equipment. This represents the backbone parameters of the local model for the high-performance computing device m during the e-th iteration. This indicates the aggregated weight of the backbone of the weak computing power equipment. This represents the backbone parameters of the local model for the weak computing device n during the e-th iteration.

[0040] Based on the above scheme, this invention provides a model training method for human behavior recognition applications based on federated learning. By designing a large-small hybrid global model, heterogeneous devices can adaptively train suitable deep learning models according to their own computing power, making fuller use of device computing resources compared to traditional device selection methods. Furthermore, this invention can develop matching algorithms for devices with heterogeneous computing power to perform collaborative training between devices, solving the problem of insufficient training data for high-computing-power devices in traditional device selection methods. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating the steps of a model training method for a human behavior recognition application based on federated learning, as described in this invention.

[0042] Figure 2 This is a structural block diagram of a model training system for a human behavior recognition application based on federated learning, according to the present invention.

[0043] Figure 3 This is a schematic diagram of the global hybrid model in a specific embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram comparing the method of this invention with ordinary federated learning; Detailed Implementation

[0045] Traditional methods for addressing the heterogeneity of user device computing power and device type primarily focus on selecting user devices to participate in federated learning. These methods fail to fully utilize device computing power and user data. Particularly in human behavior recognition applications, selecting only high-performance devices for federated learning leads to decreased model accuracy due to a lack of training data. This invention designs a large-small hybrid global model, enabling heterogeneous devices to adaptively train suitable deep learning models based on their own computing capabilities. Compared to traditional device selection methods, this approach makes more efficient use of device computing resources. Furthermore, this invention can develop a matching algorithm for devices with heterogeneous computing power to perform collaborative training, solving the problem of insufficient training data for high-performance devices in traditional device selection methods.

[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0047] It should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0048] It should be understood that the terms "system," "apparatus," "unit," and / or "module" used in this application are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0049] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "a," and / or "the" are not specifically singular and may include the plural. Generally, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements. An element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0050] In the description of the embodiments of this application, "a plurality of" refers to two or more. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0051] Furthermore, flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, the steps can be processed in reverse order or simultaneously. Additionally, other operations can be added to these processes, or one or more steps can be removed from them.

[0052] Reference Figure 1 The diagram below illustrates an optional example of a model training method for human behavior recognition applications based on federated learning proposed in this invention. This method can be applied to computer devices, and the imaging method proposed in this embodiment may include, but is not limited to, the following steps:

[0053] Step S1: Obtain the parameters of the global hybrid model and distribute them to high-performance computing devices and low-performance computing devices;

[0054] The parameters of the global hybrid model include those of the main large model and the small models. Before each round of training, the cloud server distributes the parameters of the large model and the small model to the corresponding high-performance and low-performance computing devices, respectively.

[0055] Step S2: Set training parameters and estimate the maximum number of collaborative training samples that each high-performance computing device can handle;

[0056] The training parameters include the training batch size of each high-performance computing device in this round of federated learning, the training latency of each batch of samples, the number of local training rounds, the training deadline of this round, the size of the intermediate data of the samples, and the average communication bandwidth between the low-performance computing device and the high-performance computing device.

[0057] Step S3: Obtain the list of local sample losses uploaded by the user, and combine it with a preset threshold to generate the number of samples selected by each weak computing power device to participate in collaborative training;

[0058] Among these measures, when uploading a list of local sample losses, techniques such as differential privacy can be used for encryption to protect user privacy.

[0059] Step S4: Based on the maximum number of collaborative training samples that each high-performance computing device can use and the number of samples selected by each low-performance computing device to participate in collaborative training, with the goal of maximizing the total number of samples participating in collaborative training, match high-performance computing devices with low-performance computing devices for collaborative training and obtain the matching results.

[0060] The cloud server executes a greedy device matching algorithm to match each high-computing-power device with a suitable low-computing-power device for collaborative training, maximizing the total number of samples participating in the collaborative training.

[0061] Step S5: Train the model based on the matching results to generate local model parameters;

[0062] Step S6: Aggregate the local model parameters to obtain the final global hybrid model parameters.

[0063] The cloud server aggregates the model parameters uploaded by all devices into a large-small global hybrid model.

[0064] Specifically, at the beginning of each iteration, the cloud server sends the backbone parameters of the large-small hybrid global model to each device, and sends the large model branch and small model branch of the large-small hybrid global model to the corresponding high-computing-power device and low-computing-power device respectively for local training.

[0065] In some feasible embodiments, the calculation step of estimating the maximum number of collaborative training samples per high-performance computing device includes:

[0066] High-performance computing devices can simultaneously perform local model training and receive intermediate data transmitted from low-performance computing devices.

[0067] Therefore, the total training latency of a high-performance computing device in a single round of federated learning iteration is calculated as follows:

[0068]

[0069] in, This represents the total training latency of the powerful computing device m during the e-th round of federated learning iterations. This represents the estimated total transmission latency of intermediate data from all weak computing devices matched by the powerful computing device m during the collaborative training process. This represents the training latency of a high-performance computing device m training local data. This represents the training latency of intermediate data from weak computing devices matched to the high-computing-power device m.

[0070] The calculation method is as follows:

[0071]

[0072] Among them, L m Let m represent the maximum number of collaborative training samples that the high-performance computing device m can cooperate with during this round of federated learning iterations, D represent the size of the intermediate data of the samples, and B represent the average communication bandwidth between the low-performance computing device and the high-performance computing device.

[0073]

[0074] in, This represents the latency required for a high-performance computing device m to train each batch of samples. This represents the number of local training rounds, and S represents the size of the training batch.

[0075]

[0076] Among them, s m This represents the number of local samples for each high-performance computing device m. Based on the training deadline for each high-performance computing device m in this round of federated learning iterations, the following constraints apply:

[0077]

[0078] The maximum number of collaborative training samples that each high-performance computing device can generate during this round of federated learning iterations can be estimated, where in formula (5), This indicates the training deadline for the high-performance computing device m during this round of federated learning iterations.

[0079] In some feasible embodiments, it also includes:

[0080] Starting from the first round of federated learning, the cloud server first calculates the maximum total number of collaborative training samples that all high-performance computing devices can handle. The calculation formula is as follows:

[0081]

[0082] In formula (6), M represents the total number of high-performance computing devices.

[0083] In some feasible embodiments, the step of obtaining the list of local sample losses uploaded by the user and generating the number of samples selected by each weak computing power device to participate in collaborative training, in combination with a preset threshold, specifically includes:

[0084] In this process, since the first round of federated learning does not have loss information of local samples from weak computing devices, each weak computing device selects all local samples to participate in collaborative training in the first round.

[0085] The second iteration of federated learning begins. Based on the list of local sample losses uploaded by users, the upper bound of the loss threshold for weak computing devices in this round of sample selection is defined according to a preset ratio.

[0086] Specifically, the loss value of 90% is defined as the upper bound of the loss threshold for weak computing power devices in this round of sample selection. Samples above the upper bound of the loss are considered noise.

[0087] Based on the maximum total number of collaborative training samples that all high-performance computing devices can handle, a binary search method is used to determine the lower bound of the loss threshold.

[0088] Specifically, according to Use a binary search method to determine the lower bound of the loss threshold.

[0089] Based on the upper and lower bounds of the loss threshold, output the number of samples selected by each weak computing device to participate in collaborative training.

[0090] In some feasible embodiments, the step of matching strong computing devices with weak computing devices for collaborative training, with the goal of maximizing the total number of samples participating in collaborative training, based on the maximum number of collaborative training samples that each strong computing device can use and the number of samples selected by each weak computing device to participate in collaborative training, and obtaining the matching result, specifically includes:

[0091] First, sort the high-computing-power devices in non-ascending order;

[0092] For each high-performance computing device, calculate the transmission latency of samples from each low-performance computing device participating in collaborative training to that high-performance computing device and the training latency on that high-performance computing device, and select the larger of the two values ​​as the weight w of that low-performance computing device. n , where n is the number of the weak computing device.

[0093] The number of samples participating in collaborative training from devices with weak computing power is defined as the value p.n Calculate the cost-effectiveness p of each weak computing power device relative to the strong computing power device. n / w n We sorted the devices with weak computing power in a non-ascending order based on their cost-effectiveness.

[0094] Each weak computing device is matched sequentially. If a weak computing device can be matched with a strong computing device, that is, if the total training latency of the strong computing device does not exceed its training deadline, then the weak computing device is matched with the strong computing device for collaborative training.

[0095] In some feasible embodiments, the step of training based on the matching results specifically includes:

[0096] During training, the low-computing-power device selects a loss threshold based on the samples. and Samples are selected for collaborative training, and their intermediate data is then transmitted to the corresponding high-performance computing devices for training. The high-performance devices, while training their local samples, receive intermediate data from all matched low-performance computing devices and perform training accordingly. The model parameters used by the high-performance devices in this round of training are... This indicates that the model parameters used by the weak computing devices in this round of training were... This indicates that after training is complete, the device uploads the parameters of the local model to the cloud server.

[0097] In some feasible and reasonable cases, the step of aggregating local model parameters to obtain the final global hybrid model parameters specifically includes:

[0098] The cloud server aggregates the model parameters uploaded from all devices into a large-small global hybrid model. The aggregation process consists of three parts:

[0099] S6.1 Aggregation of the backbone of the large-small hybrid global model: The backbone of the local model parameters of all devices participates in the aggregation.

[0100] The weights for model aggregation are calculated as follows:

[0101]

[0102] The training data for high-performance computing devices includes local data and intermediate data sent by all matched low-performance computing devices.

[0103] Calculate the parameters of the backbone of the large-small hybrid global model using the weights of each device parameter:

[0104]

[0105] in This represents the backbone parameters of the local model for the high-performance computing device m during the e-th iteration. This represents the backbone parameters of the local model for the weak computing device n during the e-th iteration.

[0106] S6.2 Aggregation of small model branches in the large-small hybrid global model: Small model branches of local model parameters of all weak computing power devices participate in aggregation.

[0107] The weights for model aggregation are calculated as follows:

[0108]

[0109] Calculate the parameters of the small model branch in the large-small hybrid global model using the weights of each device parameter:

[0110]

[0111] in This represents the parameters of the small model branch of the local model during the e-th iteration of the weak computing device n.

[0112] S6.3 Aggregation of Large-Small Hybrid Global Model Branches: All local model parameters from high-performance computing devices are aggregated from the large model branches. The weight calculation method for model aggregation is as follows:

[0113]

[0114] The training data for high-performance computing devices includes local data and intermediate data sent by all matched low-performance computing devices.

[0115] Calculate the parameters of the large-small hybrid global model branch using the weights of each device parameter:

[0116]

[0117] in This represents the parameters of the large model branch of the local model during the e-th iteration of the high-performance computing device m.

[0118] In human behavior recognition applications, smartphones are typically used as data collectors for user behavior data. However, the computing power of different smartphones varies significantly, with some less powerful devices lacking the computational resources to train large models. Secondly, the heterogeneity in the number of devices participating in federated learning presents a challenge. In human behavior recognition scenarios, mid-to-low-end smartphones constitute the majority, sometimes several times the number of high-end or flagship smartphones. Coordinating these devices, which are significantly heterogeneous in both quantity and computing power, further complicates the training of federated learning.

[0119] To address the aforementioned challenges, this invention utilizes BranchyNet technology to design a large-small hybrid global model. This model consists of large and small branch models, with branches sharing a common backbone. Model aggregation is divided into three parts: backbone aggregation, small model aggregation, and large model aggregation. The structure and aggregation process of the large-small hybrid global model are as follows: Figure 3 As shown. The backbone consists of convolutional layers and max-pooling layers, used to extract features from human behavior recognition data samples. The small model part is a lightweight classifier composed of fully connected layers. The large model part includes deeper convolutional layers and a more powerful classifier. User devices with limited computing power can train the small model, while user devices with sufficient computing power can train the large model. The device sends the locally trained model parameters to the cloud server, and the cloud server processes them according to... Figure 3 The method aggregates the parameters of each part of the user model separately.

[0120] Furthermore, due to the limited number of high-performance computing devices, large models often suffer from low accuracy due to a lack of training data. To address this issue, smaller models can transmit intermediate data (feature maps) generated during the forward pass to high-performance computing devices for collaborative training of the larger model. Since the original data is difficult to recover from the intermediate data, user privacy is protected.

[0121] In practical applications, intermediate data can be encrypted using various advanced privacy protection technologies such as differential privacy, which can further protect users' data privacy.

[0122] Limited by their own computing resources and bandwidth, weak computing devices can only transmit intermediate sample data to one strong computing device at a time, while strong computing devices can simultaneously receive intermediate sample data from multiple weak computing devices. To improve the efficiency of collaborative training between devices, this invention utilizes a loss-based sample selection method to screen samples from weak computing devices. Based on the principle that samples with higher loss values ​​are considered to contribute more to the gradient of model parameters during training, weak computing devices select samples with higher loss values ​​to participate in collaborative training, thereby maximizing collaborative training efficiency under the constraint of the training deadline of strong computing devices. Specifically, after each round of local training, each weak computing device uploads the sample loss value and model parameters generated during training to a cloud server. After estimating the maximum number of collaborative training samples that all strong computing devices can use in this round, the cloud server uses a customized loss threshold to allow weak computing devices to select samples for collaborative training. Based on the loss threshold, the number of samples for each weak computing device to participate in collaborative training is calculated, and then a device matching algorithm is customized for each strong computing device to match suitable weak computing devices for collaborative training.

[0123] This invention also provides simulation experiments:

[0124] (1) Obtain publicly available datasets for human behavior recognition, such as the Heterogeneity Human Activity Recognition (HHAR) dataset.

[0125] (2) Pack every 100 3-axis gyroscope devices into a sample, and the feature dimension of each sample is 3*100.

[0126] (3) The Nvidia Jeson TX2 and Nvidia Jeson AGX Xavier were used to simulate the user's low-power and high-power computing devices, respectively. The simulation environment parameters for this embodiment are shown in Table 1 below:

[0127] Table 1 Simulation Experimental Environment Parameters

[0128] parameter value Inter-device bandwidth [10,50]Mbps Large Model 3 convolutional layers + 3 fully connected layers Small model 1 convolutional layer + 2 fully connected layers Optimizer Stochastic Gradient Descent Learning rate 0.01 Batch size 64 Local training rounds 1

[0129] (4) The device performs collaborative training in each round of federated learning iteration according to the above algorithm.

[0130] Additionally, refer to Figure 4 The paper presents a comparison chart of the strong-weak collaborative federated learning method of the present invention with that of ordinary federated learning in terms of model accuracy, which shows that the present invention is superior.

[0131] like Figure 2 As shown, a model training system for human behavior recognition applications based on federated learning includes:

[0132] The distribution module is used to obtain the parameters of the global hybrid model and distribute them to high-performance computing devices and low-performance computing devices.

[0133] The first computing module is used to set training parameters and estimate the maximum number of collaborative training samples that each high-performance computing device can handle.

[0134] The second calculation module is used to obtain the list of local sample losses uploaded by the user, and combine it with a preset threshold to generate the number of samples selected by each weak computing device to participate in collaborative training.

[0135] The device matching module is used to match strong computing devices with weak computing devices for collaborative training based on the maximum number of collaborative training samples that each strong computing device can use and the number of samples selected by each weak computing device to participate in collaborative training, with the goal of maximizing the total number of samples participating in collaborative training, and to obtain the matching results.

[0136] The matching training module is used to train the model based on the matching results and generate local model parameters.

[0137] The aggregation module is used to aggregate local model parameters to obtain the final global hybrid model parameters.

[0138] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0139] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A model training method for human behavior recognition applications based on federated learning, characterized in that, Includes the following steps: Obtain the parameters of the global hybrid model and distribute them to high-performance and low-performance computing devices; Set training parameters and estimate the maximum number of collaborative training samples that each high-performance computing device can handle. Obtain the list of local sample losses uploaded by the user, and combine it with a preset threshold to generate the number of samples selected by each low-computing-power device to participate in collaborative training; Based on the maximum number of collaborative training samples that each high-performance computing device can provide and the number of samples selected by each low-performance computing device to participate in collaborative training, with the goal of maximizing the total number of samples participating in collaborative training, high-performance computing devices are matched with low-performance computing devices for collaborative training, and the matching results are obtained. Train the model based on the matching results to generate local model parameters; The process of aggregating local model parameters to obtain final global hybrid model parameters, and then generating the number of samples selected for collaborative training by each low-computing-power device based on a preset threshold, specifically includes: The second iteration of federated learning begins. Based on the list of local sample losses uploaded by users, the upper bound of the loss threshold for weak computing devices in this round of sample selection is defined according to a preset ratio. Based on the maximum total number of collaborative training samples that all high-performance computing devices can handle, a binary search method is used to determine the lower bound of the loss threshold. Based on the upper and lower bounds of the loss threshold, output the number of samples selected by each weak computing device to participate in collaborative training. The step of matching strong computing devices with weak computing devices for collaborative training, based on the maximum number of collaborative training samples that each strong computing device can use and the number of samples selected by each weak computing device to participate in collaborative training, with the goal of maximizing the total number of samples participating in collaborative training, and obtaining the matching result, specifically includes: Sort the high-performance computing devices; For each high-performance computing device, the transmission latency of samples from each low-performance computing device participating in collaborative training to the high-performance computing device and the training latency on the high-performance computing device are calculated sequentially, and the larger of the two values ​​is taken as the weight of the low-performance computing device. The number of samples that a weak computing device participates in collaborative training is defined as its value. The cost-effectiveness of each weak computing device relative to a strong computing device is calculated by combining the weight of the weak computing device with its weight. This cost-effectiveness is the ratio of the number of samples that the device participates in collaborative training to the weight of the weak computing device. Sorting low-computing-power devices based on cost-effectiveness; For each weak computing device, a matching judgment is performed in sequence. If the total training latency of the strong computing device does not exceed its training deadline, then the weak computing device is matched with the strong computing device for collaborative training.

2. The model training method for human behavior recognition applications based on federated learning according to claim 1, characterized in that, The training parameters include the training batch size of each high-performance computing device in this round of federated learning, the training latency of each batch of samples, the number of local training rounds, the training deadline of this round, the size of intermediate sample data, and the average communication bandwidth between the low-performance computing device and the high-performance computing device.

3. The model training method for human behavior recognition applications based on federated learning according to claim 2, characterized in that, The maximum number of collaborative training samples that each high-performance computing device can handle is calculated as follows: This represents the total training latency of the powerful computing device m during the e-th round of federated learning iterations. This indicates the high-performance computing equipment used in the estimated collaborative training process. The total transmission latency of intermediate data from all matched weak computing devices. Indicates high-performance computing equipment Training latency for training local data. Indicates high-performance computing equipment Training latency of intermediate data from the matched low-computing-power devices. Indicates high-performance computing equipment The maximum number of collaboratively trained samples during this round of federated learning iterations. Indicates the size of the intermediate data in the sample. This represents the average communication bandwidth between low-computing-power devices and high-computing-power devices. Indicates high-performance computing equipment The latency required to train each batch of samples Indicates the number of local training rounds. Indicates the size of the training batch. Indicates high-performance computing equipment The number of local samples, Indicates high-performance computing equipment The training deadline in this round of federated learning iterations.

4. The model training method for human behavior recognition applications based on federated learning according to claim 1, characterized in that, The step of training based on the matching results specifically includes: Based on the upper and lower bounds of the loss threshold for sample selection, the weak computing power device selects samples to participate in collaborative training and transmits the intermediate data of the samples to the corresponding strong computing power device for training. High-performance computing devices train local samples and receive intermediate data from all matched low-performance computing devices for training. After the specified number of training rounds are completed locally, the device uploads the parameters of the local model to the cloud server.

5. The model training method for human behavior recognition applications based on federated learning according to claim 4, characterized in that, The step of aggregating local model parameters to obtain the final global hybrid model parameters specifically includes: Based on the preset model aggregation weights, the backbone is aggregated by combining the local model parameters of all devices; Based on the preset model aggregation weights, the first branch is aggregated by combining the local model parameters of the weak computing power device; The second branch is aggregated based on the preset model aggregation weights and the local model parameters of the high-performance computing device.

6. The model training method for human behavior recognition applications based on federated learning according to claim 5, characterized in that, The formula for calculating the aggregation of the main part is as follows: In the above formula, This indicates the aggregated weight of the backbone of high-performance computing equipment. This represents the backbone parameters of the local model during the e-th iteration of the high-performance computing device m. This indicates the aggregated weight of the backbone of the weak computing power equipment. This represents the backbone parameters of the local model for the weak computing device n during the e-th iteration.

Images