Resource-aware model training methods, devices, and computer equipment

By using a resource-aware approach, subnet training is optimized based on the resource information of IoT devices, solving the problem of device resource mismatch in federated neural network architecture search, improving training stability and efficiency, and adapting to dynamic resource changes.

CN123114775APending Publication Date: 2026-11-17TSINGHUA UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611587658.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-14
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In the search of federated neural network architectures, fluctuations in the resource status of IoT devices can lead to a mismatch between the subnet models allocated in the cloud and the device resources, affecting training stability.

Method used

By acquiring resource information of IoT devices, the upper limit of their trainable parameters is determined, and the target subnet is optimized based on this upper limit. The optimized subnet is then sent for training to ensure that the devices do not exceed their capacity.

Benefits of technology

It improves the training stability and efficiency of IoT devices, reduces repetitive training and communication overhead, and adapts to scenarios with dynamically changing resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123114775A_ABST
    Figure CN123114775A_ABST
Patent Text Reader

Abstract

This application relates to the technical field of artificial intelligence, and in particular to a resource-aware model training method, apparatus, and computer device. The method is applied to a server and includes: determining a target subnet for each IoT device based on a candidate list corresponding to the current training stage; the candidate list includes N candidate subnets of different scales determined from a shared supernet, where N is the number of IoT devices participating in the training stage, and the shared supernet includes multiple subnets of different scales; for each IoT device, acquiring the resource information of the IoT device, and determining the upper limit of trainable parameters based on the resource information; optimizing the target subnet according to the parameter upper limit to obtain an optimized subnet; and sending the optimized subnet to the IoT device for training. Using the scheme of this application, in scenarios of cloud-edge collaborative federated model training, the stability of local device training can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a resource-aware model training method, apparatus, and computer equipment. Background Technology

[0002] Federated Neural Architecture Search (FedNAS) is a method for collaboratively training neural networks using multiple IoT devices. This method can obtain high-performance neural networks through collaborative training of IoT devices without sharing the raw data of each IoT device, and then deploy the trained neural networks specifically for each IoT device.

[0003] In related technologies, the collaborative training process of federated neural network architecture search is as follows: a supernet model containing multiple scale subnets is maintained by the cloud. In each training round, the cloud allocates a subnet model to each local device. After the IoT device trains the subnet model using local data, it uploads the parameters of the trained subnet model to the cloud. Then, the cloud further updates the parameters of the supernet model, and after multiple rounds, the training of the cloud supernet is finally completed.

[0004] In fact, the resource status of IoT devices fluctuates; however, in the collaborative training phase of related technologies, a polling or random strategy is used to assign subnet models to IoT devices in each round of training. This leads to the possibility that the subnet models assigned to IoT devices by the cloud may not match the resource status of the IoT devices, resulting in insufficient stability of IoT device training.

[0005] Therefore, in scenarios where cloud-edge collaborative federated training models are conducted, improving the stability of IoT device training is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] Therefore, it is necessary to provide a resource-aware model training method, device, and computer equipment that can improve the stability of local device training in a scenario of cloud-edge collaborative federated model training, addressing the aforementioned technical issues.

[0007] Firstly, this application provides a resource-aware model training method applied to a server, the method comprising:

[0008] Based on the candidate list corresponding to the current training phase, the target subnet corresponding to each IoT device is determined; the candidate list includes N candidate subnets of different scales determined from the shared supernet, where N is the number of IoT devices participating in the training in the current training phase, and the shared supernet includes multiple subnets of different scales;

[0009] For each IoT device, obtain the resource information of the IoT device, and determine the upper limit of the trainable parameters of the IoT device based on the resource information;

[0010] Based on the upper limit of the parameters, the target subnet is optimized to obtain an optimized subnet;

[0011] The optimized subnet is sent to the IoT device for training.

[0012] In one embodiment, the resource information includes specification information and communication status parameters, and determining the upper limit of trainable parameters for the IoT device based on the resource information includes:

[0013] The effective uplink rate of the IoT device is determined based on the communication status parameters.

[0014] Based on the energy consumption limit of the IoT device, the effective uplink rate, and the specification information, the upper limit of the parameters that the IoT device can be trained is determined.

[0015] In one embodiment, optimizing the target subnet according to the upper limit of the parameters to obtain the optimized subnet includes:

[0016] The parameter amplitudes of each reference parameter in the target subnet are obtained, and the reference parameters are sorted according to the parameter amplitudes to obtain a parameter sequence; the parameter amplitudes characterize the importance of the parameter in the subnet.

[0017] Based on the parameter upper limit and the parameter sequence, a list of valid parameters is determined; the list of valid parameters includes the parameters used by the IoT device for training.

[0018] The target subnet is optimized based on the list of valid parameters to obtain the optimized subnet.

[0019] In one embodiment, the method further includes:

[0020] In the current training phase, N-2 subnets are determined as supplementary subnets from the subnets in the shared supernet, excluding the edge subnets; the edge subnets include subnets with upper and lower scale limits;

[0021] Based on the edge subnet and each of the supplementary subnets, the candidate list corresponding to the current training phase is constructed.

[0022] In one embodiment, the method further includes:

[0023] Receive training parameter sets fed back by each of the IoT devices, the training parameter sets including parameters in the optimized subnet after being trained by the IoT devices;

[0024] By aggregating the training parameter sets, a first parameter set and a second parameter set are obtained; the first parameter set consists of the updated parameters in the subnet with the upper scale limit, and the second parameter set consists of the updated parameters in the remaining subnets.

[0025] Based on the first parameter set and the second parameter set, and the updated weight allocation relationship, the parameters in the shared supernet are updated.

[0026] Secondly, this application also provides a resource-aware model training method, executed by an IoT device, the method comprising:

[0027] Acquire resource information and send the resource information to the server; the resource information represents the hardware specifications and communication status of the IoT device;

[0028] The system receives an optimized subnet sent by the server and trains based on local data and the optimized subnet. The optimized subnet is obtained by the server by optimizing the target subnet according to the upper limit of trainable parameters of the IoT device. The upper limit of parameters is determined by the server based on the resource information, and the target subnet is determined by the server in the candidate list corresponding to the current round.

[0029] In one embodiment, the training based on local data and the optimized subnet includes:

[0030] In the nth round, the optimized subnet is trained based on local data and a list of effective parameters; the list of effective parameters is determined by the server based on the parameter upper limit and the parameter sequence, the parameter sequence is obtained by arranging the baseline parameters in the target subnet according to the parameter amplitude, and the parameter amplitude characterizes the importance in the subnet; n=1;

[0031] In the nth round, the effective parameter list is updated based on the updated parameter magnitudes of each of the baseline parameters in the optimized subnet obtained from the (n-1)th round of training;

[0032] The optimized subnet is trained based on the updated list of valid parameters and the local data.

[0033] Thirdly, this application also provides a resource-aware model training device, the device comprising an allocation module, a parameter upper limit determination module, an optimization module, and a transmission module, wherein:

[0034] The allocation module is used to determine the target subnet corresponding to each IoT device according to the candidate list corresponding to the current training stage; the candidate list includes N candidate subnets of different scales determined from the shared supernet, where N is the number of IoT devices, and the shared supernet includes multiple subnets of different scales;

[0035] The parameter upper limit determination module is used to obtain resource information of each IoT device and determine the upper limit of trainable parameters of the IoT device based on the resource information.

[0036] The optimization module is used to optimize the target subnet according to the upper limit of the parameters to obtain an optimized subnet;

[0037] The sending module is used to send the optimized subnet to the IoT device for training.

[0038] Fourthly, this application also provides a resource-aware model training device, the device comprising a parameter uploading module and a training module, wherein:

[0039] The parameter upload module is used to acquire resource information and send the resource information to the server; the resource information represents the hardware specifications and communication status of the IoT device.

[0040] The training module is used to receive the optimized subnet sent by the server and perform training based on local data and the optimized subnet; the optimized subnet is obtained by the server by optimizing the target subnet according to the upper limit of trainable parameters of the IoT device; the upper limit of parameters is determined by the server according to the resource information, and the target subnet is determined by the server in the candidate list corresponding to the current round.

[0041] Fifthly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any step in the resource-aware model training method as described in the first aspect above, or any step in the resource-aware model training method as described in the second aspect above.

[0042] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any step in the resource-aware model training method as described in the first aspect above, or any step in the resource-aware model training method as described in the second aspect above.

[0043] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, implements any step in the resource-aware model training method as described in the first aspect above, or any step in the resource-aware model training method as described in the second aspect above.

[0044] The aforementioned resource-aware model training methods, devices, and computer equipment, under optimal communication conditions, have an upper limit on the trainable parameters of the IoT device that depends on the device's specifications; conversely, under conditions with strong specifications, the upper limit depends on the device's real-time communication status. Therefore, determining the upper limit of trainable parameters based on both communication status parameters and specifications yields a more balanced result. After determining the current upper limit of trainable parameters for the IoT device, this limit is compared with the scale (parameter size) of the target subnet. If the comparison indicates that the scale (parameter size) of the target subnet exceeds the upper limit, the target subnet needs to be optimized. The optimized subnet is then sent to the IoT device for training. This prevents the IoT device from training a subnet exceeding its parameter capacity, improving both training efficiency and stability. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a diagram illustrating the application environment of a resource-aware model training method in one embodiment.

[0047] Figure 2 This is a flowchart illustrating a resource-aware model training method executed by a server in one embodiment.

[0048] Figure 3 This is a flowchart illustrating the steps for determining the upper limit of parameters in one embodiment;

[0049] Figure 4 This is a flowchart illustrating the optimized processing steps in one embodiment;

[0050] Figure 5 This is a flowchart illustrating a resource-aware model training method performed by an IoT device in one embodiment.

[0051] Figure 6 This is a structural block diagram of a resource-aware model training device in one embodiment;

[0052] Figure 7 This is a structural block diagram of a resource-aware model training device in another embodiment;

[0053] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0056] The resource-aware model training method provided in this application can be applied to, for example... Figure 1 In the application environment shown, each IoT device communicates with the server via a network. The server is configured with a shared supernet containing subnets of multiple scales. The training task for the IoT devices includes multiple training phases. In each training phase, the server selects a subnet for training from the shared supernet for each IoT device and distributes it to the corresponding IoT device for training. After receiving the subnet distributed by the server, the IoT device trains the subnet for multiple rounds based on its local data. After meeting the training requirements of the current training phase, it uploads the trained subnet or its parameters to the server. The server aggregates the parameters of the trained subnets of each IoT device to update the parameters in the shared supernet. Furthermore, the server ends the current training phase and enters the next training phase or terminates the training task.

[0057] The Internet of Things (IoT) devices can include, but are not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, and portable wearable devices. They can also include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays; head-mounted displays can include virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services.

[0058] In one exemplary embodiment, such as Figure 2 As shown, a resource-aware model training method is provided, which is then applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 10-40, where:

[0059] Step 10: Determine the target subnet for each IoT device based on the candidate list corresponding to the current training phase.

[0060] In the embodiments of this application, a shared supernet is maintained in advance on the server. The shared supernet includes multiple subnets of different scales, and a subnet list (architectural space) is constructed to retrieve each subnet in the shared supernet. The subnet list contains the unique identifier of each subnet and the scale (parameter scale) corresponding to the subnet.

[0061] The model training method includes multiple training phases. In each phase, the server determines a candidate list from the shared supernet. This candidate list includes N candidate subnets of different scales, where N is the number of IoT devices participating in the training phase. The candidate list contains the subnets that each IoT device needs to train in the current training phase. Then, based on an allocation strategy, a corresponding target subnet is assigned to each IoT device. The concept of scale describes the number of baseline parameters or network layers in the subnet. In one example, a larger subnet scale corresponds to a larger number of baseline parameters and / or network layers.

[0062] Furthermore, the allocation strategy can be a random allocation strategy, or it can be allocated according to the scale of the target subnet that each IoT device has been allocated in the previous training phase.

[0063] Step 20: For each IoT device, obtain the resource information of the IoT device, and based on the resource information, determine the upper limit of the parameters that the IoT device can train.

[0064] In the embodiments of this application, resource information includes specification information and communication status parameters; specification information characterizes the hardware performance of the IoT device, and communication status parameters characterize the IoT device's ability to transmit data.

[0065] When communication conditions are favorable, the upper limit of trainable parameters for an IoT device depends on the device's specifications; conversely, when specifications are strong, the upper limit depends on the device's real-time communication status. Therefore, determining the upper limit of trainable parameters for an IoT device based on both communication status parameters and specifications yields a more balanced result.

[0066] The specifications of each IoT device can be pre-stored on the server, allowing for direct scheduling. Alternatively, these specifications can be uploaded to the server by the IoT devices during the initial training phase. The server receives this information and can directly access it in subsequent training phases. Communication status parameters, however, are time-sensitive. Therefore, they are fed back to the server by the IoT devices in response to communication status retrieval requests. In other words, at the start of each training phase, the server sends communication status retrieval requests to each IoT device to obtain its latest, up-to-date communication status parameters.

[0067] Step 30: Optimize the target subnet according to the upper limit of the parameters to obtain the optimized subnet.

[0068] Step 40: Send the optimized subnet to the IoT device for training.

[0069] In this embodiment of the application, for each IoT device, after determining the target subnet corresponding to the IoT device, the scale (parameter size) of the target subnet is obtained from the subnet list based on the unique identifier of the target subnet, and the upper limit of the trainable parameters of the IoT device is compared with the scale (parameter size) of the target subnet. If the comparison result indicates that the scale (parameter size) of the target subnet is greater than the upper limit of parameters, then the target subnet needs to be optimized. If the comparison result indicates that the scale (parameter size) of the target subnet is less than or equal to the upper limit of parameters, then the target subnet is directly sent to the IoT device for training.

[0070] In the resource-aware model training method described above, under optimal communication conditions, the upper limit of trainable parameters for an IoT device depends on the device's specifications; conversely, under strong specifications, the upper limit depends on the device's real-time communication status. Therefore, determining the upper limit of trainable parameters based on both communication status parameters and specifications yields a more balanced result. After determining the current upper limit of trainable parameters for the IoT device, it is compared with the scale (parameter size) of the target subnet. If the comparison indicates that the scale (parameter size) of the target subnet exceeds the upper limit, the target subnet needs to be optimized. The optimized subnet is then sent to the IoT device for training. This prevents the IoT device from training a subnet exceeding its parameter capacity, improving both training efficiency and stability.

[0071] To ensure that subnets of different sizes within the shared supernet have training opportunities, the scheme in this application retains the largest and smallest subnets during the construction of the candidate list at each training stage. Simultaneously, several random subnets are randomly sampled from the shared subnet list as supplementary subnets. These supplementary subnets can be non-uniformly sampled based on their parameter size, allowing subnets with larger parameter sizes and relatively difficult training to receive a higher sampling opportunity.

[0072] The specific implementation involves determining the candidate list for each training phase. This process may include: in the current training phase, selecting N-2 subnets from the subnets in the shared supernet (excluding the edge subnets) as supplementary subnets; and constructing the candidate list corresponding to the current training phase based on the edge subnets and each supplementary subnet.

[0073] Specifically, the edge subnets include subnets with upper and lower scale limits. That is, in each training phase, the largest and smallest subnets in the shared supernet will inevitably be assigned to IoT devices for training. This is because the smallest subnet provides a faster training rate, while the largest subnet provides higher training accuracy; therefore, by ensuring that both the largest and smallest subnets participate in training at each phase, training efficiency can be achieved without sacrificing the accuracy of the final trained model.

[0074] Specifically, the candidate list can be represented by formula (1):

[0075] , formula (1);

[0076] in, For the first The set of candidate subnets in the candidate list of the round. and These represent the largest and smallest subnets, respectively. This represents the set of supplementary subnets sampled from the subnet list (architecture space).

[0077] Furthermore, after determining the candidate list for the current stage, step 10, the process of assigning a corresponding target subnet to each IoT device, includes: obtaining the cumulative allocation score for each IoT device, and determining the target subnet corresponding to each IoT device based on the cumulative allocation score. Specifically, for each IoT device, its cumulative allocation score is determined based on the scale of each target subnet actually assigned to the IoT device during the historical training phases.

[0078] Specifically, a base allocation score is pre-set for each IoT device. At the end of any training phase, for each IoT device, a corresponding allocation score is determined based on the scale of the target subnet assigned to the IoT device in that training phase. The relationship between the subnet scale and the allocation score is pre-defined, and the allocation score is positively correlated with the scale; that is, the larger the subnet scale, the larger the corresponding allocation score. The cumulative score of an IoT device is the sum of its base allocation score and the allocation scores determined in each training round.

[0079] By accumulating scores from IoT device allocations, the scale of subnets trained by IoT devices during historical training phases can be determined. In this application's scheme, the allocation principle is to prioritize assigning edge subnets to IoT devices that have not been trained or have had fewer training iterations (accumulated classification scores in the middle range). This subnet-aware allocation method avoids long-term lack of training for some IoT devices or edge subnets and improves the coverage of shared supernets for subnets of different sizes.

[0080] The core logic for determining the upper limit of trainable parameters for IoT devices is to uniformly convert communication, computing, and energy conditions into a budget of trainable parameters. In one embodiment, such as... Figure 3 As shown, step 20: the process of determining the upper limit of trainable parameters for IoT devices based on resource information, may specifically include steps 21 and 22, wherein:

[0081] Step 21: Determine the effective uplink rate of the IoT device based on the communication status parameters;

[0082] Step 22: Determine the upper limit of trainable parameters for IoT devices based on their energy consumption limit, effective uplink speed, and specifications.

[0083] Specifically, communication status parameters include uplink bandwidth, transmit power, channel gain, and noise conditions. Based on these communication status parameters, the computing IoT device is determined. In the The effective uplink rate for each training phase is calculated as shown in formula (2).

[0084] , formula (2);

[0085] in, For effective uplink speed, To be assigned to IoT devices Uplink bandwidth, Transmit power for IoT devices, Let be the channel gain for the t-th training phase. This represents the noise power spectral density.

[0086] Let the number of active parameters actually trained by the IoT device in the current training phase be . The local energy consumption can be expressed by formula (3):

[0087] Formula (3); where, To calculate energy consumption, For the effective switching capacitor coefficient, For the CPU frequency of IoT devices, The number of computation cycles required to process a parameter.

[0088] The communication energy consumption generated by IoT devices uploading corresponding parameter updates can be expressed by formula (4);

[0089] Formula (4); where, For communication energy consumption, Update the corresponding transmission bit overhead for each parameter; the meanings of the remaining symbols are the same as described above.

[0090] The total energy consumption of IoT devices in the current round consists of computing energy consumption and communication energy consumption, and its structure is represented by formula (5);

[0091] , formula (5);

[0092] in, For IoT devices In the The total energy consumed by performing local training computations and uploading trained subnets in each training phase.

[0093] The available energy of IoT devices per round does not exceed Under the given conditions, the trainable parameter budget for each round can be expressed by formula (6);

[0094] , formula (6);

[0095] in, Indicates Internet of Things (IoT) devices In the The upper limit (in terms of number) of parameters that can be trained in a training phase. This indicates the energy budget available for the current training phase of the IoT device.

[0096] After obtaining the communication status parameters corresponding to each IoT device, the upper limit of trainable parameters for each IoT device can be determined according to the above formulas (2)-(6); after determining the target subnet corresponding to each IoT device according to the allocation strategy, for each IoT device, the server sets the upper limit of trainable parameters for the IoT device. The parameter size of the target subnet is compared with that of the target subnet. If the comparison result indicates that the scale (parameter size) of the target subnet is greater than the upper limit of the parameters, the target subnet needs to be optimized. The optimized subnet is then sent to the IoT device for training. If the comparison result indicates that the scale (parameter size) of the target subnet is less than or equal to the upper limit of the parameters, the target subnet is directly sent to the IoT device for training.

[0097] In this embodiment, when the target subnet allocated to the IoT device exceeds the upper limit of the parameters that the IoT device can train, the target subnet is not directly pruned to delete parameters. Instead, the selection is based on the importance of each benchmark parameter in the target subnet, and only some benchmark parameters whose importance ranking is within the parameter upper limit range are allowed to participate in training.

[0098] In one embodiment, such as Figure 4 As shown, in step 30, the target subnet is optimized based on the upper limit of the parameters to obtain the optimized subnet. This process can specifically include steps 31-33, where:

[0099] Step 31: Obtain the parameter magnitudes of each reference parameter in the target subnet, and sort the reference parameters according to their magnitudes to obtain the parameter sequence;

[0100] Step 32: Determine the list of valid parameters based on the parameter upper limit and parameter sequence; the list of valid parameters includes all parameters used by the IoT device for training.

[0101] Step 33: Optimize the target subnet based on the list of valid parameters to obtain the optimized subnet.

[0102] Specifically, parameter magnitude represents the importance / priority of a parameter within a subnet; a larger magnitude indicates greater importance / priority. For the target subnet, the baseline parameters are sorted in descending order of magnitude to obtain a parameter sequence. An optimization threshold is then determined based on the upper limit of the parameter values. Then, based on the upper limit of trainable parameters for IoT devices, the Top-K baseline parameters are determined as effective parameters, and an effective parameter list is constructed based on each effective parameter. Here, Top-K represents the top K baseline parameters in the parameter sequence; that is, the parameter magnitudes of the top-K baseline parameters in the parameter sequence are all greater than or equal to the optimization threshold. The baseline parameters not included in the effective parameter list all have parameter amplitudes smaller than the optimization threshold. The list of valid parameters contains the parameters actually used by the IoT devices for training, and the Top-K valid parameters do not exceed the upper limit of the parameters corresponding to the IoT devices.

[0103] The process of optimizing the target subnet based on the effective parameter list refers to generating a binary threshold mask for each reference parameter in the target subnet, and multiplying each reference parameter in the target subnet element by element with the corresponding binary threshold mask to obtain the optimized subnet.

[0104] Specifically, the process of generating the mask is shown in formula (7);

[0105] , formula (7);

[0106] in, Indicates parameters The corresponding threshold mask, This indicates that it is a valid parameter that can be used in the training of IoT devices; This indicates that it cannot participate in the training of IoT devices.

[0107] The process of generating the optimized subnet is shown in formula (8);

[0108] , formula (8);

[0109] in, Indicates allocation to IoT devices The target subnet, This represents a binary threshold mask calculated based on the upper limit and magnitude of the parameters. This indicates element-wise multiplication; This indicates that the number of active parameters in the extracted optimized subnet does not exceed that of IoT devices. The upper limit of trainable parameters in the current training phase. .

[0110] In one embodiment, when the server distributes the optimized subnet to the corresponding IoT device for training, the IoT device trains the optimized subnet based on its stored local data, which includes multiple training rounds. However, if the effective parameter list of the first training round is used throughout the entire training process, some parameters whose magnitudes are close to the optimization threshold still need to be trained even if they become less important after training.

[0111] To further optimize the stability and accuracy of training, the scheme in this application optimizes the logic of threshold-controlled bias gradient descent (TCB-GD). In each local training round, the effective parameter list is dynamically updated based on the relationship between each benchmark parameter in the current optimized subnet and the optimization threshold. The mask of each benchmark parameter is updated based on the updated effective parameter list, and bias gradient correction is applied to the benchmark parameters that are close to the optimization threshold.

[0112] The logic of TCB-GD is to allow parameters that fall below the optimization threshold after training to have the opportunity to be removed from the list of valid parameters, without exceeding the upper limit of the parameters corresponding to the IoT device, thereby reducing the training rigidity caused by fixed sparse selection.

[0113] Furthermore, after each IoT device trains its corresponding optimized subnet / target subnet for multiple rounds, it only uploads updates to the actual activation parameters in the trained subnet to the server. The server receives the training parameter sets from each IoT device, then aggregates them to obtain a first parameter set and a second parameter set. Based on the first and second parameter sets, and the updated weight allocation relationship, the server updates the parameters in the shared supernet. At this point, the current training phase ends.

[0114] The training parameter set includes the parameters in the optimized subnet trained by the IoT device; the first parameter set is the updated parameters in the subnet with the upper scale limit, and the second parameter set is the updated parameters in the remaining subnets.

[0115] Since the optimization subnet / target subnet trained by different IoT devices may contain different parameters, the server cannot simply average the training parameter sets uploaded by all IoT devices as parameter updates for the same complete model. Instead, it needs to determine which IoT devices actually trained and updated each baseline parameter in the shared supernet.

[0116] The process of the server aggregating the training parameter sets uploaded by various IoT devices is shown in formula (9);

[0117] , formula (9);

[0118] in, To share reference parameters in supernets The aggregation results For IoT devices For reference parameters Update To actually include and update the basic parameters in the current training phase The number of IoT devices. If a certain baseline parameter has not been updated by any Internet device during training, then that parameter will not be aggregated in the current training phase.

[0119] After completing the parameter-level aggregation, the server further distinguishes between the first parameter updated from the largest-scale subnet and the second parameter from other-scale subnets. The largest-scale subnet can cover a more complete set of baseline parameters, while other-scale subnets are beneficial for improving the adaptability of different architectures. Therefore, in this scheme, the aggregation results of the first and second parameters are fused using weights that change with the training process. The fusion process is shown in formula (10).

[0120] , formula (10);

[0121] in, For the server's learning rate, For adaptive or decaying fusion weights, This indicates the aggregated update related to the largest subnet. This represents the aggregated updates related to subnets at other scales. Employing a parameter update method that combines classification aggregation and fusion, it can balance the overall training stability of the shared supernet with the optimization strength of the edge subnets.

[0122] After completing the training task of the shared supernet, federated training is not re-executed for different IoT devices based on their deployment resource conditions. Instead, a personalized subnet that meets the deployment budget of each IoT device is searched in the already trained shared supernet. The search considers both the loss of the subnet on the IoT device and the model size constraint. The objective can be expressed by formula (11).

[0123] , formula (11);

[0124] in, This represents the shared supernet parameters after training is complete. Indicates Internet of Things (IoT) devices Deployment budget, This indicates the evaluation targets on IoT devices. This indicates the parameter size of the subnet. Using the above method, a single shared supernet training can search for corresponding subnets across multiple deployment budgets, thereby reducing redundant training and communication overhead.

[0125] The resource-aware model training method provided in this application is applied to model training and deployment scenarios for various IoT devices with limited resources. Using this method improves the training feasibility for resource-constrained IoT devices, enhances the accuracy of deployed models across IoT devices with different deployment budgets, and reduces the cost of repetitive training and uplink communication overhead under multiple deployment objectives. Experimental results demonstrate that, under conditions of non-independent and identically distributed data and dynamic resources, the resource-aware model training method provided in this application, compared to baseline methods such as FedAvg, FedNAS, and FedPNAS, exhibits better deployment accuracy and lower training and communication costs under different deployment budgets and IoT device participation ratios. Specifically, in low MAC budget scenarios, this solution can achieve a maximum accuracy improvement of approximately 13.97% compared to the baseline and reach approximately 90% test accuracy. Meanwhile, the resource-aware model training method provided in this application supports multiple deployment targets through a single shared supernet training, avoiding repeated federated training for different deployment budgets, so that the training cost and total uplink transmission bits remain basically stable as the number of deployment targets increases; under sparse participation conditions with a low participation rate of IoT devices, this solution can still maintain deployment accuracy better than the baseline, indicating that it can adapt to the scenario of sparse participation of IoT devices and dynamic changes in resources in large-scale IoT.

[0126] The above embodiments are a resource-aware model training method based on parameters from the server's perspective; the following embodiments illustrate the process of IoT devices training the optimized subnet / target subnet issued by the server from the perspective of IoT devices.

[0127] In one exemplary embodiment, this application provides a resource-aware model training method, applicable to, for example... Figure 1 The IoT devices shown, such as Figure 5 As shown, the method specifically includes steps 510 and 520, wherein:

[0128] Step 510: Obtain resource information and send it to the server; the resource information represents the hardware specifications and communication status of the IoT device;

[0129] Step 520: Receive the optimized subnet sent by the server and train based on local data and the optimized subnet; the optimized subnet is obtained by the server optimizing the target subnet according to the upper limit of the trainable parameters of the IoT device; the upper limit of parameters is determined by the server based on resource information, and the target subnet is determined by the server in the candidate list corresponding to the current round.

[0130] In the embodiments of this application, a shared supernet is maintained in advance on the server. The shared supernet includes multiple subnets of different scales, and a subnet list (architectural space) is constructed to retrieve each subnet in the shared supernet. The subnet list contains the unique identifier of each subnet and the scale (parameter scale) corresponding to the subnet.

[0131] The model training method includes multiple training phases. In each phase, the server determines a candidate list from the shared supernet. This candidate list includes N candidate subnets of different scales, where N is the number of IoT devices participating in the training phase. The candidate list contains the subnets that each IoT device needs to train in the current training phase. Then, based on an allocation strategy, a corresponding target subnet is assigned to each IoT device. The concept of scale describes the number of baseline parameters or network layers in the subnet. In one example, a larger subnet scale corresponds to a larger number of baseline parameters and / or network layers.

[0132] Furthermore, the allocation strategy can be a random allocation strategy, or it can be allocated according to the scale of the target subnet that each IoT device has been allocated in the previous training phase.

[0133] Resource information for IoT devices includes specifications and communication status parameters. Specifications characterize the hardware performance of the IoT device, while communication status parameters characterize its data transmission capabilities. Under optimal communication conditions, the upper limit of trainable parameters for an IoT device depends on its specifications; conversely, under strong specifications, the upper limit depends on its real-time communication status. Therefore, determining the upper limit of trainable parameters for an IoT device based on both communication status parameters and specifications yields a more balanced result.

[0134] The specifications of each IoT device can be pre-stored on the server, allowing for direct scheduling. Alternatively, these specifications can be uploaded to the server by the IoT devices during the initial training phase. The server receives this information and can directly access it in subsequent training phases. Communication status parameters, however, are time-sensitive. Therefore, they are fed back to the server by the IoT devices in response to communication status retrieval requests. In other words, at the start of each training phase, the server sends communication status retrieval requests to each IoT device to obtain its latest, up-to-date communication status parameters.

[0135] Furthermore, for each IoT device, after determining the target subnet corresponding to the IoT device, the scale (parameter size) of the target subnet is obtained from the subnet list based on its unique identifier, and the upper limit of trainable parameters of the IoT device is compared with the scale (parameter size) of the target subnet. If the comparison result indicates that the scale (parameter size) of the target subnet is greater than the upper limit of parameters, then the target subnet needs to be optimized. If the comparison result indicates that the scale (parameter size) of the target subnet is less than or equal to the upper limit of parameters, then the target subnet is directly sent to the IoT device for training.

[0136] In the resource-aware model training method described above, under optimal communication conditions, the upper limit of trainable parameters for an IoT device depends on the device's specifications; conversely, under strong specifications, the upper limit depends on the device's real-time communication status. Therefore, determining the upper limit of trainable parameters based on both communication status parameters and specifications yields a more balanced result. After determining the current upper limit of trainable parameters for the IoT device, it is compared with the scale (parameter size) of the target subnet. If the comparison indicates that the scale (parameter size) of the target subnet exceeds the upper limit, the target subnet needs to be optimized. The optimized subnet is then sent to the IoT device for training. This prevents the IoT device from training a subnet exceeding its parameter capacity, improving both training efficiency and stability.

[0137] In this embodiment, when the target subnet allocated to the IoT device exceeds the upper limit of the parameters that the IoT device can train, the target subnet is not directly pruned to delete parameters. Instead, the selection is based on the importance of each benchmark parameter in the target subnet, and only some benchmark parameters whose importance ranking is within the parameter upper limit range are allowed to participate in training.

[0138] The server optimizes the target subnet based on the parameter upper limit to obtain the optimized subnet. Specifically, this process may include: obtaining the parameter amplitude of each benchmark parameter in the target subnet, sorting the benchmark parameters according to the parameter amplitude to obtain a parameter sequence; then determining the effective parameter list based on the parameter upper limit and the parameter sequence; the effective parameter list includes the parameters used by IoT devices for training; and then optimizing the target subnet based on the effective parameter list to obtain the optimized subnet.

[0139] Specifically, parameter magnitude represents the importance / priority of a parameter within a subnet; a larger magnitude indicates greater importance / priority. For the target subnet, the baseline parameters are sorted in descending order of magnitude to obtain a parameter sequence. An optimization threshold is then determined based on the upper limit of the parameter values. Then, based on the upper limit of trainable parameters for IoT devices, the Top-K baseline parameters are determined as effective parameters, and an effective parameter list is constructed based on each effective parameter. Here, Top-K represents the top K baseline parameters in the parameter sequence; that is, the parameter magnitudes of the top-K baseline parameters in the parameter sequence are all greater than or equal to the optimization threshold. The baseline parameters not included in the effective parameter list have amplitudes smaller than the optimization threshold. The effective parameter list contains the parameters actually used for training by the IoT device, and the Top-K effective parameters do not exceed the upper limit of the parameters corresponding to the IoT device.

[0140] The process of optimizing the target subnet based on the effective parameter list refers to generating a binary threshold mask for each reference parameter in the target subnet, and multiplying each reference parameter in the target subnet element by element with the corresponding binary threshold mask to obtain the optimized subnet.

[0141] When the server distributes the optimized subnet to the corresponding IoT devices for training, the IoT devices train the optimized subnet based on their stored local data, which includes multiple training rounds. However, if the effective parameter list from the first training round is used throughout the entire training process, some parameters whose magnitudes are close to the optimization threshold still need to be trained even if they become less important after training.

[0142] In one possible implementation, the process of training the optimized subnet by the IoT device may specifically include: in the nth round, training the optimized subnet based on local data and a list of effective parameters; the list of effective parameters is determined by the server based on the parameter upper limit and the parameter sequence, which is obtained by arranging the baseline parameters in the target subnet according to the parameter amplitude, where the parameter amplitude represents the importance in the subnet; n=1; after the nth round of training, updating the list of effective parameters based on the updated parameter amplitudes of the baseline parameters in the optimized subnet obtained from the previous round of training; and training the optimized subnet based on the updated list of effective parameters and local data.

[0143] Specifically, to further optimize the stability and accuracy of training, in this embodiment, the effective parameter list is dynamically updated in each local training round based on the relationship between each benchmark parameter and the optimization threshold in the current optimization subnet. The mask of each benchmark parameter is also updated based on the updated effective parameter list, and bias gradient correction is applied to benchmark parameters close to the optimization threshold. The process of dynamically updating the effective parameter list is as follows: after each local training round of the IoT device, all benchmark parameters in the optimization subnet are re-sorted according to their parameter amplitudes to obtain a new parameter sequence. The Top-K benchmark parameters are then re-determined as effective parameters, and an effective parameter list is constructed based on each effective parameter.

[0144] In other words, in each local training round, the parameters trained by the IoT device are always kept within the corresponding upper limit of the parameter range, and are the batch of parameters with the highest amplitude.

[0145] In another possible implementation, to further optimize training stability and accuracy, the scheme in this application optimizes the logic of biased gradient descent (TCB-GD) by controlling the threshold. The effective parameter list is dynamically updated based on the relationship between each baseline parameter in the current optimized subnet and the optimization threshold. The masks of each baseline parameter are then updated based on the updated effective parameter list, and bias gradient corrections are applied to baseline parameters close to the optimization threshold. Specifically, after each local training epoch, the parameter magnitudes of each effective parameter in the effective parameter list are updated. Furthermore, effective parameters with magnitudes lower than the optimization threshold are removed from the effective parameter list, thus updating it. The next local epoch is then trained based on the updated effective parameter list.

[0146] Specifically, the gradient correction process is shown in equation (12);

[0147] , formula (12);

[0148] in, This indicates the local training step for IoT devices. Indicates the first Parameters for each local training step. Indicates the local learning rate. Indicates Internet of Things (IoT) devices The local training objective of TCB-GD is to allow parameters that fall below the optimization threshold after training to have the opportunity to be removed from the list of valid parameters, without exceeding the parameter limit corresponding to the IoT device, thereby mitigating the training rigidity caused by fixed sparse selection.

[0149] Furthermore, after each IoT device trains its corresponding optimized subnet / target subnet for multiple rounds, it only uploads updates to the actual activation parameters in the trained subnet to the server. The server receives the training parameter sets from each IoT device, then aggregates them to obtain a first parameter set and a second parameter set. Based on the first and second parameter sets, and the updated weight allocation relationship, the server updates the parameters in the shared supernet. At this point, the current training phase ends.

[0150] In this embodiment of the application, after the training of the shared supernet is completed, the federated training is not re-executed for different IoT devices based on their deployment resource conditions. Instead, a personalized subnet that meets the deployment budget of each IoT device is searched in the already trained shared supernet.

[0151] For each IoT device, when searching for the corresponding personalized subnet to be deployed in the shared supernet, the loss of local data of each subnet on the IoT device and the model size constraint are considered at the same time. The search target can be represented by formula (13).

[0152] , formula (13);

[0153] in, This represents the shared supernet parameters after training is complete. This indicates the deployment budget for IoT devices. This indicates the evaluation targets on IoT devices. This indicates the parameter size of the subnet. In this way, a single shared supernet training can search for corresponding personalized subnets for multiple IoT devices with different deployment budgets, reducing redundant training and communication overhead.

[0154] The resource-aware model training method provided in this application can be applied to application scenarios involving a large number of heterogeneous intelligent terminals, such as smart cities, intelligent transportation, industrial IoT, and vehicle-to-everything (V2X) networks. In one possible example, the resource-aware model training method provided in this application is applied to a smart city. Taking intelligent cameras in a smart city as an example, a large number of cameras are typically deployed in urban roads, traffic intersections, parking lots, commercial areas, and residential communities for tasks such as vehicle recognition, pedestrian recognition, traffic status analysis, and abnormal event detection. Since the amount of image and video data collected by the cameras is large and may contain privacy information, it is not possible to directly upload local data to the cloud for model training. Using the resource-aware model training method of this application, each camera, as an IoT device, can participate in the training of an artificial intelligence model using locally collected data without uploading the original data.

[0155] In practical applications, the hardware performance and operating environments of different cameras (IoT devices) vary significantly. For example, cameras deployed at important traffic intersections may have strong computing power and stable network connections, while ordinary community or low-power cameras have relatively limited computing, storage, and communication capabilities. Furthermore, even for the same IoT device, its network status, available processor resources, and energy status may change over time. Therefore, different IoT devices (cameras) can handle different model training scales at the same time, making it difficult to simultaneously meet the training needs of all devices using a fixed-size neural network model.

[0156] To address the characteristics of this scenario, a shared supernet can be maintained on a cloud server. This shared supernet is not a single neural network that can only operate with a fixed structure, but rather can contain multiple candidate subnetworks of different sizes and structures. Cameras (IoT devices) with sufficient resources can obtain larger subnetworks from the shared supernet for training, while cameras (IoT devices) with limited computing power, communication conditions, or available energy can obtain smaller subnetworks that match their current resource conditions. Although different IoT devices (cameras) actually train models of different sizes, these subnetworks all originate from the same shared supernet and share the corresponding model parameters. Therefore, the training results of each device can be used collectively to improve the shared supernet in the cloud.

[0157] For example, in the same training round, road cameras (IoT devices) with good network conditions and strong computing power can undertake larger sub-network training tasks; community cameras with average resources can train medium-sized sub-networks; and cameras experiencing network congestion, low battery, or limited computing resources can train smaller models. As the network, computing, or energy status of a device changes, the subsequent training scale it can undertake can also be adjusted accordingly. This eliminates the need for all cameras to always train the exact same model, enabling devices with different resource conditions to participate in collaborative training.

[0158] Furthermore, the shared supernet can adapt to varying device resources during the training phase and different model requirements during actual deployment. After the shared supernet completes one round of joint training, a final sub-network suitable for the actual deployment of each IoT device can be selected from the pre-trained shared supernet based on the hardware capabilities and resource limitations of each device during actual deployment. For example, high-performance road cameras can deploy larger-scale models with stronger recognition capabilities, ordinary community cameras can deploy medium-scale models, and low-power cameras can deploy lighter-weight models. Since these models all originate from the same pre-trained shared supernet, there is no need to re-train a complete federated network for different models or devices with different resource conditions.

[0159] In addition to smart cameras, the resource-aware model training method provided in this application embodiment can also be extended to IoT terminals such as smart vehicles, drones, industrial sensors, smart robots, and wearable devices. It is especially suitable for application environments with a large number of terminals, large differences in device performance, dynamic changes in resource status, and different terminals need to deploy artificial intelligence models of different scales.

[0160] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0161] Based on the same inventive concept, this application also provides a resource-aware model training apparatus for implementing the resource-aware model training method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more resource-aware model training apparatus embodiments provided below can be found in the limitations of the resource-aware model training method described above, and will not be repeated here.

[0162] In one exemplary embodiment, such as Figure 6 As shown, a resource-aware model training device 600 is provided, including an allocation module 601, a parameter upper limit determination module 602, an optimization module 603, and a transmission module 604, wherein:

[0163] The allocation module 601 is used to determine the target subnet corresponding to each IoT device based on the candidate list corresponding to the current training phase. The candidate list includes N candidate subnets of different scales determined from the shared supernet, where N is the number of IoT devices participating in the training in the current training phase, and the shared supernet includes multiple subnets of different scales.

[0164] The parameter upper limit determination module 602 is used to obtain the resource information of each IoT device and determine the upper limit of the trainable parameters of the IoT device based on the resource information.

[0165] The optimization module 603 is used to optimize the target subnet according to the upper limit of the parameters to obtain the optimized subnet;

[0166] The sending module 604 is used to send the optimized subnet to IoT devices for training.

[0167] In one embodiment, the resource information includes specification information and communication status parameters, and the parameter upper limit determination module 602 is specifically used for:

[0168] Determine the effective uplink rate of IoT devices based on communication status parameters;

[0169] Based on the power consumption limit, effective uplink speed, and specification information of the IoT device, determine the upper limit of the parameters that the IoT device can be trained on.

[0170] In one embodiment, the optimization module 603 is specifically used for:

[0171] Obtain the parameter magnitudes of each reference parameter in the target subnet, and sort the reference parameters according to their magnitudes to obtain a parameter sequence; the parameter magnitudes characterize the importance of each parameter in the subnet.

[0172] Based on the parameter upper limit and parameter sequence, determine the list of valid parameters; the list of valid parameters includes the parameters used by the IoT device for training.

[0173] The target subnet is optimized based on the list of valid parameters to obtain the optimized subnet.

[0174] In one embodiment, the resource-aware model training device 600 further includes a candidate list construction module, specifically used for:

[0175] In the current training phase, N-2 subnets are selected from the subnets of the shared supernet, excluding the edge subnets, as supplementary subnets; the edge subnets include subnets with upper and lower scale limits;

[0176] Based on the edge subnet and each supplementary subnet, construct the candidate list corresponding to the current training stage.

[0177] In one embodiment, the resource-aware model training device 600 further includes a parameter update module, specifically used for:

[0178] Receive training parameter sets from each IoT device. The training parameter sets include the parameters in the optimized subnet after being trained by the IoT devices.

[0179] Aggregate the training parameter sets to obtain the first parameter set and the second parameter set; the first parameter set contains the updated parameters in the subnet with the upper scale limit, and the second parameter set contains the updated parameters in the remaining subnets;

[0180] Based on the first and second parameter sets, and the updated weight allocation relationship, the parameters in the shared supernet are updated.

[0181] In one exemplary embodiment, such as Figure 7 As shown, a resource-aware model training device 700 is provided, including a parameter uploading module 701 and a training module 702, wherein:

[0182] The parameter upload module 701 is used to acquire resource information and send it to the server; the resource information represents the hardware specifications and communication status of the IoT device.

[0183] The training module 702 is used to receive the optimized subnet sent by the server and perform training based on local data and the optimized subnet. The optimized subnet is obtained by the server optimizing the target subnet according to the upper limit of the trainable parameters of the IoT device. The upper limit of parameters is determined by the server based on resource information, and the target subnet is determined by the server in the candidate list corresponding to the current round.

[0184] In one embodiment, the training module 702 is specifically used for:

[0185] In the nth round, the optimized subnet is trained based on local data and the effective parameter list; the effective parameter list is determined by the server according to the parameter upper limit and the parameter sequence, and the parameter sequence is obtained by arranging the benchmark parameters in the target subnet according to the parameter amplitude, and the parameter amplitude represents the importance in the subnet; n=1;

[0186] In the nth round, the effective parameter list is updated based on the updated parameter magnitudes of each of the baseline parameters in the optimized subnet obtained from the (n-1)th round of training;

[0187] The optimized subnet is trained based on the updated list of valid parameters and local data.

[0188] Each module in the resource-aware model training device 600 and 700 described above can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0189] In one exemplary embodiment, a computer device is provided, which may be an Internet of Things (IoT) device or a server, and its internal structure diagram may be as follows. Figure 8As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements either a resource-aware model training method executed by a server or a resource-aware model training method executed by an Internet of Things (IoT) device. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0190] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0191] In one exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in an embodiment of a resource-aware model training method executed by a server, or in an embodiment of a resource-aware model training method executed by an Internet of Things device.

[0192] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in an embodiment of a resource-aware model training method executed by a server, or in an embodiment of a resource-aware model training method executed by an Internet of Things device.

[0193] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps as described in an embodiment of a resource-aware model training method executed by a server, or in an embodiment of a resource-aware model training method executed by an Internet of Things device.

[0194] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0195] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0196] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0197] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A resource-aware model training method, characterized in that, Applied to a server, the method includes: Based on the candidate list corresponding to the current training phase, the target subnet corresponding to each IoT device is determined; the candidate list includes N candidate subnets of different scales determined from the shared supernet, where N is the number of IoT devices participating in the training in the current training phase, and the shared supernet includes multiple subnets of different scales; For each IoT device, obtain the resource information of the IoT device, and determine the upper limit of the trainable parameters of the IoT device based on the resource information; Based on the upper limit of the parameters, the target subnet is optimized to obtain an optimized subnet; The optimized subnet is sent to the IoT device for training.

2. The method according to claim 1, characterized in that, The resource information includes specification information and communication status parameters. Determining the upper limit of trainable parameters for the IoT device based on the resource information includes: The effective uplink rate of the IoT device is determined based on the communication status parameters. Based on the energy consumption limit of the IoT device, the effective uplink rate, and the specification information, the upper limit of the parameters that the IoT device can be trained is determined.

3. The method according to claim 1, characterized in that, The step of optimizing the target subnet according to the upper limit of the parameters to obtain an optimized subnet includes: The parameter amplitudes of each reference parameter in the target subnet are obtained, and the reference parameters are sorted according to the parameter amplitudes to obtain a parameter sequence; the parameter amplitudes characterize the importance of the parameter in the subnet. Based on the parameter upper limit and the parameter sequence, a list of valid parameters is determined; the list of valid parameters includes the parameters used by the IoT device for training. The target subnet is optimized based on the list of valid parameters to obtain the optimized subnet.

4. The method according to claim 1, characterized in that, The method further includes: In the current training phase, N-2 subnets are determined as supplementary subnets from the subnets in the shared supernet, excluding the edge subnets; the edge subnets include subnets with upper and lower scale limits; Based on the edge subnet and each of the supplementary subnets, the candidate list corresponding to the current training phase is constructed.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Receive training parameter sets fed back by each of the IoT devices, the training parameter sets including parameters in the optimized subnet after being trained by the IoT devices; By aggregating the training parameter sets, a first parameter set and a second parameter set are obtained; the first parameter set consists of the updated parameters in the subnet with the upper scale limit, and the second parameter set consists of the updated parameters in the remaining subnets. Based on the first parameter set and the second parameter set, and the updated weight allocation relationship, the parameters in the shared supernet are updated.

6. A resource-aware model training method, characterized in that, Performed by an IoT device, the method includes: Acquire resource information and send the resource information to the server; the resource information represents the hardware specifications and communication status of the IoT device; The system receives an optimized subnet sent by the server and trains based on local data and the optimized subnet. The optimized subnet is obtained by the server by optimizing the target subnet according to the upper limit of trainable parameters of the IoT device. The upper limit of parameters is determined by the server based on the resource information, and the target subnet is determined by the server in the candidate list corresponding to the current round.

7. The method according to claim 6, characterized in that, The training based on local data and the optimized subnet includes: In the nth round, the optimized subnet is trained based on local data and a list of effective parameters; the list of effective parameters is determined by the server based on the parameter upper limit and the parameter sequence, the parameter sequence is obtained by arranging each benchmark parameter in the target subnet according to the parameter amplitude, and the parameter amplitude characterizes the importance in the subnet; n=1; In the nth round, the effective parameter list is updated based on the updated parameter magnitudes of each of the baseline parameters in the optimized subnet obtained from the (n-1)th round of training; The optimized subnet is trained based on the updated list of valid parameters and the local data.

8. A resource-aware model training device, characterized in that, The device includes an allocation module, a parameter upper limit determination module, an optimization module, and a transmission module, wherein: The allocation module is used to determine the target subnet corresponding to each IoT device according to the candidate list corresponding to the current training phase; the candidate list includes N candidate subnets of different scales determined from the shared supernet, where N is the number of IoT devices participating in the training in the current training phase, and the shared supernet includes multiple subnets of different scales; The parameter upper limit determination module is used to obtain resource information of each IoT device and determine the upper limit of trainable parameters of the IoT device based on the resource information. The optimization module is used to optimize the target subnet according to the upper limit of the parameters to obtain an optimized subnet; The sending module is used to send the optimized subnet to the IoT device for training.

9. A resource-aware model training device, characterized in that, The device includes a parameter uploading module and a training module, wherein: The parameter upload module is used to acquire resource information and send the resource information to the server; the resource information represents the hardware specifications and communication status of the IoT device. The training module is used to receive the optimized subnet sent by the server and perform training based on local data and the optimized subnet; the optimized subnet is obtained by the server by optimizing the target subnet according to the upper limit of trainable parameters of the IoT device; the upper limit of parameters is determined by the server according to the resource information, and the target subnet is determined by the server in the candidate list corresponding to the current round.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5, or the steps of the method according to any one of claims 6 to 7.