Federal learning node resource allocation method and device, equipment and medium

Abstract

The invention discloses a federated learning node resource allocation method and device, equipment and a medium, and belongs to the field of computing resource management. Determining a first mapping relationship between the training loss of each local model in the next round of training and the correspondingly used training setting parameters; constructing a second mapping relationship between the node resource demand of each local model during single training and the corresponding used training setting parameter; according to the first mapping relationship and the second mapping relationship, constructing and solving an optimization model which maximizes the node resource utilization rate as an optimization target during the next round of training, and obtaining an optimal training setting parameter of each local model during the next round of training; and distributing node resources required by the next round of training to the corresponding local model according to each optimal training setting parameter. The problem that the node resource utilization rate is low in the process of processing the operation data of the power distribution network through federal learning can be solved.

Description

Technical Field

[0001] This application relates to the field of computing resource management, and in particular to a method, apparatus, device and medium for allocating resources for federated learning nodes. Background Technology

[0002] As the proportion of distributed power sources connected to smart grids continues to rise, the business needs of distribution networks are becoming increasingly diversified. Furthermore, data from different business operations is stored on different devices within the distribution network, making direct centralized aggregation impossible. Therefore, federated learning is required to process the business data of the distribution network in a distributed manner.

[0003] While federated learning can address the issue of centralized data upload, existing technologies generally employ uniform training configurations or static resource strategies, meaning that each node uses the same model parameter size, data batch size, and number of iterations in each training round. However, the distributed nodes in a smart grid vary significantly in terms of communication bandwidth, network latency, computing power performance, and task priorities. This causes some nodes to fail to complete training on time, thus slowing down the global model aggregation process. Furthermore, nodes with sufficient resources cannot flexibly upgrade their training capabilities, resulting in wasted computing power. Simultaneously, the lack of a clear mapping between different training settings and resource consumption prevents dynamic optimization based on training loss, training progress, and resource requirements, ultimately leading to low resource utilization and poor model convergence efficiency.

[0004] Therefore, improving the utilization rate of node resources in the process of federated learning to process power distribution network operation data is a technical problem that needs to be solved. Summary of the Invention

[0005] This application provides a method, apparatus, device, and medium for allocating node resources in federated learning, which can solve the problem of how to improve the utilization rate of node resources in the process of federated learning for processing power distribution network operation data in the prior art.

[0006] Some embodiments of this application provide a method for allocating resources for federated learning nodes, including: Based on the local models obtained in the current training round, determine the first mapping relationship between the training loss of each local model in the next training round and the corresponding training setting parameters used; wherein, the training setting parameters include: the number of model parameters, the amount of training data, and the number of training iterations per round; Construct a second mapping relationship between the node resource requirements of each local model during a single training session and the corresponding training settings parameters used; the node resource requirements include: bandwidth resource requirements, latency resource requirements, and computing power resource requirements; Based on the first mapping relationship and the second mapping relationship, an optimization model is constructed and solved with the optimization objective of maximizing node resource utilization in the next round of training, so as to obtain the optimal training setting parameters for each local model in the next round of training. Based on the optimal training settings parameters, allocate the node resources required for the next round of training to the corresponding local model.

[0007] Compared to existing technologies, the above embodiments have the following advantages: First, a first mapping relationship between training loss and training setting parameters is constructed for each local model, enabling the system to quantitatively characterize the impact of different training parameter configurations on model convergence. Simultaneously, since the number of model parameters involved in training during local model training directly determines the number of bytes in communication and the computational cost required for each sample during forward / backward propagation, and the amount of training data and the number of training iterations determine the computational cost required to process sample data during iterative training, a second mapping relationship between training setting parameters and node bandwidth, latency, and computational resource requirements can be constructed, effectively achieving a predictable expression of node resource consumption during the training process. Based on this, an optimization model is established with the goal of maximizing node resource utilization. Combining the differentiated training loss trends and resource requirement characteristics of each local model, the corresponding optimal training setting parameters are solved, allowing each node to occupy resources as needed while ensuring training accuracy and progress constraints, thereby improving node resource utilization.

[0008] Furthermore, the process of constructing the optimization objective of the optimization model includes: The reliability index of each local model is obtained, and the node efficiency optimization target for each local model in the next training round is constructed based on the second mapping relationship corresponding to the local model and the reliability index; wherein, the reliability index is obtained based on the historical running data of the local model; After a single training iteration, a third mapping relationship is constructed between the first training loss of the global model and the second training losses of each of the local models. Based on the third mapping relationship and all the first mapping relationships, an optimization objective for the training loss of the global model is constructed. The global model is obtained by aggregating all the local models. After a single training iteration, a fourth mapping relationship is established between the first training progress of the global model and the second training progress of each local model. Based on the fourth mapping relationship and all the first mapping relationships, an optimization target for the training progress of the global model is constructed. The training progress is determined based on the rate of change of the training loss of the corresponding model. The optimization objective of the optimized model is obtained by weighted fusion of the node benefit optimization objective of each local model, the training loss optimization objective of the global model, and the training progress optimization objective.

[0009] Compared with existing technologies, the above embodiments have the following beneficial effects: by optimizing the node efficiency, node resources are preferentially allocated to nodes with higher resource utilization and reliability, thereby reducing training tail nodes from a system perspective; by optimizing the training loss, node resources are preferentially scheduled to nodes that can significantly reduce global model error, resulting in better model training performance; and by optimizing the training progress, the training process is prevented from failing to aggregate the global model due to a small number of high-loss nodes lagging behind for a long time, thereby enabling faster model training convergence speed.

[0010] Furthermore, the node benefit optimization objective is specifically as follows: in, For local models The goal of optimizing node benefits; , , as well as These are the impact coefficients for the corresponding bandwidth resource requirements, latency resource requirements, computing power resource requirements, and reliability indicators, respectively. For local models bandwidth resource requirements; For local models The latency resource requirements; For local models The demand for computing resources; For local models Reliability indicators; It is a natural number.

[0011] Furthermore, the second mapping relationship includes: a fifth mapping relationship between the amount of training data and the bandwidth resource requirement, a sixth mapping relationship between the number of model parameters and the latency resource requirement, and a seventh mapping relationship between the training setting parameters and the computing power resource requirement.

[0012] Compared with existing technologies, the above embodiments have the following beneficial effects: by decoupling the training settings parameters from the resource requirements of each node, the estimation of each node resource is transformed from empirical to predictable, enabling the calculation of node resource consumption for different training configurations in advance, providing calculable constraints for subsequent model optimization, and also enabling the optimal solution of node resource allocation in a mathematical sense rather than based on empirical thresholds. This avoids the difficulty in identifying resource bottlenecks caused by merging all resources into a single variable in traditional federated learning, and improves the accuracy of the system in allocating node resources.

[0013] Furthermore, the second mapping relationship between the node resource requirements for constructing each local model for a single training iteration and the corresponding training settings parameters includes: The fifth mapping relationship is constructed by evaluating the bandwidth resources required to transmit different amounts of training data. The sixth mapping relationship is constructed by evaluating the minimum latency required to update different numbers of model parameters; The seventh mapping relationship is constructed by evaluating the impact of each parameter in the training settings on the computing power requirement during training, and by weighting and fusing each parameter in the training settings according to the impact.

[0014] Compared to existing technologies, the above embodiments have the following advantages: First, by collecting metrics such as communication volume and computation time required for training tasks in an actual node operating environment, corresponding samples of training parameters and resource consumption are obtained; second, the evaluation results are used to fit a functional relationship, giving the resource prediction model an observable basis; finally, by calling this mapping function during the optimization process, resource cost prediction and scheduling decisions for different parameter configurations are achieved. Since the mapping relationship originates from actual measurement results, its prediction is closer to real operating characteristics, avoiding the problem of discrepancies between theoretical assumptions and actual performance. Ultimately, dynamic updates and adaptive allocation are achieved, making the optimization results more reliable and universal.

[0015] Furthermore, the seventh mapping relationship is specifically as follows: in, For local models The demand for computing resources; , as well as These are the coefficients representing the impact of the number of model parameters, the amount of training data, and the number of training iterations per session on the computational resource requirements. , as well as Local models The number of model parameters, the amount of training data, and the number of training iterations per session.

[0016] Further, the step of allocating node resources required for the next round of training to the corresponding local model according to the optimal training settings parameters includes: Based on the priority of the business processed by each local model, a corresponding priority weight is set for the corresponding local model; Based on the optimal training settings parameters and the second mapping relationship, the resource requirements of the first node corresponding to each local model are determined, and the resource requirements of the first node are determined to determine the required proportion of each local model. Based on the currently available node resources, combined with the priority weights and the demand ratios, the node resources to be used for the next round of training are allocated to each of the corresponding local models.

[0017] Compared with the prior art, the above embodiments have the following beneficial effects: Since the node business types in the distribution network are different, such as the priority of the dispatch control node being higher than that of the ordinary status monitoring node, the resource allocation cannot be determined solely by the training cost. This application uses a priority weighting mechanism to ensure that the training process of key business nodes can be maintained without interruption when resources are scarce, thereby ensuring the safety requirements of power grid operation. Then, combined with the optimized node resource demand ratio, resources are not infinitely occupied by high-priority nodes, achieving a balance between fairness and business importance, thereby maximizing the utilization rate of system-level node resources.

[0018] Another embodiment of this application provides a federated learning node resource allocation device, including: a first mapping relationship construction module, a second mapping relationship construction module, a training setting parameter optimization module, and a node resource allocation module; The first mapping relationship construction module is used to acquire and determine the first mapping relationship between the training loss of each local model in the next training round and the corresponding training setting parameters based on each local model obtained in the current round of training; wherein the training setting parameters include: the number of model parameters, the number of training data, and the number of training iterations per round. The second mapping relationship construction module is used to construct a second mapping relationship between the node resource requirements of each local model during a single training session and the corresponding training setting parameters used; the node resource requirements include: bandwidth resource requirements, latency resource requirements, and computing power resource requirements; The training setting parameter optimization module is used to construct and solve an optimization model based on the first mapping relationship and the second mapping relationship, with the optimization objective of maximizing node resource utilization in the next round of training, to obtain the optimal training setting parameters for each local model in the next round of training. The node resource allocation module is used to allocate the node resources required for the next round of training to the corresponding local model according to the optimal training setting parameters.

[0019] Another embodiment of this application also provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of the federated learning node resource allocation method of this application.

[0020] Another embodiment of this application also provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform the steps of the federated learning node resource allocation method of this application. Attached Figure Description

[0021] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating a federated learning node resource allocation method provided in some embodiments of this application; Figure 2 This is a schematic diagram of the structure of a federated learning node resource allocation device provided in some embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0025] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0026] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0027] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0028] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0029] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0030] While federated learning can address the issue of centralized data upload, existing technologies generally employ uniform training configurations or static resource strategies, meaning that each node uses the same model parameter size, data batch size, and number of iterations in each training round. However, the distributed nodes in a smart grid vary significantly in terms of communication bandwidth, network latency, computing power performance, and task priorities. This causes some nodes to fail to complete training on time, thus slowing down the global model aggregation process. Furthermore, nodes with sufficient resources cannot flexibly upgrade their training capabilities, resulting in wasted computing power. Simultaneously, the lack of a clear mapping between different training settings and resource consumption prevents dynamic optimization based on training loss, training progress, and resource requirements, ultimately leading to low resource utilization and poor model convergence efficiency.

[0031] Please refer to Figure 1 To address the low node resource utilization problem in the federated learning process of power distribution network operation data in existing technologies, this application provides a federated learning node resource allocation method, including steps S101 to S104, specifically as follows: S101: Obtain and determine the first mapping relationship between the training loss of each local model in the next training round and the corresponding training setting parameters based on the local models obtained in the current round of training; wherein, the training setting parameters include: the number of model parameters, the number of training data, and the number of training iterations per round.

[0032] Preferably, in some embodiments of this application, each local model processes business data corresponding to a business scenario under the distribution network, wherein the business scenarios include: fast charging of electric vehicles, distributed power source access, and residential daily electricity consumption. Different business scenarios have different bandwidth, latency, and computing power requirements. Among them, fast charging of electric vehicles requires high bandwidth, low latency, and high computing power resources; distributed power source access has relatively moderate requirements for bandwidth and latency, mainly focusing on the stable transmission and processing capabilities of data; residential daily electricity consumption requires lower bandwidth and latency, and uses fewer resources.

[0033] Preferably, in some embodiments of this application, based on the characteristics of federated learning, each local model is independently trained on different independent clients and does not share original data. Collaborative learning is achieved by exchanging model parameters through a global model deployed on a central server. Therefore, federated learning can effectively protect data privacy and promote collaborative data analysis. Thus, before training, each local model first models battery charging status, power distribution, load prediction, etc., within the power distribution network. The federated learning tasks corresponding to each local model, divided according to business scenarios, include electric vehicle fast charging, distributed power access, and residential daily electricity consumption scenarios. Each business scenario corresponds to one local model.

[0034] Assume there is Each local model It has its own independent local dataset: in, yes Input of the sample yes Sample labels It is a local model The amount of data.

[0035] Before the formal training of federated learning begins, the central server initializes a global model. This model contains the initial model parameters for all nodes to begin local training. It is represented as a vector or matrix that contains all the parameters that need to be learned.

[0036] Each local model uses gradient descent to update its parameters, but different local learning rates are used for training depending on the characteristics of the local dataset. It should be noted that before training, the local model needs to receive the global model parameters from the central server and trains based on these parameters. Furthermore, in some embodiments of this application, during local model training, the number of model parameters to be frozen in this training round is determined by the number of model parameters in the current training epoch. That is, during local model training, not all model parameters of the currently received global model are trained; the number of model parameters participating in training is the number of model parameters in the training settings, and other untrained model parameters remain frozen.

[0037] Each local model In local dataset Train the model independently. Assume the current local model determines the model parameters to participate in training as follows: Local model It selects a sample dataset for training from its local dataset and performs gradient descent iteratively multiple times to update the local model parameters. The number of iterations is determined by setting the parameters during training. Local model The training objective is to minimize its local loss function, which can typically be expressed as: in, For local models The second training loss is obtained by training according to the training settings parameters in the current training round. It is a local model Upper Sample loss function, For local models The sample dataset selected in the current training round For local models The model parameters used in the current training epoch. The loss function is implemented as follows: yes The transpose of the matrix is ​​used to optimize the model for each local model using its chosen sample dataset, resulting in updated model parameters. The update steps are as follows: in, It is the local learning rate. It is the gradient of the local loss function, representing the gradient at the current... The gradient of the local model k is below. For local models Model parameters before update.

[0038] Therefore, it can be understood that after obtaining each local model from the current training round, by performing a pre-training of the local model according to the determined training settings parameters, the training loss of the local model in the next training round can be known. That is, the pre-training process of the local model is the first mapping relationship.

[0039] S102: Construct a second mapping relationship between the node resource requirements of each local model during a single training session and the corresponding training settings parameters used; the node resource requirements include: bandwidth resource requirements, latency resource requirements, and computing power resource requirements.

[0040] Furthermore, in some embodiments of this application, the second mapping relationship includes: a fifth mapping relationship between the amount of training data and the bandwidth resource requirement, a sixth mapping relationship between the number of model parameters and the latency resource requirement, and a seventh mapping relationship between the training setting parameters and the computing power resource requirement.

[0041] Preferably, in some embodiments of this application, the bandwidth resource requirement reflects the network bandwidth requirement of the local model during local training, and the amount of sample data used by the local model in training affects its bandwidth requirement. The latency resource requirement reflects the latency requirement of the local model during local training and model updates. The computing power resource requirement reflects the computing power required by the local model during local training.

[0042] By decoupling the training settings parameters from the resource requirements of each node, the estimation of node resources is transformed from empirical to predictable. This allows for the pre-calculation of node resource consumption for different training configurations, providing calculable constraints for subsequent model optimization. Furthermore, it enables the optimal solution for node resource allocation in a mathematical sense, rather than based on empirical thresholds. This avoids the difficulty in identifying resource bottlenecks caused by merging all resources into a single variable in traditional federated learning, thus improving the accuracy of the system's node resource allocation.

[0043] Furthermore, in some embodiments of this application, the second mapping relationship between the node resource requirements for constructing each local model for a single training iteration and the corresponding training setting parameters includes: The fifth mapping relationship is constructed by evaluating the bandwidth resources required to transmit different amounts of training data. The sixth mapping relationship is constructed by evaluating the minimum latency required to update different numbers of model parameters; The seventh mapping relationship is constructed by evaluating the impact of each parameter in the training settings on the computing power requirement during training, and by weighting and fusing each parameter in the training settings according to the impact.

[0044] Preferably, in some embodiments of this application, the fifth mapping relationship is specifically as follows: in, For local models bandwidth resource requirements; To participate in the local model The amount of training data used in training; The total amount of training data used when training all local models; This refers to the total network bandwidth.

[0045] Preferably, in some embodiments of this application, the sixth mapping relationship is obtained by measuring the historical network latency from the local model to the central server under different model parameter settings. Specifically, it can be obtained by curve fitting or any fitting method, and this application does not limit the process.

[0046] Preferably, in some embodiments of this application, the seventh mapping relationship is specifically as follows: in, For local models The demand for computing resources; , as well as These are the coefficients representing the impact of the number of model parameters, the amount of training data, and the number of training iterations per session on the computational resource requirements. , as well as Local models The number of model parameters, the amount of training data, and the number of training iterations per session.

[0047] First, metrics such as communication volume and computation time required for training tasks are collected in a real-world node operating environment to obtain corresponding samples of training parameters and resource consumption. Second, the evaluation results are used to fit a functional relationship, giving the resource prediction model an observable basis. Finally, by calling this mapping function during the optimization process, resource cost prediction and scheduling decisions for different parameter configurations are achieved. Since the mapping relationship originates from actual measurement results, its predictions are closer to real-world operating characteristics, avoiding the problem of discrepancies between theoretical assumptions and actual performance. Ultimately, dynamic updates and adaptive allocation are achieved, making the optimization results more reliable and universal.

[0048] S103: Based on the first mapping relationship and the second mapping relationship, construct and solve an optimization model with the goal of maximizing node resource utilization in the next round of training, and obtain the optimal training settings parameters for each local model in the next round of training.

[0049] Furthermore, in some embodiments of this application, the process of constructing the optimization objective of the optimization model includes: The reliability index of each local model is obtained, and the node efficiency optimization target for each local model in the next training round is constructed based on the second mapping relationship corresponding to the local model and the reliability index; wherein, the reliability index is obtained based on the historical running data of the local model; After a single training iteration, a third mapping relationship is constructed between the first training loss of the global model and the second training losses of each of the local models. Based on the third mapping relationship and all the first mapping relationships, an optimization objective for the training loss of the global model is constructed. The global model is obtained by aggregating all the local models. After a single training iteration, a fourth mapping relationship is established between the first training progress of the global model and the second training progress of each local model. Based on the fourth mapping relationship and all the first mapping relationships, an optimization target for the training progress of the global model is constructed. The training progress is determined based on the rate of change of the training loss of the corresponding model. The optimization objective of the optimized model is obtained by weighted fusion of the node benefit optimization objective of each local model, the training loss optimization objective of the global model, and the training progress optimization objective.

[0050] Preferably, in some embodiments of this application, the training loss optimization objective is specifically: in, This represents the first training loss of the global model after a single training round. This represents the total number of local models. For local models The number of training data in a single training round is and the number of model parameters is The second training loss at that time; The total amount of training data used when training all local models.

[0051] Preferably, in some embodiments, the training progress optimization objective is specifically: in, This represents the first training progress of the global model after a single training round. For local models The second training phase following a single training session; For local models The second training loss during the previous round of training.

[0052] Preferably, in some embodiments of this application, the node benefit optimization objective is specifically: in, For local models The goal of optimizing node benefits; , , as well as These are the impact coefficients for the corresponding bandwidth resource requirements, latency resource requirements, computing power resource requirements, and reliability indicators, respectively. For local models bandwidth resource requirements; For local models The latency resource requirements; For local models The demand for computing resources; For local models Reliability indicators; It is a natural number.

[0053] Preferably, in some embodiments of this application, the optimization objective is specifically: in, To optimize the objective; , as well as These are the weight coefficients corresponding to the training loss optimization objective, the training progress optimization objective, and the node benefit optimization objective, respectively. For local models The weight coefficients of the corresponding node benefit optimization objectives.

[0054] By optimizing node efficiency, node resources are preferentially allocated to nodes with higher resource utilization and reliability, reducing training tail nodes from a system perspective. By optimizing training loss, node resources are preferentially scheduled to nodes that can significantly reduce global model error, resulting in better model training performance. By optimizing training progress, the training process is prevented from being hindered by a small number of high-loss nodes lagging behind for a long time, thus enabling faster model training convergence.

[0055] S104: Based on the optimal training settings parameters, allocate the node resources required for the next round of training to the corresponding local model.

[0056] Furthermore, in some embodiments of this application, the step of allocating node resources required for the next round of training to the corresponding local model according to the optimal training setting parameters includes: Based on the priority of the business processed by each local model, a corresponding priority weight is set for the corresponding local model; Based on the optimal training settings parameters and the second mapping relationship, the resource requirements of the first node corresponding to each local model are determined, and the resource requirements of the first node are determined to determine the required proportion of each local model. Based on the currently available node resources, combined with the priority weights and the demand ratios, the node resources to be used for the next round of training are allocated to each of the corresponding local models.

[0057] Preferably, in some embodiments of this application, the priority of the services processed by the local model can be set by preset rules or according to historical emergency situations, and this application does not limit the process.

[0058] Preferably, in some embodiments of this application, the priority weight of each local model can also be determined by the following formula: in, Represents the local model Priority weights; and Local models The second training loss and local models Second training progress The influence coefficient on priority.

[0059] Preferably, in some embodiments of this application, bandwidth resources are allocated according to the following formula: in, For local models The actual broadband resources allocated; This represents the total number of local models.

[0060] Preferably, in some embodiments of this application, latency resources are allocated according to the following formula: in, For local models The actual latency resources allocated; This represents the total number of latency resources.

[0061] Preferably, in some embodiments of this application, computing resources are allocated according to the following formula: in, For local models The actual computing power resources allocated; This represents the total number of computing resources.

[0062] Preferably, in some embodiments of this application, once the specific node resources required for each local model's corresponding business scenario are known, the node resources are allocated to the corresponding local model in the following way: Dedicated network slice resources are allocated to each local model corresponding to each business scenario using 5G network slicing technology. An independent network slice is defined for each business scenario, including parameters such as required bandwidth, latency, reliability, and computing power resources. When creating a slice, the resource configuration needs to be adjusted according to specific business requirements. For example: Power dispatch slice: Configure high-priority, low-latency, and high-reliability network resources. Load forecasting slice: Configure medium-priority, high-bandwidth, and high-reliability resources. Fault detection slice: Configure low-latency and high-reliability network resources to ensure rapid response.

[0063] Since nodes in the distribution network have different business types, such as dispatch control nodes having higher priority than ordinary status monitoring nodes, resource allocation cannot be determined solely based on training costs. This application uses a priority weighting mechanism to ensure that critical business nodes can maintain the training process without interruption when resources are scarce, thereby guaranteeing the safety requirements of power grid operation. Then, combined with the optimized node resource demand ratio, resources are not infinitely occupied by high-priority nodes, achieving a balance between fairness and business importance, thereby maximizing the utilization rate of system-level node resources.

[0064] Preferably, in some embodiments of this application, after each local model completes the training of the current round, the updated model parameters of all local models are aggregated in the following manner to obtain the parameters of the updated global model, specifically using the following formula: in, These are the model parameters of the updated global model. These are the model parameters of the current global model. It represents the total amount of training data used when training all local models. These are the weighting coefficients of the local model k, representing the importance of the local model k in the global model update. This is the gradient of the local model k, reflecting the contribution of local data to the model. After aggregation, the central server will update the global model. Distribute back to each local model.

[0065] Therefore, after multiple rounds of training, model training is completed when the training loss of the global model meets the following conditions, thereby dynamically allocating node resources during the federated learning process and effectively improving the utilization rate of node resources: in, and These are the first training losses obtained in the two training rounds before and after the global model, respectively; This is a preset threshold.

[0066] In summary, the federated learning node resource allocation method provided in this application has the following advantages compared to the prior art: First, a first mapping relationship between training loss and training setting parameters is constructed for each local model, enabling the system to quantitatively characterize the impact of different training parameter configurations on model convergence. Simultaneously, since the number of model parameters participating in training during local model training directly determines the number of bytes in communication and the computational cost required for each sample during forward / backward propagation, and the amount of training data and the number of training iterations determine the computational cost required to process sample data during iterative training, a second mapping relationship between training setting parameters and node bandwidth, latency, and computational resource requirements can be constructed to effectively achieve a predictable expression of node resource consumption during the training process. Based on this, an optimization model is established with the goal of maximizing node resource utilization. Combining the differentiated training loss trends and resource requirement characteristics of each local model, the corresponding optimal training setting parameters are solved, allowing each node to occupy resources as needed while ensuring training accuracy and progress constraints, thereby improving node resource utilization.

[0067] like Figure 2 As shown, based on the above method embodiments, one embodiment of this application provides a federated learning node resource allocation device, including: a first mapping relationship construction module 201, a second mapping relationship construction module 202, a training setting parameter optimization module 203, and a node resource allocation module 204.

[0068] Further, in some embodiments of this application, the first mapping relationship construction module 201 is used to acquire and determine a first mapping relationship between the training loss of each local model in the next training round and the corresponding training setting parameters based on each local model obtained in the current training round; wherein, the training setting parameters include: the number of model parameters, the number of training data, and the number of training iterations per round; the second mapping relationship construction module 202 is used to construct a second mapping relationship between the node resource requirements of each local model in a single training round and the corresponding training setting parameters; the node resource requirements include: bandwidth resource requirements, latency resource requirements, and computing power resource requirements; the training setting parameter optimization module 203 is used to construct and solve an optimization model based on the first mapping relationship and the second mapping relationship, with the optimization objective of maximizing node resource utilization in the next training round, to obtain the optimal training setting parameters for each local model in the next training round; the node resource allocation module 204 is used to allocate the node resources required for the next training round to the corresponding local model based on each optimal training setting parameter.

[0069] Further, in some embodiments of this application, the process of constructing the optimization objective of the optimization model includes: obtaining the reliability index of each local model, and constructing a node benefit optimization objective for each local model in the next round of training based on the second mapping relationship corresponding to the local model and the reliability index; wherein, the reliability index is obtained based on the historical running data of the local model; constructing a third mapping relationship between the first training loss of the global model and the second training loss of each local model after a single training, and constructing a training loss optimization objective of the global model based on the third mapping relationship and all the first mapping relationships; wherein, the global model is obtained by aggregating all the local models; constructing a fourth mapping relationship between the first training progress of the global model and the second training progress of each local model after a single training, and constructing a training progress optimization objective of the global model based on the fourth mapping relationship and all the first mapping relationships; wherein, the training progress is determined based on the training loss change rate of the corresponding model; and obtaining the optimization objective of the optimization model by weighted fusion of the node benefit optimization objective of each local model, the training loss optimization objective of the global model, and the training progress optimization objective.

[0070] Furthermore, in some embodiments of this application, the node benefit optimization objective is specifically: in, For local models The goal of optimizing node benefits; , , as well as These are the impact coefficients for the corresponding bandwidth resource requirements, latency resource requirements, computing power resource requirements, and reliability indicators, respectively. For local models bandwidth resource requirements; For local models The latency resource requirements; For local models The demand for computing resources; For local models Reliability indicators; It is a natural number.

[0071] Furthermore, in some embodiments of this application, the second mapping relationship includes: a fifth mapping relationship between the amount of training data and the bandwidth resource requirement, a sixth mapping relationship between the number of model parameters and the latency resource requirement, and a seventh mapping relationship between the training setting parameters and the computing power resource requirement.

[0072] Further, in some embodiments of this application, the second mapping relationship construction module 202 includes: a fifth mapping relationship construction unit, a sixth mapping relationship construction unit, and a seventh mapping relationship construction unit; the second mapping relationship construction module 202 is used to construct a second mapping relationship between the node resource requirements of each local model during a single training run and the corresponding training setting parameters, including: the fifth mapping relationship construction unit is used to construct the fifth mapping relationship by evaluating the bandwidth resources required to transmit different amounts of training data; the sixth mapping relationship construction unit is used to construct the sixth mapping relationship by evaluating the minimum latency required to update different amounts of model parameters; the seventh mapping relationship construction unit is used to construct the seventh mapping relationship by evaluating the degree of influence of each parameter in the training setting parameters on the computing power requirements during training, and weighting and fusing each parameter in the training setting parameters according to the degree of influence.

[0073] Furthermore, in some embodiments of this application, the seventh mapping relationship is specifically as follows: in, For local models The demand for computing resources; , as well as These are the coefficients representing the impact of the number of model parameters, the amount of training data, and the number of training iterations per session on the computational resource requirements. , as well as Local models The number of model parameters, the amount of training data, and the number of training iterations per session.

[0074] Further, in some embodiments of this application, the node resource allocation module 204 includes: a priority weight determination unit, a demand ratio calculation unit, and a node resource allocation calculation unit; the node resource allocation module 204 is used to allocate node resources required for the next round of training for the corresponding local model according to the optimal training setting parameters, including: the priority weight determination unit is used to set a corresponding priority weight for the corresponding local model according to the priority of the business processed by each local model; the demand ratio calculation unit is used to determine the first node resource demand for each local model according to the optimal training setting parameters and the second mapping relationship, and determine the demand ratio for each local model according to the first node resource demand; the node resource allocation calculation unit is used to allocate node resources used for the next round of training for each local model according to the currently allocable node resources, combined with the priority weight and the demand ratio.

[0075] It is understood that the above-described device embodiments correspond to the method embodiments of this application, and can implement the federated learning node resource allocation method provided by any of the above-described method embodiments of this application.

[0076] In summary, the federated learning node resource allocation device provided in this application has the following advantages compared to the prior art: First, a first mapping relationship between training loss and training setting parameters is constructed for each local model, enabling the system to quantitatively characterize the impact of different training parameter configurations on model convergence. Simultaneously, since the number of model parameters participating in training during local model training directly determines the number of bytes in communication and the computational cost required for each sample during forward / backward propagation, and the amount of training data and the number of training iterations determine the computational cost required to process sample data during iterative training, a second mapping relationship between training setting parameters and node bandwidth, latency, and computational resource requirements can be constructed to effectively achieve a predictable expression of node resource consumption during the training process. Based on this, an optimization model is established with the goal of maximizing node resource utilization. Combining the differentiated training loss trends and resource requirement characteristics of each local model, the corresponding optimal training setting parameters are solved, thereby allowing each node to occupy resources as needed while ensuring training accuracy and progress constraints, thus improving node resource utilization.

[0077] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0078] Based on the above embodiments of the federated learning node resource allocation method, another embodiment of this application provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the federated learning node resource allocation method of any embodiment of this application.

[0079] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete this application. The one or more module units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0080] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0081] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0082] Based on the above-described method embodiments, another embodiment of this application provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the federated learning node resource allocation method described in any of the above-described method embodiments of this application.

[0083] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

Claims

1. A method for allocating resources for federated learning nodes, characterized in that, include: Based on the local models obtained in the current training round, determine the first mapping relationship between the training loss of each local model in the next training round and the corresponding training setting parameters used; wherein, the training setting parameters include: the number of model parameters, the amount of training data, and the number of training iterations per round; Construct a second mapping relationship between the node resource requirements of each local model during a single training session and the corresponding training settings parameters used; the node resource requirements include: bandwidth resource requirements, latency resource requirements, and computing power resource requirements; Based on the first mapping relationship and the second mapping relationship, an optimization model is constructed and solved with the optimization objective of maximizing node resource utilization in the next round of training, so as to obtain the optimal training setting parameters for each local model in the next round of training. Based on the optimal training settings parameters, allocate the node resources required for the next round of training to the corresponding local model.

2. The federated learning node resource allocation method as described in claim 1, characterized in that, The process of constructing the optimization objective of the optimization model includes: The reliability index of each local model is obtained, and the node efficiency optimization target for each local model in the next training round is constructed based on the second mapping relationship corresponding to the local model and the reliability index; wherein, the reliability index is obtained based on the historical running data of the local model; After a single training iteration, a third mapping relationship is constructed between the first training loss of the global model and the second training losses of each of the local models. Based on the third mapping relationship and all the first mapping relationships, an optimization objective for the training loss of the global model is constructed. The global model is obtained by aggregating all the local models. After a single training iteration, a fourth mapping relationship is established between the first training progress of the global model and the second training progress of each local model. Based on the fourth mapping relationship and all the first mapping relationships, an optimization target for the training progress of the global model is constructed. The training progress is determined based on the rate of change of the training loss of the corresponding model. The optimization objective of the optimized model is obtained by weighted fusion of the node benefit optimization objective of each local model, the training loss optimization objective of the global model, and the training progress optimization objective.

3. The federated learning node resource allocation method as described in claim 2, characterized in that, The node benefit optimization objective is specifically as follows: in, For local models The goal of optimizing node benefits; , , as well as These are the impact coefficients for the corresponding bandwidth resource requirements, latency resource requirements, computing power resource requirements, and reliability indicators, respectively. For local models bandwidth resource requirements; For local models The latency resource requirements; For local models The demand for computing resources; For local models Reliability indicators; It is a natural number.

4. The federated learning node resource allocation method as described in claim 1, characterized in that, The second mapping relationship includes: a fifth mapping relationship between the amount of training data and the bandwidth resource requirement, a sixth mapping relationship between the number of model parameters and the latency resource requirement, and a seventh mapping relationship between the training setting parameters and the computing power resource requirement.

5. The federated learning node resource allocation method as described in claim 4, characterized in that, The second mapping relationship between the node resource requirements for each local model during a single training iteration and the corresponding training settings parameters includes: The fifth mapping relationship is constructed by evaluating the bandwidth resources required to transmit different amounts of training data. The sixth mapping relationship is constructed by evaluating the minimum latency required to update different numbers of model parameters; The seventh mapping relationship is constructed by evaluating the impact of each parameter in the training settings on the computing power requirement during training, and by weighting and fusing each parameter in the training settings according to the impact.

6. The federated learning node resource allocation method as described in claim 5, characterized in that, The seventh mapping relationship is specifically as follows: in, For local models The demand for computing resources; , as well as These are the coefficients representing the impact of the number of model parameters, the amount of training data, and the number of training iterations per session on the computational resource requirements. , as well as Local models The number of model parameters, the amount of training data, and the number of training iterations per session.

7. The federated learning node resource allocation method as described in claim 1, characterized in that, The step of allocating node resources for the next round of training to the corresponding local model according to the optimal training settings parameters includes: Based on the priority of the business processed by each local model, a corresponding priority weight is set for the corresponding local model; Based on the optimal training settings parameters and the second mapping relationship, the resource requirements of the first node corresponding to each local model are determined, and the resource requirements of the first node are determined to determine the required proportion of each local model. Based on the currently available node resources, combined with the priority weights and the demand ratios, the node resources to be used for the next round of training are allocated to each of the corresponding local models.

8. A federated learning node resource allocation device, characterized in that, include: The module includes a first mapping relationship construction module, a second mapping relationship construction module, a training setting parameter optimization module, and a node resource allocation module. The first mapping relationship construction module is used to acquire and determine the first mapping relationship between the training loss of each local model in the next training round and the corresponding training setting parameters based on each local model obtained in the current round of training; wherein the training setting parameters include: the number of model parameters, the number of training data, and the number of training iterations per round. The second mapping relationship construction module is used to construct a second mapping relationship between the node resource requirements of each local model during a single training session and the corresponding training setting parameters used; the node resource requirements include: bandwidth resource requirements, latency resource requirements, and computing power resource requirements; The training setting parameter optimization module is used to construct and solve an optimization model based on the first mapping relationship and the second mapping relationship, with the optimization objective of maximizing node resource utilization in the next round of training, to obtain the optimal training setting parameters for each local model in the next round of training. The node resource allocation module is used to allocate the node resources required for the next round of training to the corresponding local model according to the optimal training setting parameters.

9. A terminal device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement a federated learning node resource allocation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a federated learning node resource allocation method as described in any one of claims 1 to 7.

Images