A method and system for collaborative optimization of wireless communication and computing resources

Abstract

The present application provides a method and system for collaborative optimization of wireless communication and computing resources. The method provided by the present application includes: establishing an initialized global model that matches the business needs for each service provider, and matching the service resources of each service provider according to the business needs; establishing a service agent corresponding to each service provider; each service agent generates an optimal strategy based on the corresponding service resources and the status information of other service providers; sending the initialized global model to the allocation client according to the optimal strategy; each allocation client trains the received model to be trained based on the allocated service resources, and sends the trained model to the corresponding service provider; each service agent aggregates the received trained model to obtain the global model of the corresponding business. The method and system provided by the present application are used to achieve the collaborative optimization of wireless communication and computing resources, and improve the overall performance and resource utilization efficiency of federated learning.

Description

Technical Field

[0001] The present application relates to the field of federated learning technology, and in particular to a method and system for collaborative optimization of wireless communication and computing resources. Background Art

[0002] With the rapid development of smart devices and the Internet of Things, federated learning, as a distributed machine learning framework, has gradually become an important technology for improving model training efficiency and protecting data privacy. Federated learning avoids centralized storage of user data by training models on local devices and only uploading model updates, thereby effectively protecting data privacy. This approach is particularly suitable for scenarios that require user data privacy protection and involve a large number of distributed devices, such as smartphones, sensors, healthcare, and financial services.

[0003] However, in practical applications, the deployment of federated learning often faces limitations in wireless communication and computing resources, especially in non-cooperative environments where multiple service providers (SPs) share network resources, and the resource allocation problem becomes more complicated. With the emergence of multi-service provider environments, the complexity of collaborative optimization of wireless communication and computing resources has greatly increased. Due to privacy protection and competition factors, service providers are often reluctant to share strategies and status information. In addition, resource allocation among multiple service providers is not only affected by the dynamic network environment, but also needs to consider the heterogeneity of tasks, which makes balancing wireless communication efficiency, computing performance and energy consumption a complex multi-objective optimization task. The above factors make it extremely difficult to achieve collaborative optimization in this environment.

[0004] Existing traditional methods, especially centralized optimization algorithms and simple multi-agent reinforcement learning frameworks, are unable to effectively cope with the complexity of such non-cooperative environments. Especially in the early stages of training, these methods often exhibit poor coordination and are even prone to falling into suboptimal equilibrium, resulting in insufficient utilization of wireless communication and computing resources and failure to fully realize the potential of the system. In addition, due to the existence of multi-dimensional discrete-continuous hybrid action spaces, traditional optimization algorithms have poor scalability in high-dimensional scenarios, further limiting their application in practical environments.

[0005] Therefore, there is an urgent need for a method to solve the complexity of wireless communication and computing resource allocation in a non-cooperative environment with multiple service providers, to achieve coordinated optimization of wireless communication and computing resources, and to improve the overall performance and resource utilization efficiency of federated learning. Summary of the invention

[0006] In view of this, the present application provides a method and system for collaborative optimization of wireless communication and computing resources to solve the complexity of wireless communication and computing resource allocation in a non-cooperative environment of multiple service providers, realize collaborative optimization of wireless communication and computing resources, and improve the overall performance and resource utilization efficiency of federated learning.

[0007] Specifically, the present application is implemented through the following technical solutions:

[0008] A first aspect of the present application provides a method for collaborative optimization of wireless communication and computing resources, the method comprising:

[0009] Establishing an initialization global model matching the business requirements for each service provider in the service provider set, and matching service resources of each service provider according to the business requirements; the service provider set includes multiple service providers, and there is no cooperative relationship between each service provider;

[0010] Establish a service agent corresponding to each service provider;

[0011] The service agent corresponding to each service provider generates an optimal strategy based on the corresponding service resources and status information of other service providers, wherein the optimal strategy is the allocation method of the service resources and the allocation client;

[0012] Wherein, each service agent predicts the virtual strategies of other service providers and generates the optimal strategy of the current service agent based on the virtual strategies;

[0013] sending the initialized global model to the allocation client according to the optimal strategy;

[0014] Each of the allocated clients trains the received models to be trained based on the allocated service resources, and sends the trained models to the corresponding service providers;

[0015] Each service provider receives the trained model of the allocated client corresponding to the optimal strategy, and each of the service agents is aggregated to obtain a global model of the corresponding business.

[0016] A second aspect of the present application provides a wireless communication and computing resource collaborative optimization system, the wireless communication and computing resource collaborative optimization system comprising a service provider set and an allocation client; wherein the service provider set comprises a plurality of service providers, and there is no cooperative relationship between each service provider;

[0017] The service provider set is used to establish an initialization global model matching the business requirements for each service provider, and match the service resources of each service provider according to the business requirements;

[0018] The service provider set is further used to send the initialized global model to the allocation client according to the optimal strategy; wherein the optimal strategy is the allocation method of the service resources and the allocation client;

[0019] The service agent corresponding to each service provider generates the optimal strategy based on the corresponding service resources and status information of other service providers; each service agent predicts the virtual strategy of other service providers and generates the optimal strategy of the current service agent based on the virtual strategy;

[0020] The allocation client is used to train the received model to be trained based on the allocated service resources, and send the trained model to the corresponding service provider;

[0021] Each of the service providers is further configured to receive a trained model of the allocated client corresponding to the optimal strategy, and aggregate the trained models to obtain a global model of the corresponding business.

[0022] The wireless communication and computing resource collaborative optimization method and system provided by the present application, firstly, in the scenario of multiple service providers, if each service provider is in a non-cooperative relationship, it usually makes the best decision for the node alone, which will lead to the situation of decision conflict in the overall environment of multiple service providers. In the method provided by the present invention, each service provider is a separate service agent, and each service agent predicts the virtual optimal strategy of other service providers, and generates the current optimal strategy in combination with the state value function. This prediction not only considers local information, but also combines the possible strategy choices of other service providers. By predicting the virtual optimal strategy of other service providers, the foresight of the optimal strategy can be effectively improved. Through this method, the service provider can not only consider the current local optimal solution, but also optimize it according to the possible competitive environment, so that the optimal strategy is more adaptable, which can reduce the probability of the system falling into the local optimal solution, enhance the globality and long-term nature of the decision, realize the collaborative optimization of wireless communication and computing resources, and improve the overall performance and resource utilization efficiency of federated learning. Secondly, each service provider establishes a corresponding service agent and generates the optimal strategy independently. These service agents make independent decisions based on the local resource status and the status information of other service providers. Since there is no cooperative relationship between service providers, independent decision-making can protect their privacy and competitive interests and avoid direct resource sharing or strategy exchange. Intelligent agents can independently optimize based on real-time data, reducing external interference in the decision-making process and improving individual optimization efficiency, so that each service provider can maximize performance based on its own resources without external constraints, avoiding coordination problems that may be caused by centralized decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of a method for collaborative optimization of wireless communication and computing resources provided in Embodiment 1 of the present application;

[0024] Figure 2 A schematic diagram of the structure of the wireless communication and computing resource collaborative optimization system provided in Example 2 of the present application. DETAILED DESCRIPTION

[0025] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0026] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items.

[0027] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0028] Specific embodiments are given below to introduce the technical solution of the present application in detail.

[0029] Figure 1 This is a flow chart of the method for collaborative optimization of wireless communication and computing resources provided in Example 1 of the present application. Figure 1 , the method provided in this embodiment may include:

[0030] S101. Establishing an initialization global model matching business requirements for each service provider in a service provider set, and matching service resources of each service provider according to the business requirements; the service provider set includes multiple service providers, and there is no cooperation relationship between each service provider.

[0031] Specifically, the service provider set includes multiple service providers. A service provider refers to an entity that provides specific services or resources in a distributed system. There is no cooperative relationship between each service provider. Business requirements include processing specific types of data (such as video, audio, text, etc.), providing specific service functions (such as reasoning, training, data processing, etc.), and responding to service quality requirements (such as latency, bandwidth, throughput, etc.).

[0032] It should be noted that, since there are multiple service providers, in this embodiment, only one service provider is used as an example to introduce how to achieve coordinated optimization of wireless communication and computing resources. Other service providers are similar and will not be described in detail here.

[0033] In the specific implementation, the service type and target required by each service provider in the service provider set, that is, the business demand, is determined. Furthermore, according to the business needs of each service provider, an initialization global model is constructed for each service provider, taking into account factors such as data sets, computing power, bandwidth, and latency. This initialization global model is customized according to the specific needs of the service provider. If the service provider needs to process video data, the initialization global model may focus on image processing tasks and select appropriate algorithms and parameters. If the service provider needs to perform real-time data processing, the design of the initialization global model will specifically consider latency and bandwidth optimization. Furthermore, based on the computing requirements and real-time requirements of each service provider, the service resources of each service provider are determined. For example, for service providers that need to process large amounts of data, more computing resources (such as powerful GPU clusters or high-performance computing servers) should be allocated. For service providers with higher real-time requirements, sufficient bandwidth and low-latency network connections should be ensured.

[0034] It should be noted that different service providers have different business requirements, and their corresponding initialization global models and service resources are also different. For example, there are two service providers, one responsible for video stream processing and the other responsible for text analysis. The initialization global model of the first service provider may require more computing resources (such as GPU) to process video data while ensuring sufficient bandwidth to cope with high bandwidth requirements. The second service provider may require fewer computing resources but lower latency to analyze text data in real time.

[0035] S102: Establish a service agent corresponding to each service provider.

[0036] Specifically, the service agent is used to predict the optimal strategy of the service provider.

[0037] In specific implementation, the establishment of a service agent corresponding to each service provider includes:

[0038] (1) Generate the state space of the current service agent based on its own business status information and global observation information.

[0039] Specifically, global observation information refers to environmental data that the current service agent (i.e., service provider) can directly perceive and use, and is the status data of the service provider. Global observation information includes the current round of training loss, model accuracy, quantization level, computing resources, and wireless communication resource usage. Self-service status information refers to information related to the service agent's own business operations, including order processing status, service delivery status, resource usage, etc. Global observation information and self-service status information directly constitute the input elements of the state space.

[0040] In specific implementation, the current service agent (service provider) regularly obtains real-time feedback from each client (such as training loss, resource usage, etc.). In addition to real-time aggregation, the current service agent can also collect historical observation data (for example, the client's historical task completion, communication history, resource usage, etc.), and aggregate the collected status data of all clients to form global observation information and its own business status information. With global observation information and its own business status information as input elements, the state space of the current service agent is constructed to represent different resource usage, task allocation, performance evaluation, etc. The constructed state space can be expressed as:

[0041] ;

[0042] Among them, the is the state space; Information related to local training, including the training loss of the current round, the accuracy of the current model, the current quantization level used, etc. is the current training status, including total training time, total energy consumption, and wireless communication resource overhead; The current global bandwidth allocation situation; is the service provider's number; For the federated learning round; It is a business model parameter.

[0043] (2) Determine the candidate client set and service resource allocation method corresponding to the current service agent as the action space.

[0044] Specifically, the service resource allocation method includes allocation of wireless communication resources and allocation of computing resources.

[0045] In the specific implementation, according to the task requirements and resource availability of the current service agent, a set of suitable candidate clients is selected based on factors such as location, computing power, and communication bandwidth. Furthermore, the service resource allocation method for each candidate client is determined based on factors such as task priority and current resource status. The action space can be expressed as:

[0046] ;

[0047] Among them, the is the action space; is a candidate client set; Allocate wireless communication resources; For computing resource allocation; is the quantization order; is the service provider's number; For the federated learning round.

[0048] (3) Establish a multi-objective reward function based on communication evaluation, task completion evaluation, and task completion energy consumption evaluation.

[0049] Specifically, the multi-objective reward function includes a balance of multiple objectives, including communication evaluation (evaluating the communication overhead involved in the task completion process), task completion evaluation (evaluating the time to complete the task on the client and the quality of the task completion), and task completion energy consumption evaluation (evaluating the energy consumed during the execution of the task, including energy consumption during the calculation and communication process), including wireless communication efficiency, computing performance, energy consumption, task completion time and other indicators. Among them, the wireless communication efficiency is represented by the adversarial factor, which quantifies the level Incorporate optimization to balance wireless communication overhead and function convergence speed through dynamic adjustment. At the same time, introduce the quantitative resource usage of other service providers into the confrontation factor ,Modeling non-cooperative relationships among multiple service providers.

[0050] In specific implementation, after selecting any action in the action space, multiple indicators such as communication evaluation, task completion evaluation, and task completion energy consumption evaluation are obtained based on the execution of the action, and a multi-objective reward function is calculated based on these multiple indicators. Specifically, the product of wireless communication efficiency and the corresponding first weight is determined as the first reward value, and the product of computing performance and the corresponding second weight is determined as the second reward value; the product of energy consumption and the corresponding third weight is determined as the third reward value, and the product of task completion time and the corresponding fourth weight is determined as the fourth reward value; the sum of the first reward value and the second reward value is determined as the fifth reward value, and the sum of the third reward value and the fourth reward value is determined as the sixth reward value; the difference between the fifth reward value and the sixth reward value is determined as the reward value corresponding to the current action.

[0051] The established multi-objective reward function can be expressed as:

[0052] ;

[0053] Among them, the is a multi-objective reward function; is the first weight; is wireless communication efficiency; is the second weight; For computing performance; is the third weight; is energy consumption; is the fourth weight; For delay.

[0054] (4) Taking the state space as input and the multi-objective reward function as the target, a decision framework of the current service agent is established using the Markov decision process.

[0055] Specifically, in the Markov decision process, the state space (including the current state of the service agent) is the input, the action space (candidate clients and resource allocation strategies) is the possible actions, and the multi-objective reward function is used to evaluate the effect of taking a certain action in a certain state.

[0056] In specific implementation, the decision framework of the current service agent is established by using the Markov decision process according to the defined state space, action space and multi-objective reward function. The reinforcement learning algorithm is used to continuously update and optimize the strategy through interaction with the environment. After each action is selected, the service agent adjusts the strategy according to the reward obtained to maximize the cumulative reward. That is, in a given state space and action space, the service agent will evaluate the expected reward of each action in the current state and select an action that maximizes the cumulative reward. By updating the Q value and the policy network, the service agent continuously estimates the value of each state-action pair. This estimate is corrected by accumulating the discount of future rewards to optimize long-term decisions. The method provided in this embodiment introduces an adversarial factor in the multi-objective reward function when establishing the service agent corresponding to the service provider, which can effectively simulate the competitive relationship between the service providers. Because each service provider is non-cooperative, by introducing the adversarial factor, the service agent not only makes decisions based on its own resources and status, but also predicts and responds to the strategies of other service providers, thereby avoiding inefficient "prisoner's dilemma" behavior and promoting more efficient operation of the entire system. In addition, by incorporating the quantization level into the optimization, since there is often a trade-off between communication overhead and model accuracy in machine learning tasks. By dynamically adjusting the quantization level, the service agent can reduce the quantization level to reduce the communication burden when the bandwidth is limited, or improve the quantization accuracy to accelerate model training when the bandwidth is sufficient, thereby achieving a balance between wireless communication overhead and model convergence speed. This dynamic optimization strategy helps to flexibly adjust the use of resources according to the current communication conditions and task requirements, and effectively improve the system performance. In addition, the introduction of resource usage of other service providers also adds a competitive dimension to the decision-making process. In a non-cooperative multi-service provider environment, understanding the resource usage of other service providers can help each service agent more accurately evaluate the current resource availability and conflict, and avoid excessive competition or waste of resources. This design not only optimizes the efficiency of resource allocation, but also improves the stability and fairness of the system. Under the design of the multi-objective reward function, the system can simultaneously consider multiple optimization objectives, such as communication evaluation, task completion evaluation, and energy consumption evaluation. This multi-objective optimization can handle various constraints in complex environments, allowing service agents to make trade-offs between multiple objectives, thereby improving the overall resource utilization efficiency and system performance. Finally, a Markov decision process is used to establish a decision framework for each service agent, allowing the service agent to make the optimal strategy selection based on the observed state, predicted future results and accumulated rewards. This enhances the adaptability and robustness of the system, enabling each service agent to continuously optimize resource allocation in a changing network environment.

[0057] S103: The service agent corresponding to each service provider generates an optimal strategy based on the corresponding service resources and status information of other service providers. The optimal strategy is the allocation method of the service resources and the allocation client.

[0058] Each service agent predicts the virtual strategies of other service providers and generates the optimal strategy of the current service agent based on the virtual strategies.

[0059] Specifically, the optimal strategy includes the allocation method of service resources and the allocation client. Among them, service resources include wireless communication resources (bandwidth) and computing resources (CPU). Furthermore, in addition to wireless communication resources and computing resources, service resources also include resources related to model quantization, including quantization order.

[0060] In specific implementation, the service agent corresponding to each service provider generates an optimal strategy based on the corresponding service resources and the status information of other service providers, including: each service agent obtains the current operation status of the corresponding service and the observed global information; traverses all strategy combinations of other service agents as a virtual joint strategy set; each service agent constructs an optimal virtual joint strategy based on the virtual joint strategy set and the real-time decision of the current service agent; establishes the cumulative reward of the service agent under the optimal virtual joint strategy; solves the Pareto optimal decision that makes the cumulative reward optimal as the joint optimal strategy of all service agents.

[0061] Specifically, when training the service agent, the service agent corresponding to each service provider predicts all possible strategy combinations of other service providers as a virtual joint strategy set. The optimal virtual joint strategy is constructed based on the virtual joint strategy set and the real-time strategy of the current service agent generated by its own observation. In specific implementation, the predicted virtual joint strategy set of other service agents and the real-time decision of the current service agent are combined to obtain the optimal virtual joint strategy. The optimal virtual joint strategy can be expressed as:

[0062] ;

[0063] Among them, the is the optimal virtual joint strategy; Real-time decision making for the current service agent; A virtual federation strategy set for other service agents.

[0064] Furthermore, the cumulative reward of the current service agent under the optimal virtual joint strategy is established, and the cumulative reward is calculated through the Markov decision process. The reward of each action is discounted at each time step according to the current state and action selection. The cumulative reward can be expressed as:

[0065] ;

[0066] Among them, the For cumulative rewards; for expectation; is the current service agent in the initial state Joint Virtual Strategy The state value function under Number the service agent; For virtual joint strategy; is the discount factor; is a multi-objective reward function; is the initial state; is the service provider's number; For the federated learning round.

[0067] Finally, based on the optimal virtual joint strategy of other service agents, the strategy of the current service agent is optimized. In specific implementation, the Pareto optimal decision that optimizes the cumulative reward is solved as the joint optimal strategy of all service agents.

[0068] ;

[0069] Among them, the To serve intelligent agents The joint optimal strategy of In addition to the service agent A virtual joint strategy set of other service agents.

[0070] In the specific implementation, all possible action combinations of the service agent are traversed to determine the combination that maximizes the Q function. The Q function is traversed through argmax as follows:

[0071] .

[0072] Said A virtual joint strategy for other service agents; ; is a multi-objective reward function; is the service provider's number; For the federated learning round; is the discount factor; for expectation; Current service agent for the next state Joint Virtual Strategy The state value function under .

[0073] Optionally, solving the Pareto optimal decision that makes the cumulative reward optimal includes: obtaining an optimal action variable based on the optimal cumulative reward, the optimal action variable including an allocation value related to a service resource allocation method and its corresponding client, the optimal action variable being a continuous quantity; discretizing each element in the optimal action variable; determining a ternary variable corresponding to each discretized element, the representation dimension of the ternary variable being smaller than the dimension corresponding to the discretized element; converting the ternary variable into a Cartesian product to obtain a continuous representation of the optimal action variable, and obtaining a Pareto optimal decision.

[0074] Specifically, the ternary variable is represented by -1, 0, 1, and the ternary variable can be expressed as . Among them, 0 means keeping the current value unchanged, -1 means reducing the current value by one granularity, and 1 means increasing the current value by one granularity. Each ternary variable represents the change between the current resource value and the preset discretization level. The dimension of the ternary variable is smaller than the dimension of the discretized element.

[0075] In specific implementation, by setting discretization rules or discrete intervals, each element in the optimal action variable is discretized into the corresponding interval. For each discretized element, the corresponding ternary variable is determined based on the corresponding discrete interval, and each discretized element is represented by a ternary variable to obtain a triplet corresponding to each optimal action variable. All triples are combined in a Cartesian manner to obtain a discrete action space, and a Pareto optimal decision is made for the discrete action space.

[0076] Specifically, traverse each cumulative reward, determine the optimal cumulative reward with the largest cumulative reward, and determine the action variable corresponding to the optimal cumulative reward as the optimal action variable. Since the optimal action variable is a specific value related to the service resource allocation method. For example, the optimal action variable may include the numerical value of resource allocation (such as computing resources and bandwidth allocated to a certain client) and the corresponding client set. Further, the continuous optimal action variable is converted into a discretized form, and the four elements in each optimal action variable (the number of currently selected clients, the allocated CPU frequency, the wireless communication bandwidth, and the quantization order) are divided into finite discrete values. For example, the continuous range of CPU frequency from 0 to the maximum frequency may be divided into several discrete levels (such as low, medium, and high). After discretizing the optimal action variable, for each discretized element, each element is represented by a ternary variable based on the change between the current resource value and the preset discretization level. Finally, all ternary variables are combined to form a Cartesian product. The Cartesian product can be expressed as Among them, the is the Cartesian product; is a ternary variable; is the number of the ternary variable. Under this continuous representation of Cartesian product, through the game and strategy adjustment between multiple agents, we continuously seek the state that can make the cumulative rewards of all service providers reach the relatively optimal state, that is, the Pareto optimal decision. This decision can maximize the benefits of each service provider without reducing the benefits of other service providers, thereby realizing the coordinated optimization of wireless communication and computing resources of the entire system. For example, in each round of federated learning training, the service provider calculates the combination of ternary variables based on the current state and the strategy guesses of other service agents, and updates its own strategy. After multiple rounds of training, the strategy of the entire system tends to be Pareto optimal.

[0077] It should be noted that some of the optimal action variables are continuous because they may represent specific values ​​of resources (such as bandwidth, CPU frequency, etc.), which usually have the characteristics of continuous change. The size of the action space represented by the Cartesian product in the current scenario is only Through the method provided in this application, the multi-dimensional continuous action space is converted into an easy-to-handle discrete form, which effectively reduces the computational complexity of traversing the Q function, avoids the dimensionality explosion problem, and improves the efficiency and stability of strategy generation.

[0078] The method provided in this embodiment effectively optimizes the collaborative management of wireless communication resources and computing resources by introducing optimal action variables, discretization, ternary variables, and Cartesian product conversion mechanisms when solving Pareto optimal decisions. The combined effect of these designs enables the system to operate effectively in complex multi-objective and multi-constraint problems, improves the collaborative optimization capabilities of wireless communication and computing resources, and has strong scalability and flexibility.

[0079] Optionally, generating the optimal strategy for the current service agent based on the virtual strategy includes: traversing and calculating the expected cumulative rewards of each virtual strategy and real-time action combination in the current calculation cycle based on the action-value function; updating the expected cumulative rewards according to the difference of the iterative calculation cycle; selecting the real-time state and real-time action corresponding to the maximum expected cumulative reward, and obtaining the optimal strategy corresponding to the current service agent.

[0080] Specifically, when making decisions, each service agent needs to evaluate the impact of the current state and action on the future cumulative rewards. The action value function is used to measure the expected cumulative rewards that can be obtained given the current state, current action, and speculation on the actions of other service agents.

[0081] In the specific implementation, the expected cumulative reward of taking this set of actions in the current state is calculated by combining the action value function and the state value function with the current strategy. The service agent continuously adjusts the reward evaluation for different states and actions, and updates the expected cumulative reward by iteratively calculating the period difference, using the time difference algorithm. Specifically, after each calculation cycle, the service agent will update the previous estimate of the expected cumulative reward for the action in a certain state based on the new state, the action taken, and the immediate reward obtained. For example, the error is calculated according to the time difference algorithm, and then the error is used to adjust the parameters of the action value function and the state value function to make the estimate of the expected cumulative reward more accurate. After multiple calculations and updates of the expected cumulative reward, the service agent compares the expected cumulative reward size under different real-time states and real-time action combinations, and selects the real-time state and real-time action that can obtain the maximum expected cumulative reward as the optimal strategy corresponding to the current service agent.

[0082] Optionally, the service agent corresponding to each service provider generates an optimal strategy based on the corresponding service resources and status information of other service providers, including: screening target clients from candidate clients; determining the model quantization level based on the communication bandwidth limitation and task accuracy requirements corresponding to the service agent; determining the resource allocation quota of each client in the target clients according to the task requirements; the resource allocation quota includes a communication resource allocation quota and a computing resource allocation quota; establishing a corresponding relationship between the resource allocation quota and the model quantization level and each client in the target clients to generate the optimal strategy.

[0083] In the specific implementation, according to the business needs and resource conditions of the current service provider, the candidate clients are screened according to the communication bandwidth requirements and task accuracy, and the target clients that meet the communication bandwidth requirements and task accuracy are determined. Furthermore, based on the changing relationship between the communication bandwidth limit and the model quantization level, and the changing relationship between the task accuracy requirement and the model quantization level, the influence of the communication bandwidth limit and the task accuracy requirement on the model quantization level is fitted to determine the model quantization level. According to the task requirements of each target client and the service resource upper limit of the current service agent, the resource allocation quota for each client is determined, and the sum of the resource allocation quotas of all clients is not greater than the service resource upper limit. The resource allocation quota of each target client and the corresponding model quantization level are regarded as a set of corresponding relationships, and a relationship model between the resource allocation quota and the model quantization level is established. Combined with the optimization goal of the current service agent, the optimal strategy is generated.

[0084] Specifically, the service agent receives the status information of other service providers, and based on factors such as communication bandwidth limitations and task accuracy requirements, the service agent selects the target client from the candidate clients (the priority of the task, the computing resource requirements, and the communication resource limitations may be considered when selecting the target client). Further, the service agent uses its service resources and status information to evaluate the upper limit of the communication bandwidth, while taking into account the accuracy requirements of the task, such as accuracy requirements and model complexity limitations. Based on the current communication bandwidth limitations and task accuracy requirements, the service agent determines the model quantization level to be used. For example, low-precision quantization may reduce model accuracy to reduce communication burden, while high-precision quantization will improve accuracy but may increase the communication burden of transmission. The service agent determines the resource allocation quota for each client based on the task requirements, current resource usage, and the status information of the target client, taking into account the specific resource usage of the target client (such as CPU frequency, memory, communication bandwidth, etc.), as well as the service resource allocation limitations of the service agent. By combining the task requirements and the current resource limitations, the resource quota of each client can be accurately allocated to ensure that the resource allocation can meet the computing and communication requirements of the task. For example, the service agent may allocate a specific amount of CPU frequency to a target client, as well as communication bandwidth and storage resources to support the operation of the task. Finally, based on the decision of the service agent, a mapping relationship corresponding to the target client resource allocation and the model quantization level is established. The establishment of this mapping relationship reflects how each target client obtains resources under specific task requirements and how it affects the allocation of service resources and communication overhead. The service agent generates the optimal strategy based on the resource allocation amount and quantization level of each target client. For example, if a target client requires high-precision training but low communication bandwidth, the service agent may decide to use a lower quantization level to reduce the communication burden, while appropriately adjusting other resource allocations to support the completion of the task.

[0085] Optionally, each service agent generates the optimal strategy based on the trained prediction model, i.e., the allocation method of service resources and the allocation of clients, which needs to balance multi-objective optimization such as wireless communication overhead, computing delay, energy consumption, and model accuracy, namely:

[0086] ;

[0087] Among them, the is the cost function of each round; For energy consumption and time constraints; Select a scope for the client; For computing resource constraints; For bandwidth limitation; is the quantitative level limit; is the loss value of the current round; is the wireless communication overhead; is the total energy consumption; Maximum energy consumption; is the total training time; is the maximum time; is a candidate client set; is the maximum number of candidate clients; Allocate wireless communication resources; is the minimum wireless communication resource allocation; is the maximum wireless communication resource allocation; For computing resource allocation; is the minimum computing resource allocation; is the maximum computing resource allocation; is the quantization order; is the service provider's number; For the federated learning round.

[0088] The prediction model is built based on the Pareto Actor-Critic algorithm and can be expressed as the PAC-McoFL model. During training, its input is the total number of training rounds , the initial network parameters of the actor serving the agent and critic initial network parameters , Global Model of Federated Learning , global training rounds and client collection The output is the actor network parameters after service agent training and critic network parameters .

[0089] The training process of the prediction model is as follows: First, load the training dataset of the federated learning service: .when When the current training data set is smaller than the maximum training data set, the global model parameters are initialized. and client characteristics, including . In each round of training, get the observation value and From the strategy Medium sampling action , while from the strategy Sampling actions of other service agents For each selected client, perform local update according to Formula 1 to obtain ; Calculated according to Formula 2 and Formula 3 and ; Calculated according to Formula 4 and Formula 5 and After training is completed, the global model is updated according to the formula: Distribute the updated global model to all clients and obtain Calculated according to Formula 6 and ; Calculated according to formula 7 and ; Calculate the instant reward according to formula 8 ;Will Put into the current training batch and insert the current training batch into the previous playback buffer.

[0090] When the replay buffer reaches the buffer size required for training, a batch is selected from it to obtain the virtual joint strategy ; Update the critic network according to formula 9, and update the actor network according to formula 10.

[0091] Formula 1: Among them, the For Clients Under the management of the service provider, The first round of federated learning The weight after the local model parameter is updated; For Clients Under the management of the service provider, The first round of federated learning The weight after the local model parameter is updated; For the The learning rate of the local update; For Clients Gradient estimate of the loss function for local data, using the model parameters from the previous round , calculate the gradient of the loss function and adjust the update step size through the learning rate.

[0092] Formula 2: Among them, the For Clients In the In the round of training, the local computing energy consumption of the service provider is used; The effective capacitance constant determined by the chip architecture; For Clients The size of the local data set under the service provider; For Clients The number of CPU cycles to execute a sample of the service provider; For Clients The allocated CPU frequency.

[0093] Formula 3: Among them, the For Clients In the In round training, the local computation latency used by the service provider; For Clients The number of CPU cycles to execute a sample of the service provider; For Clients The CPU frequency assigned; For Clients The size of the local data set under the service provider.

[0094] Formula 4: Among them, the For Clients In the In round training, wireless communication delay for service providers; For Clients In the In the round of training, the wireless communication overhead of the service provider is for the purpose of For Clients In the The transmission rate to the service provider in the round of training.

[0095] Formula 5: Among them, the For Clients In the In the round of training, the wireless communication energy consumption used by the service provider; For Clients In the The transmission power of wireless communication in round training; For Clients In the In round training, wireless communication delay for service providers; For Clients In the In the round of training, the wireless communication overhead of the service provider is for the purpose of For Clients In the The transmission rate to the service provider in the round of training.

[0096] Formula 6: Among them, the For the The total energy consumption of the round training service provider; is a client collection; For Clients In the In the round of training, the wireless communication energy consumption used by the service provider; For Clients In the In the round of training, the local computing energy consumption of the service provider is used; For the The total delay of the round training service provider; For Clients In the In round training, wireless communication delay for service providers; For Clients In the In the training round, the local computation latency of the service provider is used.

[0097] Formula 7: Among them, the is the state space; Information related to local training, including the training loss of the current round, the accuracy of the current model, the current quantization level used, etc. is the current training status, including total training time, total energy consumption, and wireless communication resource overhead; The current global bandwidth allocation situation; is the service provider's number; For the federated learning round; is the number of training times.

[0098] Formula 8: Among them, the is a multi-objective reward function; is the first weight; is wireless communication efficiency; is the second weight; For computing performance; is the third weight; is energy consumption; is the fourth weight; The time when the task is completed.

[0099] Formula 9: Among them, the is the critic network parameter of the service provider; is the learning rate, used to update the critic network; is the time difference error About critic network parameters gradient.

[0100] Formula 10: Among them, the is the actor network parameter of the service provider; is the learning rate, used to update the actor network; To accumulate expected rewards About actor network parameters gradient.

[0101] As an optional embodiment, determining the model quantization level also includes: determining a quantization interval based on the task accuracy requirements and communication bandwidth limitations corresponding to the service agent; determining the computing power of each target client among the multiple target clients selected by the service agent, and screening candidate quantization intervals from the quantization intervals based on the computing power of each target client, wherein the span of the candidate quantization interval is smaller than the quantization interval, and the candidate quantization interval is included in the quantization interval; determining the model quantization level from the candidate quantization intervals according to the computing efficiency requirements corresponding to the service agent and the computing power of each target client, and the model quantization levels of different target clients may be completely the same or may not be completely the same.

[0102] The method provided in this embodiment can select a suitable model quantization level for each target client by comprehensively considering the computing power, task accuracy requirements and communication bandwidth limitations of each target client, thereby achieving more efficient resource use and optimization. First, by screening the quantization interval according to the computing power of each target client, the performance bottleneck caused by the large difference in computing power between different target clients can be avoided, ensuring that each target client can operate efficiently within its resource capacity. Secondly, limiting the difference in quantization levels can ensure that the quantization levels between target clients are not much different, which can effectively reduce the complexity and computational burden of subsequent fusion of models trained by multiple target clients, and avoid the increase in fusion workload caused by large quantization differences. By refining the selection and screening process of quantization levels, it is possible to optimize the allocation of communication and computing resources while ensuring task accuracy and computing efficiency, and reduce unnecessary overhead. In addition, dynamically screening and adjusting the span of the quantization interval also enhances the adaptability of the system, enabling it to be flexibly adjusted according to different task requirements and network conditions, further improving the overall performance of the system. This method can optimize the quantization level of each target client within a reasonable range, thereby improving the computing efficiency of the entire system, reducing communication overhead, and improving the efficiency of model training and fusion, ultimately achieving effective coordinated optimization of wireless communication and computing resources.

[0103] S104: Send the initialized global model to the allocation client according to the optimal strategy.

[0104] Specifically, combined with the above description, the optimal strategy includes the allocation method of service resources and the allocation client. Among them, the allocation method represents how to allocate wireless communication resources (such as bandwidth) and computing resources (such as CPU) to the client, and the allocation client refers to which clients will be allocated corresponding service resources. Each allocated client corresponds to a designated service provider. The allocated client finally receives not only the initialized global model, but also the service resources. The service resources are determined according to the allocation method of the service resources (that is, the allocation plan of wireless communication resources and computing resources), and the initialized global model is subsequently trained based on the service resources.

[0105] In specific implementation, each service provider determines a suitable client to receive the initialized global model according to the allocated client in the optimal strategy determined by the corresponding service agent. Further, after determining the allocated client, the service resources are determined according to the allocation method of the service resources in the optimal strategy. Finally, each service provider sends the initialized global model and service resources to the determined allocated client.

[0106] S105: Each of the allocated clients trains the received models to be trained based on the allocated service resources, and sends the trained models to the corresponding service providers.

[0107] Specifically, in combination with the above description, the service resources include wireless communication resources and computing resources. It should be noted that different allocation clients may correspond to different service resources. Therefore, when different allocation clients train the to-be-trained model based on the corresponding service resources, the trained models obtained are different.

[0108] In a specific implementation, each of the allocated clients trains the received models to be trained based on the allocated service resources, and sends the trained models to the corresponding service providers, including:

[0109] (1) Based on the optimal strategy, the received model to be trained is split into multiple training subtasks; each training subtask corresponds to an independent training module.

[0110] Specifically, each training subtask is usually a different part of the model to be trained or a different computing module, and each training subtask is responsible for a part of the model. For example, when the model to be trained is a deep learning model, each training subtask is responsible for an independent layer or module (such as a convolutional layer, a fully connected layer, etc.) of the model to be trained.

[0111] In the specific implementation, the model structure of the model to be trained is analyzed to obtain the model's hierarchical relationship and module division information. Based on the model structure, the model to be trained is split into multiple training subtasks according to logical modules. For each training subtask after splitting, based on the comparison of the resource requirements of each training subtask with the allocation method of service resources in the optimal strategy, the training subtask with relatively large resource requirements is further refined and divided into training subtasks with smaller resource requirements to ensure that the divided training subtasks can match the optimal strategy.

[0112] (2) Allocating initial resources to the training modules based on the resource requirements of each training module and the optimal strategy.

[0113] Specifically, each training module will have different resource requirements during the training process.

[0114] In specific implementation, the resource requirements of each training module are evaluated based on factors such as the complexity of the training module, the size of the corresponding training subtask, and the amount of computation. For example, training subtasks with large computational loads (such as forward propagation and back propagation of deep neural networks) may require more computing resources (such as CPU), while training modules with large data transmission volume may require more bandwidth. Further, based on the resource requirements of the training modules and the optimal strategy, the allocation client decides to allocate appropriate initial resources to each training module. Generally, the higher the resource requirements, the higher the initial resources allocated, but the allocated initial resources need to be less than the service resources allocated to the allocation client in the optimal strategy. For example, for training modules with higher task priorities (or higher resource requirements), more initial resources (such as higher CPU frequency or more GPU memory) will be allocated, while for tasks with lower priority (or lower resource requirements), the initial resource allocation may be less.

[0115] (3) Each training module trains the corresponding training subtask based on the local data set and the initial resources. During the training process, the working frequency and resource allocation are adjusted in real time based on the current resource status until all training modules complete the training and each trained subtask is sent to the corresponding service provider.

[0116] In specific implementation, after each training module is allocated the initial resources, it needs to train the corresponding training subtask locally. During the training, each training module performs local training on the training subtask based on the allocated initial resources. During the training process, the training module will update the training subtask according to its local data set, adjust the weight, bias or other parameters, stop training when the number of iterations reaches a large value, obtain the trained subtask, and send the trained subtask to the corresponding service provider.

[0117] It should be noted that during the training process, the allocation client will monitor the current resource status of each training module (such as CPU, memory usage, bandwidth consumption, etc.) in real time, and determine whether to adjust the working frequency and resource allocation of the training module based on the current resource status.

[0118] In specific implementation, the working frequency and resource allocation are adjusted in real time based on the current resource status during the training process, including: evaluating the resource requirements of each training module based on the current resource status; determining the resource usage of each training module based on the resource requirements; when the resource requirement of the target training module exceeds the resource upper limit, the allocation client adjusts the working frequency of the target training module, and reduces the resource allocation of other training modules in turn according to the resource usage.

[0119] Specifically, the allocation client determines the resource requirements and priority of each training module by monitoring the current resource status of each training module. Compare the resource requirements of each training module with the current resource allocation to determine the resource usage of each training module, including sufficient resources, tight resources, and resource overruns. For the target training module that exceeds the resource upper limit, that is, the target training module with overrun resources, the operating frequency of the target training module is reduced according to the frequency range supported by the training module, and the adjusted operating frequency needs to be within the frequency range supported by the training module. When the adjusted target training module is still overrun, at this time, the resource allocation of the training module with sufficient resources is reduced in turn according to the resource usage, and the resource allocation of the target training module is increased. After one adjustment, the resource usage of the target training module is detected, and the process is repeated. For example, in this step, the resource allocation of the training module with sufficient resources is first reduced, and the resource allocation of the target training module is increased. If the target training module is still overrun at this time, the resource allocation of the training module with tight resources is further reduced.

[0120] Optionally, before sending the trained model to the corresponding service provider, the method also includes: each service provider determines the quantization order of the corresponding assigned client based on status information and task requirements; and each of the assigned clients quantizes the trained model based on the quantization order.

[0121] Specifically, the status information includes computing resources, network conditions, device capabilities, etc. Different status information will affect the selection of quantization order: the computing resources of the service provider determine the quantization accuracy it can carry. Service providers with limited computing resources (such as lower CPU frequency) may not be able to efficiently process high-precision models, so they need to choose a lower quantization order (such as 8-bit quantization). Service providers with more abundant computing resources (such as high-performance devices or edge nodes) can withstand higher quantization orders (such as 16-bit or 32-bit quantization). For service providers with sufficient computing resources, higher quantization orders (such as 16-bit, 32-bit) are usually selected to ensure the training effect and performance of the model. For service providers with limited computing resources, lower quantization orders (such as 8-bit) are usually selected to reduce computing burden and storage requirements. For service providers with limited network bandwidth, lower quantization orders need to be selected to reduce model size and data transmission volume. For service providers with sufficient bandwidth, higher quantization orders can be selected to less affect data transmission speed. For network environments with high latency or packet loss rates, models with low quantization orders are more adaptable to this situation because they have less data redundancy and higher transmission reliability. For low latency and reliable networks, a higher quantization order can be selected to maintain higher training accuracy.

[0122] Furthermore, task requirements refer to the different requirements of different tasks for training accuracy and computing resources. The priority of the task and the requirement for accuracy will affect the choice of quantization order. For some tasks with high accuracy requirements (such as image recognition, speech recognition, etc.), a higher quantization order can ensure less error and improve model performance. For some tasks that require fast response and real-time processing (such as edge computing tasks, IoT device control, etc.), there may be high requirements for computing resources and time. Choosing a lower quantization order can reduce the time and resource consumption of model training and improve response speed.

[0123] In specific implementation, by collecting the status information and task requirements of the service provider, the first quantization order is comprehensively determined based on the relationship between each status parameter in the status information (such as computing resources, network conditions, and equipment capabilities) and the quantization order. The first quantization order is adjusted based on the accuracy requirements and real-time requirements of the task requirements. After determining the quantization order, the client is assigned to quantize the trained model, and the floating-point parameters in the model are converted into integers of a specified bit width using a fixed-point quantization method to obtain the quantized model.

[0124] S106: Each service provider receives the trained model of the allocated client corresponding to the optimal strategy, and each of the service agents is aggregated to obtain a global model of the corresponding business.

[0125] In specific implementation, for each service provider, the corresponding assigned client is determined based on the optimal strategy corresponding to the service provider. Each service provider receives the trained models sent by all the assigned clients corresponding to it. After receiving all the trained models, all the trained models are weighted aggregated, verified, analyzed, etc. to ensure that the aggregated global model can reflect the training update process of all the assigned clients.

[0126] In the method provided in this embodiment, the service provider sends the initialized global model to the allocated client according to the optimal strategy, and the allocated client performs local training based on the allocated service resources, and returns the trained model to the service provider. After receiving the trained model, the service provider aggregates it and finally generates a global optimization model. This distributed training method reduces the computing burden of the central server, makes computing and storage tasks more decentralized, reduces the risk of single point failures, and improves the robustness of the system. At the same time, local training can better utilize the computing power and local data of the client, avoiding data transmission bottlenecks and delays that may occur in centralized training methods. The final global model is more in line with business needs and the training speed is faster.

[0127] The wireless communication and computing resource collaborative optimization method provided in this embodiment, firstly, in the scenario of multiple service providers, if each service provider is in a non-cooperative relationship, usually makes the best decision for the node alone, which will lead to the situation of decision conflict in the overall environment of multiple service providers. In the method provided by the present invention, each service provider is a separate service agent, and each service agent predicts the virtual optimal strategy of other service providers, and generates the current optimal strategy in combination with the state value function. This prediction not only takes into account local information, but also combines the possible strategy choices of other service providers. By predicting the virtual optimal strategy of other service providers, the foresight of the optimal strategy can be effectively improved. Through this method, the service provider can not only consider the current local optimal solution, but also optimize it according to the possible competitive environment, so that the optimal strategy is more adaptable, which can reduce the probability of the system falling into the local optimal solution, enhance the globality and long-term nature of the decision, realize the collaborative optimization of wireless communication and computing resources, and improve the overall performance and resource utilization efficiency of federated learning. Secondly, each service provider establishes a corresponding service agent and generates the optimal strategy independently. These service agents make independent decisions based on the local resource status and the status information of other service providers. Since there is no cooperative relationship between service providers, independent decision-making can protect their respective privacy and competitive interests and avoid direct resource sharing or strategy exchange. The intelligent agent can independently optimize according to real-time data, reduce external interference in the decision-making process, improve individual optimization efficiency, and enable each service provider to maximize performance based on its own resources without external constraints, avoiding coordination problems that may be caused by centralized decision-making. Thirdly, when solving the Pareto optimal decision, the collaborative management of wireless communication resources and computing resources is effectively optimized by introducing the optimal action variable, discretization, ternary variables and Cartesian product conversion mechanism. When solving the optimal decision, the optimal action variable associated with the service resource allocation and the client is first obtained through the optimal cumulative reward, which ensures that the resource allocation scheme can maximize the overall benefit of the system, not just the local optimum. Then, by discretizing the optimal action variable, the complex continuous decision space is simplified to a finite discrete space, which significantly reduces the computational complexity while ensuring the computational accuracy and enhances the computational efficiency of the system. In addition, mapping the discretized elements to ternary variables further reduces the dimension, making the representation of the problem more compact and easy to handle. In this way, redundant calculations can be reduced while retaining necessary decision information. Then, through the conversion of Cartesian products, all possible decision combinations are effectively considered, which ensures that the system can find the optimal allocation decision under each possible service resource and client configuration, avoiding the omission of decision paths. Finally, all processing steps converge into Pareto optimal decisions, which not only ensures the balance of multi-objective optimization, but also finds the best compromise between multiple conflicting objectives.The Pareto optimal solution provides an unparalleled set of optimal solutions, which enables the system to maintain excellent performance in the face of dynamic changes and uncertain environments. The combined effect of these designs enables the system to operate effectively in complex multi-objective and multi-constraint problems, improves the collaborative optimization capabilities of wireless communication and computing resources, and has strong scalability and flexibility. Fourthly, when establishing the service agent corresponding to the service provider, the introduction of adversarial factors in the multi-objective reward function can effectively simulate the competitive relationship between the service providers. Because the service providers are non-cooperative, by introducing adversarial factors, the service agent not only makes decisions based on its own resources and status, but also predicts and responds to the strategies of other service providers, thereby avoiding inefficient "prisoner's dilemma" behaviors and promoting more efficient operation of the entire system. In addition, by incorporating the quantization level into the optimization, since there is often a trade-off between communication overhead and model accuracy in machine learning tasks. By dynamically adjusting the quantization level, the service agent can reduce the quantization level to reduce the communication burden when the bandwidth is limited, or improve the quantization accuracy to accelerate model training when the bandwidth is sufficient, thereby achieving a balance between wireless communication overhead and model convergence speed. This dynamic optimization strategy helps to flexibly adjust the use of resources according to the current communication conditions and task requirements, and effectively improves the system performance. In addition, the introduction of resource usage of other service providers also adds a competitive dimension to the decision-making process. In a non-cooperative multi-service provider environment, understanding the resource usage of other service providers can help each service agent more accurately evaluate the current resource availability and conflicts, and avoid excessive competition or waste of resources. This design not only optimizes the efficiency of resource allocation, but also improves the stability and fairness of the system. Under the design of the multi-objective reward function, the system can simultaneously consider multiple optimization objectives, such as communication evaluation, task completion evaluation, and energy consumption evaluation. This multi-objective optimization can handle various constraints in complex environments, allowing service agents to trade off between multiple objectives, thereby improving the overall resource utilization efficiency and system performance. Finally, a Markov decision process is used to establish a decision framework for each service agent, allowing the service agent to make the optimal strategy selection based on the observed state, predicted future results, and accumulated rewards, which enhances the adaptability and robustness of the system, allowing each service agent to continuously optimize resource allocation in a changing network environment. Fifth, the service provider sends the initialized global model to the assigned client according to the optimal strategy. The assigned client performs local training based on the assigned service resources and returns the trained model to the service provider. After receiving the trained model, the service provider aggregates it and finally generates a global optimization model. This distributed training method reduces the computing burden of the central server, makes computing and storage tasks more decentralized, reduces the risk of single point failure, and improves the robustness of the system.At the same time, local training can better utilize the client's computing power and local data, avoiding the data transmission bottlenecks and delays that may occur in centralized training methods. The final global model is more in line with business needs and has a faster training speed.

[0128] Corresponding to the aforementioned embodiment of a method for collaborative optimization of wireless communication and computing resources, the present application also provides an embodiment of a system for collaborative optimization of wireless communication and computing resources.

[0129] Figure 2 This is a schematic diagram of the structure of the wireless communication and computing resource collaborative optimization system provided in Example 2 of this application. Figure 2 , the wireless communication and computing resource collaborative optimization system provided in this embodiment includes a service provider set and an allocation client; wherein the service provider set includes a plurality of service providers, and there is no cooperative relationship between each service provider;

[0130] The service provider set is used to establish an initialization global model matching the business requirements for each service provider, and match the service resources of each service provider according to the business requirements;

[0131] The service provider set is further used to send the initialized global model to the allocation client according to the optimal strategy; wherein the optimal strategy is the allocation method of the service resources and the allocation client;

[0132] The service agent corresponding to each service provider generates the optimal strategy based on the corresponding service resources and status information of other service providers; each service agent predicts the virtual strategy of other service providers and generates the optimal strategy of the current service agent based on the virtual strategy;

[0133] The allocation client is used to train the received model to be trained based on the allocated service resources, and send the trained model to the corresponding service provider;

[0134] Each of the service providers is further configured to receive a trained model of the allocated client corresponding to the optimal strategy, and aggregate the trained models to obtain a global model of the corresponding business.

[0135] The system of this embodiment can be used to perform Figure 1 The steps, specific implementation principles and implementation processes of the method embodiment shown are similar and will not be repeated here.

[0136] The implementation process of the functions and effects of each unit in the above system is specifically described in the implementation process of the corresponding steps in the above method, which will not be repeated here.

[0137] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The system embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application. A person of ordinary skill in the art can understand and implement it without creative work.

[0138] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for collaborative optimization of wireless communication and computing resources, characterized in that: The method comprises: Establishing an initialization global model matching the business requirements for each service provider in the service provider set, and matching service resources of each service provider according to the business requirements; the service provider set includes multiple service providers, and there is no cooperative relationship between each service provider; Establish a service agent corresponding to each service provider; The service agent corresponding to each service provider generates an optimal strategy based on the corresponding service resources and status information of other service providers, wherein the optimal strategy is the allocation method of the service resources and the allocation client; Wherein, each service agent predicts the virtual strategies of other service providers and generates the optimal strategy of the current service agent based on the virtual strategies; sending the initialized global model to the allocation client according to the optimal strategy; Each of the allocated clients trains the received models to be trained based on the allocated service resources, and sends the trained models to the corresponding service providers; Each service provider receives the trained model of the allocated client corresponding to the optimal strategy, and each of the service agents is aggregated to obtain a global model of the corresponding business; The service agent corresponding to each service provider generates an optimal strategy based on the corresponding service resources and status information of other service providers, including: Each service agent obtains the current operation status of the corresponding service and the observed global information; Traverse all strategy combinations of other service agents as a virtual joint strategy set; Each service agent constructs an optimal virtual joint strategy based on the virtual joint strategy set and the real-time decision of the current service agent; Establishing a cumulative reward of the service agent under the optimal virtual joint strategy; Solve the Pareto optimal decision that optimizes the cumulative reward as the joint optimal strategy of all service agents; The step of generating an optimal strategy for the current service agent based on the virtual strategy comprises: Based on the action value function, the expected cumulative reward of each virtual strategy and real-time action combination in the current calculation cycle is calculated; Updating the expected cumulative reward according to the iterative calculation period difference; Select the real-time state and real-time action corresponding to the maximum expected cumulative reward to obtain the optimal strategy corresponding to the current service agent.

2. The method according to claim 1, characterized in that The solving of the Pareto optimal decision that optimizes the cumulative reward includes: Obtaining an optimal action variable based on the optimal cumulative reward, wherein the optimal action variable includes an allocation value related to a service resource allocation method and a corresponding client, and the optimal action variable is a continuous quantity; Discretizing each element in the optimal action variable; Determine a ternary variable corresponding to each discretized element, wherein the representation dimension of the ternary variable is smaller than the dimension corresponding to the discretized element; The ternary variables are converted into Cartesian products to obtain a continuous representation of the optimal action variable, thereby obtaining a Pareto optimal decision.

3. The method according to claim 1, characterized in that The step of establishing a service agent corresponding to each service provider includes: Generate the state space of the current service agent based on its own business state information and global observation information; Determine the candidate client set and service resource allocation method corresponding to the current service agent as the action space; Establish a multi-objective reward function based on communication evaluation, task completion evaluation and task completion energy consumption evaluation; The state space is used as input, the multi-objective reward function is used as a target, and a decision framework of the current service agent is established using a Markov decision process.

4. The method according to claim 1, characterized in that The service agent corresponding to each service provider generates an optimal strategy based on the corresponding service resources and status information of other service providers, including: Screening target clients from candidate clients; Determine the model quantization level based on the communication bandwidth limitation and task accuracy requirements corresponding to the service agent; Determining a resource allocation quota for each of the target clients according to the task requirements; the resource allocation quota includes a communication resource allocation quota and a computing resource allocation quota; A corresponding relationship between the resource allocation quota and the model quantization level and each client in the target client is established to generate an optimal strategy.

5. The method according to claim 1, characterized in that Each of the allocated clients trains the received models to be trained based on the allocated service resources, and sends the trained models to the corresponding service providers, including: Splitting the received model to be trained based on the optimal strategy to obtain multiple training subtasks; each training subtask corresponds to an independent training module; Allocate initial resources to the training modules based on the resource requirements of each training module and the optimal strategy; Each training module trains the corresponding training subtask based on the local data set and the initial resources. During the training process, the working frequency and resource allocation are adjusted in real time based on the current resource status until all training modules complete the training and each trained subtask is sent to the corresponding service provider.

6. The method according to claim 1, characterized in that Before sending the trained model to the corresponding service provider, the method further includes: Each service provider determines the quantization order of the corresponding assigned client based on the status information and task requirements; Each of the allocated clients performs quantization processing on the trained model based on the quantization order.

7. The method according to claim 5, characterized in that The real-time adjustment of the working frequency and resource allocation based on the current resource status during the training process includes: Evaluate resource requirements for each training module based on current resource status; Determining resource usage of each training module based on the resource demand; When the resource demand of the target training module exceeds the resource upper limit, the allocation client adjusts the working frequency of the target training module and reduces the resource allocation of other training modules in turn according to the resource usage.

8. A wireless communication and computing resource collaborative optimization system, characterized in that: The wireless communication and computing resource collaborative optimization system includes a service provider set and an allocation client; wherein the service provider set includes a plurality of service providers, and there is no cooperative relationship between each service provider; The service provider set is used to establish an initialization global model matching the business requirements for each service provider, and match the service resources of each service provider according to the business requirements; The service provider set is further used to send the initialized global model to the allocation client according to the optimal strategy; wherein the optimal strategy is the allocation method of the service resources and the allocation client; The service agent corresponding to each service provider generates the optimal strategy based on the corresponding service resources and status information of other service providers; each service agent predicts the virtual strategy of other service providers and generates the optimal strategy of the current service agent based on the virtual strategy; The allocation client is used to train the received model to be trained based on the allocated service resources, and send the trained model to the corresponding service provider; Each service provider is further configured to receive a trained model of the allocated client corresponding to the optimal strategy, and aggregate the trained models to obtain a global model of the corresponding business; The service agent corresponding to each service provider generates an optimal strategy based on the corresponding service resources and status information of other service providers, including: Each service agent obtains the current operation status of the corresponding service and the observed global information; Traverse all strategy combinations of other service agents as a virtual joint strategy set; Each service agent constructs an optimal virtual joint strategy based on the virtual joint strategy set and the real-time decision of the current service agent; Establishing a cumulative reward of the service agent under the optimal virtual joint strategy; Solve the Pareto optimal decision that optimizes the cumulative reward as the joint optimal strategy of all service agents; The step of generating an optimal strategy for the current service agent based on the virtual strategy comprises: Based on the action value function, the expected cumulative reward of each virtual strategy and real-time action combination in the current calculation cycle is calculated; Updating the expected cumulative reward according to the iterative calculation period difference; Select the real-time state and real-time action corresponding to the maximum expected cumulative reward to obtain the optimal strategy corresponding to the current service agent.

Images