General calculation fusion hybrid resource scheduling method

Abstract

The invention discloses a general calculation fusion hybrid resource scheduling method, which avoids the process that an AP (Access Point) uploads a task to a CPU (Central Processing Unit) under the condition of excessively occupying computing resources through a mode of centrally scheduling communication computing hybrid resources, thereby preventing node resources from being completely exhausted due to the occupation of the communication resources, and improving the resource utilization rate. Scheduling nodes with relatively rich communication resources to complete an uploading task; the problems that resource competition is aggravated and service time delay is increased due to parallel unloading under the high task request rate are solved. According to the real-time load change trend, a parallel unloading mechanism is adjusted in time, and when the task load rises, the maximum parallel unloading number is reduced in time; a reinforcement learning mode is adopted, load change conditions of nodes and links are obtained through real-time interaction with the environment, and a current resource scheduling strategy is dynamically adjusted. According to the invention, under the UC cell-free framework, communication and computing resources are utilized to the maximum extent, the service quality requirements of different service types are met, the load balance is ensured, and the overall efficiency of the network is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of communication systems, and in particular to a hybrid resource scheduling method with general computing integration. Background Art

[0002] In the 6G era, the number of smart devices in communication application scenarios such as smart factories, smart homes, and smart cities has increased dramatically. Computation-intensive and delay-sensitive services such as augmented / virtual reality, digital twins, and autonomous driving have prompted the deep integration of communication and computing functions. However, the rapid increase in the number of devices and types of services has posed a challenge to the rational allocation of network resources. Single-node resource scheduling solutions are unable to meet the diverse business needs due to limited resources, and face a series of problems such as limited scope of application and low network resource utilization.

[0003] Due to the development trend of highly dense networks, the user-centric approach can make full use of the proximity between terminals and base stations to provide users with better services. At the same time, there are a large number of direct-view paths in some scenarios, which enables terminals to achieve a high degree of macro-diversity gain. User-centric cell-free technology has become a key technology for future network development. Based on a user-centric cell-free network, by utilizing the resources of multiple nodes in the network for collaborative management, reasonably building a system model and formulating corresponding collaborative resource allocation strategies, the effectiveness of network resource utilization can be improved and service execution time and energy consumption can be reduced.

[0004] At present, the hybrid scheduling problem of communication computing resources in the user-centric non-cell environment mainly considers the solution of offloading computing services to a single node. Most literatures consider a solution such as Figure 1 The simple computing model shown in the figure selects one of the local or central processing units (CPU) to offload the computing business, that is, the business needs are met by the resources of a single node. For example, the computing task can be calculated locally by the terminal, or it can be offloaded to the edge service node through wireless transmission, or the task can be transmitted to the CPU for processing through a multi-hop wireless link, or the available node with the smallest instantaneous load can be selected for task offloading. Therefore, this solution has a low complexity. In addition, the current research mainly focuses on the optimization of parameters such as the allocation of computing resources and the ratio of task offloading. The single optimization goals of minimizing latency and energy consumption are achieved by jointly optimizing the offloading ratio between local computing and edge cloud computing and the uplink transmission power.

[0005] Some literature considers unified management of multiple nodes and collaborative processing of node services. Since actual tasks are usually divisible, by splitting a single task into multiple subtasks, they can be unloaded to multiple service nodes for parallel processing. For example, multiple service nodes can implement system resource allocation through auctions, which can effectively reflect the supply and demand of computing resources and maximize the utility of the entire system. The iterative bilateral auction algorithm uses convex optimization to solve the optimal allocation problem of computing brokers through multiple iterations, making the auction process computationally expensive and time-consuming. In addition, some studies are based on the assumption that each node has sufficient communication resources, and tasks are unloaded between nodes to ensure network load balancing.

[0006] The above-mentioned resource scheduling scheme for user-centric cell-free networks mostly considers using the computing resources of a single node to meet business needs. However, in a user-centric cell-free network architecture, users often access multiple edge nodes with computing resources at the same time. At this time, the resource scheduling scheme will find it difficult to fully utilize the idle resources in the network. At the same time, there is a problem that the queuing delay is large when offloading tasks to nodes with strong service capabilities, and the nodes that offload them to other nodes do not have the corresponding memory and processing capabilities.

[0007] Multi-node resource scheduling can alleviate the problem of load imbalance to a certain extent, but the above technology is not directly applicable to user-centric cell-free networks. In the user-centric cell-free framework, due to the high dispersion of service nodes and the limited resources and load capacity of a single node, resource scheduling schemes will face new challenges: on the one hand, due to cost constraints, it is difficult to connect all the deployed access points (APs) to computing resources. Even if computing servers are deployed on some APs, it is difficult to achieve a large amount of data interaction between APs. It is difficult to use distributed algorithms such as the above auction algorithm to split the business and distribute it to each node for parallel unloading. On the other hand, due to the limited resources of edge node APs, when the computing resources of AP nodes are saturated, their remaining communication resources may also be difficult to undertake the forwarding task of relay computing services.

[0008] Therefore, due to the limited resources of a single node in a user-centric cell-free network, and the large latency and memory of accessing multiple edge nodes with computing resources at the same time, there is an urgent need for a hybrid scheduling scheme for general computing resources that comprehensively considers the load conditions and transmission costs of each node in the system, so that the collaborative work between nodes can not only meet user needs, but also maintain network load balance and fully utilize network resources. Summary of the invention

[0009] In order to solve the above problems in the prior art, the present invention proposes a hybrid resource scheduling method with integrated computing, which aims to coordinate the integrated computing resources of each node to meet the service quality requirements of different business types while ensuring load balancing and improving the overall efficiency of the network.

[0010] In order to achieve the above object, the present invention provides the following technical solutions:

[0011] A hybrid resource scheduling method with general computing integration includes the following steps:

[0012] S1. After generating computing services or communication services, the terminal sends a request to the main AP node; the request contains the estimated required communication and computing resources, priority information, maximum transmission power, and maximum upload delay; each terminal sorts the AP nodes in descending order according to the gain provided by each AP node, and selects the AP node with the largest gain as the main AP node and maintains the connection, and the others are secondary AP nodes;

[0013] S2, each AP node sends its own resource load information to the CPU in real time;

[0014] S3, each AP node forwards the received service request information to the CPU;

[0015] S4: The CPU determines a resource scheduling scheme based on the service request information of each AP node and the load information of the link through a hybrid resource scheduling algorithm and feeds it back to each AP node;

[0016] S5. The primary AP node notifies the terminal to uninstall the secondary AP node and the task offloading ratio;

[0017] S6. The terminal splits the task according to the resource scheduling plan and unloads it to each AP node in parallel;

[0018] S7, each AP node updates load information and forwards the task to the CPU;

[0019] S8. The CPU returns the business processing result offloaded to the CPU according to the demand;

[0020] S9, each AP node returns the service processing result unloaded to itself according to demand;

[0021] S10. Each AP node sends updated load information to the CPU because the occupied resources are released.

[0022] Furthermore, in step S4, the process of determining the resource scheduling solution by using the hybrid resource scheduling algorithm is as follows:

[0023] S401, the CPU calculates the signal-to-noise ratio of received data;

[0024] S402, the CPU calculates the overall energy consumption and delay consumption of the network;

[0025] S403, the CPU calculates the channel gain, and the load information of each network node and link;

[0026] S404. The CPU forms a decision problem model based on the processed data. The objective function is expressed as a comprehensive evaluation index of network energy consumption and delay, and the optimal decision is obtained by solving the problem.

[0027] Furthermore, in step S401, the process in which the CPU calculates the signal-to-noise ratio of received data includes:

[0028] Using g m,n To represent the channel coefficient between terminal n and the mth AP node, β m,n and h m,n They represent the large-scale fading coefficient and the small-scale fading coefficient between the n-th terminal and the m-th AP node, respectively. All elements of the small-scale fading coefficient are independent and identically distributed complex normal random variables. The channel coefficient is expressed as:

[0029] g m,n =β m,n h m,n

[0030] Let Φ n represents the pilot sequence of terminal n, and the pilot vector received at the mth AP node is expressed as:

[0031]

[0032] in, represents the pilot transmission power, Represents τ p dimensional additive noise vector, which satisfies σ is the noise power coefficient;

[0033] Based on the received pilot vector, the channel estimate obtained using least squares is:

[0034]

[0035] is the transposed conjugate of the pilot sequence of terminal n;

[0036] Using x n Indicates the data sent by terminal n, p n Indicates the data transmission power, p n Satisfy p n =η n p n,max , p n,max represents the maximum transmit power of terminal n, ηn represents the power transmission constraint factor of terminal n, then the data received at the mth AP node is:

[0037]

[0038] γ m is the additive noise vector received at the mth AP node;

[0039] The data merged at the CPU is:

[0040]

[0041] is the adjoint matrix of the channel estimation matrix, M n To represent a set, it means that terminal n is served by the APs in this set;

[0042] The expression of the uplink signal-to-noise ratio of terminal n is:

[0043]

[0044] p n' is the power between the n'th user and each AP node, g m,n' is the channel coefficient between the n'th user and the mth AP node.

[0045] Furthermore, in step S402, the process in which the CPU calculates the overall network energy consumption and delay consumption includes:

[0046] The transmission rate between the terminal and the CPU is expressed as:

[0047]

[0048] τ c For the duration of each coherence interval, B CPU，n The bandwidth allocated to the nth user for transmitting data with the CPU;

[0049] The signal-to-noise ratio between terminal n and the mth AP node is expressed as:

[0050]

[0051] The transmission rate between terminal n and the mth AP node is expressed as:

[0052]

[0053] B m,n is the transmission bandwidth between the nth user and the mth node. The bandwidth that can be allocated at the mth AP cannot exceed the total bandwidth resource B it has. m ;;

[0054] Using ν n Indicates the number of processor clocks required to process 1 bit of data, and specifies the computing server clock speed as f m,n , C n is the size of the data transmitted, α m,n It represents the parallel offloading coefficient of terminal n assigned to computing server m. BBU is equivalent to CPU. All BBUs in the text have been replaced by CPUs. Then the computing delay is:

[0055]

[0056] The energy consumption required by the computing server to process tasks is expressed as:

[0057]

[0058] Among them, the κ parameter depends on the chip architecture design of each terminal;

[0059] The total energy consumed by the calculation is expressed as:

[0060]

[0061] Among them, the ω parameter is used to describe the energy consumed to execute one clock cycle;

[0062] Using R link Describing the maximum transmission rate of the fronthaul link, the transmission delay of each link is expressed as:

[0063]

[0064] The total delay is equal to the maximum delay sum of the transmission and calculation delays of a single link, that is:

[0065]

[0066] The transmission energy consumption is expressed as:

[0067]

[0068] The total energy consumption is expressed as:

[0069]

[0070] Furthermore, in step S403, the process in which the CPU calculates the channel gain and the load information of each network node and link includes:

[0071] Link gain score between the nth user and the mth AP node Measured by the function of the corresponding channel coefficient:

[0072]

[0073] Define an L×L binary diagonal matrix to describe the association relationship between terminal n and the mth AP:

[0074]

[0075] Use L m represents the forward link bandwidth of the mth AP, and the link congestion degree is expressed as:

[0076]

[0077] Using F m represents the total computing resources at computing server m, and the computing resource utilization is expressed as:

[0078]

[0079] Furthermore, in step S404, the objective function is expressed as:

[0080]

[0081]

[0082] Among them, ε is the weight factor of energy consumption and time, N1 and N2 represent the terminal set sending computing services and the terminal set sending communication services respectively; C1 constrains the range of task offloading ratio; C2 constraint reflects the delay requirement of each computing service; C3 constraint reflects the transmission rate requirement of each communication service; C4 constraint reflects that the channel gain obtained by the terminal connection must be greater than the set minimum channel gain score value C5 constrains the frequency resource allocation range; C6 constraints reflect the occupancy of communication and computing resources of each node; C7 constraints limit the size of data generated by the terminal; C8 constraints adjust the maximum number of connections for parallel unloading in time according to the network load; C9 constraints limit the maximum number of connections for each AP node, making the network scalable; C10 constraints are weight factor range constraints.

[0083] Furthermore, in step S404, a reinforcement learning method is used to solve and obtain the optimal decision, and the process includes:

[0084] Use (S, A, P, R, G) to describe the discrete-time Markov decision process, where S represents the state space, A represents the action space, P(a|s) is the probability of taking action a in state s, R(s, a, s') represents the immediate reward for taking action a to change the state from s to s', and G represents the long-term reward;

[0085] Each element in the state space consists of: m, τ m , Task n ,Task n Indicates the task request information sent by each terminal to the CPU at the current moment, including the task data size C n , maximum transmission power p n And the maximum task delay

[0086] Each element in the action space includes: {D m,n},{B m,n},{α m,n},{η n};

[0087] The reward is set as: -[εT n +(1-ε)E n ];

[0088] The update formula of the Actor network model parameter θ is:

[0089]

[0090] Among them, α is the iteration step variable, is the gradient of the derivative with respect to θ, π θ is the policy function containing the parameter θ, the score function Optionally use a Gaussian score function, v t is the state value at each time position t;

[0091] The update formula of critic network model parameter ω is:

[0092] ω=ω+βδφ(s,a)

[0093] Among them, β is the iteration step variable, φ is the characteristic describing the state s and action a, δ is the temporal difference error, and the expression is: δ = R t+1 +uQ(S t+1 ,A t+1 )-Q(S t ,A t ), R t+1 represents the immediate reward at time t+1, u is the decay factor variable, and Q is the linear Q function.

[0094] Further, in step S4, the CPU sets the task processing strategy parameter {D m,n},{B m,n},{α m,n},{η n}Sent to the requesting terminal and each AP node providing service.

[0095] Compared with the prior art, the present invention has the following beneficial effects:

[0096] The present invention takes into account the load balancing problem in the UC cell-free network architecture where service nodes are highly dispersed and the capacity of a single computing node is limited. With the goal of minimizing the comprehensive overhead of latency and energy consumption, a hybrid resource scheduling method for flexibly scheduling multi-AP communication and computing resources is provided, which avoids the process of AP uploading tasks to the CPU when the computing resources are too high, thereby preventing the occupation of communication resources from causing the node resources to be completely exhausted, and scheduling nodes with relatively abundant communication resources to complete the upload tasks. At the same time, the method of the present invention alleviates the problem of parallel unloading exacerbating resource competition and increasing service latency under high task request rates. According to the real-time load change trend, the parallel unloading mechanism is adjusted in time, and when the task volume increases, the maximum number of parallel unloading is adjusted down in time. In addition, the present invention adopts a reinforcement learning method to obtain the load changes of nodes and links through real-time interaction with the environment, and dynamically adjusts the current resource scheduling strategy.

[0097] In summary, the hybrid resource scheduling method of general computing fusion provided by the present invention takes into account the communication and computing resource calling conditions of each AP node, and by centrally scheduling communication and computing hybrid resources, it can meet the service quality requirements of different business types while ensuring load balancing, and can achieve maximum utilization of communication and computing resources under the UC cell-free framework to improve the overall network efficiency. Compared with the non-collaborative solution, it has better energy consumption and latency performance, and fully considers the potential problems of parallel offloading under high task request rates, and adjusts the parallel offloading solution to be closer to the non-collaborative solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0099] Figure 1 It is a single-node computing resource scheduling mechanism.

[0100] Figure 2 The invention provides a UC Cell-free multi-user communication computing resource hybrid scheduling model.

[0101] Figure 3 A flow chart of the hybrid resource scheduling method with integrated computing provided by the present invention. DETAILED DESCRIPTION

[0102] In order to better understand the technical solution, the method of the present invention is described in detail below with reference to the accompanying drawings.

[0103] The present invention considers Figure 2 In the user-centric cell-free multi-user wireless offloading scenario shown, the system consists of a single-antenna terminal, a multi-antenna AP (server) and a CPU. There are two types of terminals: one is a communication demand terminal that generates communication services, and the other is a computing demand terminal that generates computing services with low latency requirements; APs can be divided into two types according to whether they have computing service capabilities: APs that only support communication services and APs that support communication and computing services. The AP and the terminal are set to a Rayleigh fading channel, and the AP is connected to the CPU through a fronthaul link; it is assumed that the baseband processing units in the CPU can jointly process baseband signals, the interaction time is negligible and there are sufficient computing resources.

[0104] Each computing processing node divides the computing resources into several equal parts. The accessed services are processed on one resource on a first-come, first-served basis. No waiting area is set, that is, there is no queuing delay. The hybrid resource scheduling method for general computing fusion provided by the present invention has a process as follows: Figure 3 As shown, the specific steps are as follows.

[0105] S1. After generating computing services or communication services, the terminal sends a request to the main AP node; the request contains the estimated required communication and computing resources, priority information, maximum transmission power, and maximum upload delay; each terminal sorts the AP nodes in descending order according to the gain provided by each AP node, and selects the AP node with the largest gain as the main AP node and maintains the connection, and the others are secondary AP nodes;

[0106] S2, each AP node sends its own resource load information to the CPU in real time;

[0107] S3, each AP node forwards the received service request information to the CPU;

[0108] S4: The CPU determines a resource scheduling scheme based on the service request information of each AP node and the load information of the link through a hybrid resource scheduling algorithm and feeds it back to each AP node;

[0109] Communication business: mainly consider its impact on the limited fronthaul links of each node and multi-user interference.

[0110] Computing services: After retaining a portion of computing tasks locally, the remaining tasks are offloaded in parallel to AP nodes and CPUs with computing capabilities using OMA frequency division multiplexing. In this process, considering the distance between the CPU and the terminal and the communication load of the computing node, the AP nodes near the terminal that only support communication are used to offload to the CPU in a joint transmission manner, thereby increasing the transmission rate to resist the delay caused by the longer transmission distance;

[0111] S5. The primary AP node notifies the terminal to uninstall the secondary AP node and the task offloading ratio;

[0112] S6. The terminal splits the task according to the resource scheduling plan and unloads it to each AP node in parallel;

[0113] S7, each AP node updates load information and forwards the task to the CPU;

[0114] S8. The CPU returns the business processing result offloaded to the CPU according to the demand;

[0115] S9, each AP node returns the service processing result unloaded to itself according to demand;

[0116] S10. Each AP node sends updated load information to the CPU because the occupied resources are released.

[0117] In step S4, the hybrid resource scheduling algorithm executed in the CPU is designed as follows: the CPU collects the maximum transmission power p uploaded by terminal n n , task size, maximum acceptable delay, etc., and combined with the real-time load information of each AP node as the input known items of the hybrid resource scheduling algorithm. The CPU calls the hybrid resource scheduling algorithm to complete the decision process. The CPU sets the task processing strategy parameter {D m,n},{B m,n},{α m,n},{η n}Sent to the requesting terminal and each AP node providing service. All terminals implement unloading according to the above optimal unloading scheme, and all service nodes complete the calculation and return the results after receiving the task data; each AP node updates the load information to the CPU after releasing the corresponding occupied resources.

[0118] Specifically, in the above step S4, the process of determining the resource scheduling scheme by using the hybrid resource scheduling algorithm is as follows:

[0119] S401, the CPU calculates the signal-to-noise ratio of received data;

[0120] S402, the CPU calculates the overall energy consumption and delay consumption of the network;

[0121] S403, the CPU calculates the channel gain, and the load information of each network node and link;

[0122] S404. The CPU forms a decision problem model based on the processed data. The objective function is expressed as a comprehensive evaluation index of network energy consumption and delay, and the optimal decision is obtained by solving the problem.

[0123] Specifically, in step S401, the process in which the CPU calculates the signal-to-noise ratio of received data includes:

[0124] Using g m,n To represent the channel coefficient between terminal n and the mth AP node, β m,n and h m,n They represent the large-scale fading coefficient and the small-scale fading coefficient between the n-th terminal and the m-th AP node, respectively. All elements of the small-scale fading coefficient are independent and identically distributed complex normal random variables. The channel coefficient is expressed as:

[0125] g m,n =β m,n h m,n

[0126] Let Φ n represents the pilot sequence of terminal n, and the pilot vector received at the mth AP node is expressed as:

[0127]

[0128] in, represents the pilot transmission power, Represents τ p dimensional additive noise vector, which satisfies

[0129] Based on the received pilot vector, the channel estimate obtained using least squares is:

[0130]

[0131] is the transposed conjugate of the pilot sequence of terminal n;

[0132] Using x n Indicates the data sent by terminal n, p n Indicates the data transmission power, p n Satisfy p n =η n p n,max , p n,max represents the maximum transmit power of terminal n, η n represents the power transmission constraint factor of terminal n, then the data received at the mth AP node is:

[0133]

[0134] γ m is the additive noise vector received at the mth AP node;

[0135] The data merged at the CPU is:

[0136]

[0137] is the adjoint matrix of the channel estimation matrix, M n To represent a set, it means that terminal n is served by the APs in this set;

[0138] The expression of the uplink signal-to-noise ratio (SINR) of terminal n is:

[0139]

[0140] p n' is the power between the n'th user and each AP node, g m,n' is the channel coefficient between the n'th user and the mth AP node, σ is the noise power coefficient, which is equal to the variable mentioned above Satisfied σ in .

[0141] In step S402, the process of the CPU calculating the overall network energy consumption and delay consumption includes:

[0142] According to the above SINR expression, the transmission rate between the terminal and the CPU is expressed as:

[0143]

[0144] τ c For the duration of each coherence interval, B CPU，n The bandwidth allocated to the nth user for transmitting data with the CPU;

[0145] Similarly, the signal-to-noise ratio between terminal n and the mth AP node is expressed as:

[0146]

[0147] The transmission rate between terminal n and the mth AP node is expressed as:

[0148]

[0149] B m,n is the transmission bandwidth between the nth user and the mth node. The bandwidth that can be allocated at the mth AP cannot exceed the total bandwidth resource B it has. m ;;

[0150] Tasks are independent and fine-grained, i.e., they can be decomposed into arbitrary parts, which can participate in parallel offloading between edge servers and CPUs, using ν n Indicates the number of processor clocks required to process 1 bit of data, and specifies the computing server clock speed as f m,n , C n is the size of the data transmitted, α m,nrepresents the parallel offloading coefficient assigned to computing server m by terminal n, and its computing delay is:

[0151]

[0152] The energy consumption required by the computing server to process tasks is expressed as:

[0153]

[0154] Among them, the κ parameter depends on the chip architecture design of each terminal;

[0155] The total energy consumed by the calculation is expressed as:

[0156]

[0157] Among them, the ω parameter is used to describe the energy consumed to execute one clock cycle;

[0158] Using R link Describing the maximum transmission rate of the fronthaul link, the transmission delay of each link is expressed as:

[0159]

[0160] Since parallel offloading is adopted, the total delay is equal to the maximum delay sum of the transmission delay and the calculation delay of a single link, that is:

[0161]

[0162] The transmission energy consumption is expressed as:

[0163]

[0164] The total energy consumption is expressed as:

[0165]

[0166] In step S403, the process of the CPU calculating the channel gain and the load information of each network node and link includes:

[0167] In the present invention, each terminal sorts the APs in descending order according to the gain provided by each AP, selects the AP with the largest gain as the main AP and maintains the connection. The other APs in the AP cluster serving the terminal are determined by the resource scheduling algorithm. The link gain metric depends on the large-scale fading coefficient:

[0168]

[0169] Using M nTo represent a set, terminal n is served by the APs in this set, and an L×L binary diagonal matrix is ​​defined to describe the association between terminal n and the mth AP:

[0170]

[0171] Use L m represents the forward link bandwidth of the mth AP, and the link congestion degree is expressed as:

[0172]

[0173] Similarly, using F m represents the total computing resources at computing server m, and the computing resource utilization is expressed as:

[0174]

[0175] In step S404, the network energy consumption, time consumption, network load balancing, etc. are comprehensively considered to form a decision problem. The objective function is expressed as a comprehensive evaluation index of network energy consumption and delay, where ε is the weight factor of energy consumption and time. In practice, it can be flexibly adjusted according to specific network requirements. The objective function is expressed as:

[0176]

[0177]

[0178] Among them, ε is the weight factor of energy consumption and time, N1 and N2 represent the terminal set sending computing services and the terminal set sending communication services respectively; C1 constrains the range of task offloading ratio; C2 constraint reflects the delay requirement of each computing service; C3 constraint reflects the transmission rate requirement of each communication service; C4 constraint reflects that the channel gain obtained by the terminal connection must be greater than the set minimum channel gain score value C5 constrains the frequency resource allocation range; C6 constraints reflect the occupancy of communication and computing resources of each node; C7 constraints limit the size of data generated by the terminal; C8 constraints adjust the maximum number of connections for parallel unloading in time according to the network load; C9 constraints limit the maximum number of connections for each AP node, making the network scalable; C10 constraints are weight factor range constraints.

[0179] In step S404, a reinforcement learning method is used to obtain the optimal decision, and the process includes:

[0180] The objective function of the above problem is non-convex, and it is difficult to directly obtain the optimal solution. Usually, a suboptimal solution can be obtained based on the optimization algorithm. The present invention adopts the reinforcement learning method to construct the action space and the state space, and obtains an optimal unloading solution by setting a reasonable reward function.

[0181] Specifically, the present invention uses (S, A, P, R, G) to describe the discrete-time Markov decision process, where S represents the state space, A represents the action space, P(a|s) is the probability of taking action a in state s, R(s, a, s') represents the immediate reward for taking action a to change the state from s to s', and G represents the long-term reward.

[0182] Each element in the state space consists of: m , τ m , Task n ,Task n Indicates the task request information sent by each terminal to the CPU at the current moment, including the task data size C n , maximum transmission power p n And the maximum task delay

[0183] Each element in the action space includes: {D m,n},{B m,n},{α m,n},{η n};

[0184] Since the objective function is to be minimized, the reward is set to: -[εT n +(1-ε)E n ].

[0185] The Actor-Critic algorithm is a hybrid of the policy-based and value-based algorithms. Actor and Critic are the two components of Actor-Critic. The policy-based part can be roughly described as:

[0186] π θ (s,a)=P(a∣s,θ)≈π(a∣s)

[0187] The value-based part can be roughly described as:

[0188]

[0189]

[0190] The Monte Carlo policy gradient reinforcement algorithm can be modified to the Actor-Critic algorithm. In the Monte Carlo policy gradient reinforcement algorithm, the update formula of the Actor network model parameter θ is:

[0191]

[0192] Among them, α is the iteration step variable, is the gradient of the derivative with respect to θ, π θ is the policy function containing the parameter θ, the score function Optionally use a Gaussian score function. t is the state value for each time position t;

[0193] Critic calculates the current state v through the Q network t The optimal value of , and the actor uses v t Iteratively update the policy function parameters θ, then select an action, and get the feedback result and the new state. The critic uses the feedback result and the new state to update the Q network parameters ω. After that, the critic will use the new network parameters ω to help the actor calculate the state v t The optimal value of .

[0194] For the critic's own model parameter ω, the mean square error loss function is usually used for iterative update. Use a linear Q function, such as Q(s,a,w)=φ(s,a) T ω, then the critic network model parameter ω update formula is:

[0195] ω=ω+βδφ(s,a)

[0196] Among them, β is the iteration step variable, φ is the characteristic describing the state s and action a, δ is the temporal difference error, and the expression is: δ = R t+1 +uQ(S t+1 ,A t+1 )-Q(S t ,A t ), R t+1 represents the immediate reward at time t+1, u is the decay factor variable, and Q is the linear Q function.

[0197] The Actor-Critic reinforcement learning algorithm process of the present invention is as follows:

[0198] Input: action set K, step size α, β, decay factor u, exploration rate r, iteration round T, state feature dimension Num, Critic network structure and Actor network structure

[0199] Output: actor's network parameters θ, critic's network parameters ω;

[0200] Initialization: Randomly initialize the Q values ​​corresponding to all states and actions.

[0201] For i=1to T, do

[0202] (a) Initialize S to the first state of the current state sequence and obtain its feature vector φ(S);

[0203] (b) In the actor network, input φ(S), output action A, and feedback R based on the new state S' obtained by A;

[0204] (c) In the critic network, input φ(S), φ(S') and obtain the Q value output V(S), V(S');

[0205] (d) Calculate the time difference error TD error: δ = R + uV(S') - V(S);

[0206] (e) Use the mean variance loss function to perform gradient update on the critic network parameter ω: ∑(R+uV(S')-V(S,W)) 2 ;

[0207] (f) Update actor network parameters θ:

[0208] In summary, the present invention provides a hybrid resource scheduling method for flexibly scheduling multi-AP communication and computing resources, avoiding the process of AP uploading tasks to the CPU when the computing resources are too high, thereby preventing the occupation of communication resources from causing complete exhaustion of node resources, and scheduling nodes with relatively abundant communication resources to complete the upload tasks; it alleviates the problem of parallel unloading exacerbating resource competition and increasing service delays under high task request rates. According to the real-time load change trend, the parallel unloading mechanism is adjusted in time, and when the task volume increases, the maximum number of parallel unloading is adjusted in time; the reinforcement learning method is adopted to obtain the load changes of nodes and links through real-time interaction with the environment, and dynamically adjust the current resource scheduling strategy. Under the UCcell-free framework, the maximum utilization of communication and computing resources is achieved to meet the service quality requirements of different business types while ensuring load balancing and improving the overall efficiency of the network.

[0209] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hybrid resource scheduling method with general computing integration, characterized in that: The following steps are involved: S1. After generating computing services or communication services, the terminal sends a request to the main AP node; The request contains the estimated required communication and computing resources, priority information, as well as the maximum transmit power and the maximum upload delay; Each terminal sorts the AP nodes in descending order according to the gain provided by each AP node, and selects the AP node with the largest gain as the main AP node and maintains the connection, and the others are secondary AP nodes; S2, each AP node sends its own resource load information to the CPU in real time; S3, each AP node forwards the received service request information to the CPU; S4: The CPU determines a resource scheduling scheme based on the service request information of each AP node and the load information of the link through a hybrid resource scheduling algorithm and feeds it back to each AP node; S5. The primary AP node notifies the terminal to uninstall the secondary AP node and the task offloading ratio; S6. The terminal splits the task according to the resource scheduling plan and unloads it to each AP node in parallel; S7, each AP node updates load information and forwards the task to the CPU; S8. The CPU returns the business processing result offloaded to the CPU according to the demand; S9, each AP node returns the service processing result unloaded to itself according to demand; S10. Each AP node sends updated load information to the CPU because the occupied resources are released.

2. The hybrid resource scheduling method of general computing fusion according to claim 1 is characterized in that: In step S4, the process of determining the resource scheduling solution by using the hybrid resource scheduling algorithm is as follows: S401, the CPU calculates the signal-to-noise ratio of received data; S402, the CPU calculates the overall energy consumption and delay consumption of the network; S403, the CPU calculates the channel gain, and the load information of each network node and link; S404. The CPU forms a decision problem model based on the processed data. The objective function is expressed as a comprehensive evaluation index of network energy consumption and delay, and the optimal decision is obtained by solving the problem.

3. The hybrid resource scheduling method of general computing fusion according to claim 2 is characterized in that: In step S401, the process of the CPU calculating the signal-to-noise ratio of received data includes: Using g m,n To represent the channel coefficient between terminal n and the mth AP node, β m,n and h m,n They represent the large-scale fading coefficient and the small-scale fading coefficient between the n-th terminal and the m-th AP node, respectively. All elements of the small-scale fading coefficient are independent and identically distributed complex normal random variables. The channel coefficient is expressed as: g m，n =b m，n h m，n Let Φ n represents the pilot sequence of terminal n, and the pilot vector received at the mth AP node is expressed as: in, represents the pilot transmission power, Represents τ p dimensional additive noise vector, which satisfies σ is the noise power coefficient; Based on the received pilot vector, the channel estimate obtained using least squares is: is the transposed conjugate of the pilot sequence of terminal n; Using x n Indicates the data sent by terminal n, p n Indicates the data transmission power, p n Satisfy p n =η n p n，max , p n，max represents the maximum transmit power of terminal n, η n represents the power transmission constraint factor of terminal n, then the data received at the mth AP node is: γ m is the additive noise vector received at the mth AP node; The data merged at the CPU is: is the adjoint matrix of the channel estimation matrix, M n To represent a set, it means that terminal n is served by the APs in this set; The expression of the uplink signal-to-noise ratio of terminal n is: p n′ is the power between the n′th user and each AP node, g m，n′ is the channel coefficient between the n′th user and the mth AP node.

4. The hybrid resource scheduling method of general computing fusion according to claim 3 is characterized in that: In step S402, the process of the CPU calculating the overall network energy consumption and delay consumption includes: The transmission rate between the terminal and the CPU is expressed as: τ c For the duration of each coherence interval, B CPU，n The bandwidth allocated to the nth user for transmitting data with the CPU; The signal-to-noise ratio between terminal n and the mth AP node is expressed as: The transmission rate between terminal n and the mth AP node is expressed as: B m，n is the transmission bandwidth between the nth user and the mth node. The bandwidth that can be allocated at the mth AP cannot exceed the total bandwidth resource B it has. m ; Using v n Indicates the number of processor clocks required to process 1 bit of data, and specifies the computing server clock speed as f m，n , C n is the size of the data transmitted, α m，n represents the parallel offloading coefficient assigned to computing server m by terminal n, and its computing delay is: The energy consumption required by the computing server to process tasks is expressed as: Among them, the κ parameter depends on the chip architecture design of each terminal; The total energy consumed by the calculation is expressed as: Among them, the ω parameter is used to describe the energy consumed to execute one clock cycle; Using R link Describing the maximum transmission rate of the fronthaul link, the transmission delay of each link is expressed as: The total delay is equal to the maximum delay sum of the transmission and calculation delays of a single link, that is: The transmission energy consumption is expressed as: The total energy consumption is expressed as:

5. The hybrid resource scheduling method of general computing fusion according to claim 4 is characterized in that: In step S403, the process of the CPU calculating the channel gain and the load information of each network node and link includes: Link gain score between the nth user and the mth AP node Measured by the function of the corresponding channel coefficient: Define an L×L binary diagonal matrix to describe the association relationship between terminal n and the mth AP: Use L m represents the forward link bandwidth of the mth AP, and the link congestion degree is expressed as: Using F m represents the total computing resources at computing server m, and the computing resource utilization is expressed as:

6. The hybrid resource scheduling method of general computing fusion according to claim 5 is characterized in that: In step S404, the objective function is expressed as: Among them, ε is the weight factor of energy consumption and time, N1 and N2 represent the terminal set sending computing services and the terminal set sending communication services respectively; C1 constrains the range of task offloading ratio; C2 constraint reflects the delay requirement of each computing service; C3 constraint reflects the transmission rate requirement of each communication service; C4 constraint reflects that the channel gain obtained by the terminal connection must be greater than the set minimum channel gain score value C5 constrains the frequency resource allocation range; C6 constraints reflect the occupancy of communication and computing resources of each node; C7 constraints limit the size of data generated by the terminal; C8 constraints adjust the maximum number of connections for parallel unloading in time according to the network load; C9 constraints limit the maximum number of connections for each AP node, making the network scalable; C10 constraints are weight factor range constraints.

7. The hybrid resource scheduling method of general computing fusion according to claim 6 is characterized in that: In step S404, a reinforcement learning method is used to obtain the optimal decision, and the process includes: Use (S, A, P, R, G) to describe the discrete-time Markov decision process, where S represents the state space, A represents the action space, P(a|s) is the probability of taking action a in state s, R(s, a, s') represents the immediate reward for taking action a to transform the state from s to s', and G represents the long-term reward; Each element in the state space consists of: m , τ m , Task n , Task n Indicates the task request information sent by each terminal to the CPU at the current moment, including the task data size C n , maximum transmission power p n And the maximum task delay Each element in the action space includes: {D m，n },{B m，n }, {α m，n },{η n }; The reward is set as: -[εT n +(1-ε)E n ]; The update formula of the Actor network model parameter θ is: Among them, α is the iteration step variable, is the gradient of the derivative with respect to θ, π θ is the policy function containing the parameter θ, the score function Optionally use a Gaussian score function, v t is the state value at each time position t; The update formula of critic network model parameter ω is: ω=ω+βδφ(s,a) Among them, β is the iteration step variable, φ is the characteristic describing the state s and action a, δ is the temporal difference error, and the expression is: δ = R t+1 +uQ(S t+1 , A t+1 )-Q(S t , A t ), R t+1 represents the immediate reward at time t+1, u is the decay factor variable, and Q is the linear Q function.

8. The hybrid resource scheduling method of general computing fusion according to claim 6 is characterized in that: In step S4, the CPU sets the task processing strategy parameter {D m，n },{B m，n }, {α m，n },{η n }Sent to the requesting terminal and each AP node providing service.

Images