A mixed resource scheduling method for sum-product fusion

Abstract

The application discloses a kind of mixed resource scheduling methods of total fusion, by the way of centralized scheduling communication computing mixed resource, avoid the process that AP will task upload to CPU in the case where computing resource occupies too high, to prevent the node resource from being completely exhausted by occupying communication resource, and scheduling communication resource relatively abundant node completes upload task;Relieve the problem of increasing business delay under high task request rate Parallel unloading aggravates resource competition;According to the real-time load change trend, timely adjust parallel unloading mechanism, when the task quantity rises, timely reduce the maximum parallel unloading number;Using reinforcement learning, by real-time interaction with the environment, obtain the load change of node and link, dynamically adjust the current resource scheduling strategy. Realize the maximum degree of utilization of communication and computing resources under the UC cell-free framework, meet the quality of service requirements of different service types, while ensuring load balancing and improving overall network efficiency.

Description

Technical Field

[0001] This invention relates to the field of communication system technology, and in particular to a hybrid resource scheduling method that integrates communication and computing. Background Technology

[0002] In the 6G era, the number of smart devices in communication application scenarios such as smart factories, smart homes, and smart cities is increasing dramatically. Computationally intensive and latency-sensitive services such as augmented / virtual reality, digital twins, and autonomous driving are driving the deep integration of communication and computing functions. However, the rapid increase in the number of devices and the types of services poses a challenge to the rational allocation of network resources. Single-node resource scheduling schemes, due to resource constraints, can no longer meet the diverse service needs and face a series of problems such as limited applicability and low network resource utilization.

[0003] Due to the trend towards highly dense networks, a user-centric approach can fully leverage the proximity between terminals and base stations to provide better services to users. Simultaneously, some scenarios exhibit numerous line-of-sight paths, enabling terminals to achieve high macro-diversity gain. Therefore, user-centric cellless technology has become a key technology for future network development. Based on user-centric cellless networks, by utilizing the resources of multiple nodes in the network for collaborative management, rationally constructing system models, and formulating corresponding collaborative resource allocation strategies, the effectiveness of network resource utilization can be improved, and service execution time and energy consumption can be reduced.

[0004] Currently, user-centric, cell-free hybrid scheduling of communication and computing resources mainly considers schemes for offloading computing services to a single node. Most literature considers a method such as... Figure 1 The simplified computing model shown selects either the local machine or the central processing unit (CPU) to offload computing tasks, thus meeting service requirements with single-node resources. For example, computing tasks can be performed locally by the terminal, offloaded to edge service nodes via wireless transmission, transmitted to the CPU for processing via multi-hop wireless links, or offloaded to the available node with the lowest instantaneous load. Therefore, this approach has low complexity. Furthermore, current research primarily focuses on optimizing parameters such as computing resource allocation and task offloading ratios. By jointly optimizing the offloading ratio between local and edge computing, as well as uplink transmission power, single optimization objectives such as minimizing latency and energy consumption can be achieved.

[0005] Some literature considers unified management of multiple nodes and collaborative processing of node services. Since practical tasks are usually divisible, a single task can be broken down into multiple subtasks, which can then be offloaded to multiple service nodes for parallel processing. For example, multiple service nodes can allocate system resources through an auction, which can effectively reflect the supply and demand of computing resources and maximize the overall system utility. However, iterative bilateral auction algorithms, through multiple iterations, use convex optimization to solve the optimal allocation problem for computational brokers, resulting in high computational cost and long processing time. Furthermore, some studies, based on the assumption that each node has sufficient communication resources, perform task offloading among nodes to ensure network load balancing.

[0006] The aforementioned resource scheduling schemes for user-centric cellless networks mostly consider using the computing resources of a single node to meet business needs. However, in a user-centric cellless network architecture, users often connect to multiple edge nodes with computing resources simultaneously. In this case, the resource scheduling scheme will find it difficult to make full use of the idle resources in the network. At the same time, there are problems such as large queuing latency when offloading tasks to nodes with strong service capabilities, and the lack of corresponding memory and processing capabilities when offloading them to other nodes.

[0007] Multi-node resource scheduling can alleviate the problem of load imbalance to some extent; however, the aforementioned techniques are not directly applicable to user-centric cellless networks. In a user-centric cellless framework, due to the highly dispersed service nodes and the limited resources and load capacity of individual nodes, resource scheduling schemes face new challenges: Firstly, due to cost constraints, it is difficult to connect all the deployed access points (APs) to computing resources. Even if some APs are used to deploy computing servers, it is difficult to achieve large-scale data interaction between APs. Using distributed algorithms, such as the auction algorithm mentioned above, to split and distribute services to various nodes for parallel offloading is difficult to implement. Secondly, due to the limited resources of edge node APs, when the computing resources of an AP node are saturated, its remaining communication resources may also be insufficient to handle the forwarding tasks of relay computing services.

[0008] Therefore, due to the limited resources of a single node in a user-centric cellless network, and the large latency and memory requirements of connecting multiple edge nodes with computing resources, there is an urgent need for a hybrid scheduling scheme for general computing resources that comprehensively considers the load status and transmission costs of each node in the system. This scheme can enable the collaborative work between nodes to meet user needs while maintaining network load balance and achieving full utilization of network resources. Summary of the Invention

[0009] In order to solve the above-mentioned problems of the prior art, the present invention proposes a hybrid resource scheduling method that integrates computing resources of each node, which aims to coordinate the computing resources of each node, meet the service quality requirements of different service types, ensure load balancing, and improve the overall network efficiency.

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] A hybrid resource scheduling method integrating general computing and data processing includes the following steps:

[0012] S1. After generating computing or communication services, the terminal sends a request to the primary AP node. The request includes the estimated required communication and computing resources, priority information, maximum transmit power, and maximum upload latency. Each terminal sorts the AP nodes in descending order according to the gain provided by each AP node, selects the AP node with the largest gain as the primary AP node and maintains the connection, and the others are secondary AP nodes.

[0013] S2. Each AP node sends its own resource load information to the CPU in real time.

[0014] S3. Each AP node will forward the received service request information to the CPU;

[0015] S4. The CPU determines a resource scheduling scheme based on the service request information of each AP node and the load information of the link, and feeds it back to each AP node through a hybrid resource scheduling algorithm.

[0016] S5. The main AP node notifies the terminal to uninstall the secondary AP node and the task uninstallation ratio.

[0017] S6. The terminal splits the task according to the resource scheduling scheme and unloads it to each AP node in parallel.

[0018] S7. Each AP node updates its load information and forwards the task to the CPU;

[0019] S8, CPU returns the business processing results that were offloaded to CPU as required;

[0020] S9. Each AP node returns the business processing results that were unloaded to itself as needed;

[0021] S10. Each AP node sends updated load information to the CPU as it releases occupied resources.

[0022] Furthermore, in step S4, the process of determining the resource scheduling scheme using the hybrid resource scheduling algorithm is as follows:

[0023] S401, The CPU calculates the signal-to-noise ratio of the received data;

[0024] S402, CPU calculates the overall energy consumption and latency consumption of the network;

[0025] S403, CPU calculates channel gain, load information of each network node and link;

[0026] S404 and CPU form a decision problem model based on the processed data. The objective function is expressed as a comprehensive evaluation index of network energy consumption and latency. The optimal decision is obtained by solving the model.

[0027] Furthermore, in step S401, the process of the CPU calculating the signal-to-noise ratio of the received data includes:

[0028] Use g m,n Let β represent the channel coefficient between terminal n and the m-th AP node. m,n and h m,n Let represent the large-scale fading coefficient and the small-scale fading coefficient between the nth terminal and the mth AP node, respectively. The small-scale fading coefficient consists of independent and identically distributed complex normal random variables. Then, the channel coefficient is expressed as:

[0029] g m,n =β m,n h m,n

[0030] Let Φ n The pilot sequence of terminal n, and the pilot vector received at the m-th AP node are represented as:

[0031]

[0032] in, Indicates the pilot transmission power. Represents τ p An additive noise vector that satisfies σ is the noise power factor;

[0033] Based on the received pilot vector, the channel estimate obtained using least squares is:

[0034]

[0035] For the transpose and conjugate of the terminal n pilot sequence;

[0036] Use x n p represents the data sent by terminal n. n p represents the power of transmitted data. n Satisfy p n =η n p n,max p n,max η represents the maximum transmit power of terminal n.n Let n represent the power transmit constraint factor of terminal n. Then the data received at the m-th AP node is:

[0037]

[0038] γ m Let be the additive noise vector received at the m-th AP node;

[0039] The data obtained by merging at the CPU is:

[0040]

[0041] M is the adjoint matrix of the channel estimation matrix. n To represent a set, terminal n is served by APs in this set;

[0042] The expression for the uplink signal-to-noise ratio of terminal n is:

[0043]

[0044] p n' Let g be the power between the n'-th user and each AP node. m,n' Let be the channel coefficient between the n'th user and the m'th AP node.

[0045] Furthermore, in step S402, the process of the CPU calculating the overall energy consumption and latency consumption of the network includes:

[0046] The transmission rate between the terminal and the CPU is expressed as:

[0047]

[0048] τ c For the duration of each coherent interval, B CPU，n This refers to the bandwidth allocated to the nth user for transmitting data with the CPU.

[0049] The signal-to-noise ratio between terminal n and the m-th AP node is expressed as:

[0050]

[0051] The transmission rate between terminal n and the m-th AP node is expressed as:

[0052]

[0053] B m,n The bandwidth allocated to the m-th AP is the transmission bandwidth between the n-th user and the m-th node. The bandwidth allocated to the m-th AP cannot exceed its total available bandwidth resource B. m ;;

[0054] Use ν n This indicates the number of processor clock cycles required to process 1 bit of data, and specifies the computing server clock speed as f. m,n C n α is the size of the data to be transmitted. m,n Let represent the parallel offloading coefficient allocated to computing server m by terminal n. Since BBU is equivalent to CPU, and all BBUs have been replaced with CPUs, the computation latency is:

[0055]

[0056] The energy consumption required by the computing server to process tasks is expressed as:

[0057]

[0058] Among them, the κ parameter depends on the architecture design of each terminal chip;

[0059] The total energy consumed is expressed as follows:

[0060]

[0061] The ω parameter describes the energy consumed in executing one clock cycle.

[0062] Using R link To describe the maximum transmission rate of the fronthaul link, the transmission delay of each link is expressed as:

[0063]

[0064] The total delay equals the sum of the maximum delay for transmission and computation on a single link, that is:

[0065]

[0066] The transmission energy consumption is then expressed as:

[0067]

[0068] Total energy consumption is expressed as:

[0069]

[0070] Furthermore, in step S403, the process by which the CPU calculates the channel gain and the load information of each network node and link includes:

[0071] Link gain score between the nth user and the mth AP node It is measured by a function of the corresponding channel coefficients:

[0072]

[0073] Define an L×L binary diagonal matrix to describe the association between terminal n and the m-th AP:

[0074]

[0075] Use L m Let represent the forward link bandwidth of the m-th AP. Then, the link congestion level is expressed as:

[0076]

[0077] Use F m Let m represent the total computing resources at computing server m. Then, the computing resource utilization rate is expressed as:

[0078]

[0079] Furthermore, in step S404, the objective function is expressed as:

[0080]

[0081]

[0082] Where ε is the weighting factor for energy consumption and time, N1 and N2 represent the set of terminals sending computing services and the set of terminals sending communication services, respectively; C1 constrains the range of task offloading ratio; C2 constraint reflects the latency requirement of each computing service; C3 constraint reflects the transmission rate requirement of each communication service; C4 constraint reflects that the channel gain obtained by the terminal connection must be greater than the set minimum channel gain score. C5 constraint limits the frequency resource allocation range; C6 constraint reflects the usage of communication and computing resources by each node; C7 constraint limits the size of data generated by the terminal; C8 constraint adjusts the maximum number of connections for parallel offloading in a timely manner according to network load; C9 constraint limits the maximum number of connections for each AP node, making the network scalable; C10 constraint is a weight factor range constraint.

[0083] Furthermore, in step S404, reinforcement learning is used to obtain the optimal decision, and the process includes:

[0084] Let (S,A,P,R,G) describe a discrete-time Markov decision process, where S represents the state space, A represents the action space, P(a|s) is the probability of taking action a in state s, R(s,a,s') represents the immediate reward of taking action a to change state from s to s', and G represents the long-term reward.

[0085] Each element in the state space includes: χ mτ m , Task n Task n This indicates the task request information sent to the CPU by each terminal at the current moment, including the task data size C. n Maximum transmit power p n and maximum task latency

[0086] Each element in the action space includes: {D m,n},{B m,n},{α m,n},{η n};

[0087] The reward is set to: -[εT n +(1-ε)E n ];

[0088] The formula for updating the Actor network model parameter θ is:

[0089]

[0090] Where α is the iteration step size variable, The gradient of the derivative with respect to θ, π θ The policy function containing the parameter θ, the score function You can choose to use the Gaussian score function, v t The state value at each time position t;

[0091] The formula for updating the parameter ω of the critic network model is:

[0092] ω=ω+βδφ(s,a)

[0093] Where β is the iteration step size variable, φ is the feature describing state s and action a, and δ is the temporal difference error, expressed as: δ = R t+1 +uQ(S t+1 A t+1 )-Q(S t A t ), R t+1 Let represent the instantaneous reward at time t+1, u be the decay factor variable, and Q be the linear Q-function.

[0094] Further, in step S4, the CPU will process the task processing strategy parameter {D} m,n},{B m,n},{α m,n},{η n} is sent to the requesting terminal and each AP node providing the service.

[0095] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0096] This invention addresses the load balancing problem in a UC cell-free network architecture where service nodes are highly distributed and the capacity of individual computing nodes is limited. Aiming to minimize the combined overhead of latency and energy consumption, it provides a hybrid resource scheduling method that flexibly schedules communication and computing resources across multiple access points (APs). This avoids the process of APs uploading tasks to the CPU when computing resources are excessively occupied, thus preventing the complete depletion of node resources due to communication resource consumption. Nodes with relatively abundant communication resources are scheduled to complete the uploading tasks. Simultaneously, this invention alleviates the problem of increased resource contention and service latency caused by parallel offloading under high task request rates. The parallel offloading mechanism is adjusted promptly based on real-time load trends; when the task volume increases, the maximum number of parallel offloads is reduced. Furthermore, this invention employs reinforcement learning to dynamically adjust the current resource scheduling strategy by interacting with the environment in real time to obtain the load changes of nodes and links.

[0097] In summary, the hybrid resource scheduling method integrating communication and computing provided by this invention takes into account the communication and computing resource usage of each AP node. By centrally scheduling hybrid communication and computing resources, it meets the service quality requirements of different service types while ensuring load balancing. This allows for maximum utilization of communication and computing resources within a UC cell-free framework, thereby improving overall network efficiency. Compared to non-cooperative solutions, it exhibits better energy consumption and latency performance and fully considers the potential problems of parallel offloading under high task request rates, adjusting the parallel offloading scheme to align with the non-cooperative solution. Attached Figure Description

[0098] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0099] Figure 1 It is a single-node computing resource scheduling mechanism.

[0100] Figure 2 This invention provides a hybrid scheduling model for UC Cell-free multi-user communication computing resources.

[0101] Figure 3 The flowchart of the hybrid resource scheduling method for general computing integration provided by the present invention is shown. Detailed Implementation

[0102] To better understand this technical solution, the method of the present invention will be described in detail below with reference to the accompanying drawings.

[0103] This invention considers as follows Figure 2 The illustrated user-centric, cell-free, multi-user wireless offloading scenario consists of a single-antenna terminal, a multi-antenna AP (server), and a CPU. Terminals are divided into two types: communication-demand terminals that generate communication services, and computing-demand terminals that generate low-latency computing services. APs are categorized based on their computing service capabilities: APs that only support communication services and APs that support both communication and computing services. A Rayleigh fading channel is configured between the AP and the terminal, and the AP is connected to the CPU via a fronthaul link. It is assumed that the baseband processing units within the CPU can collaboratively process baseband signals, with negligible interaction time and ample computing resources.

[0104] Each computing node divides its computing resources into several equal parts. Incoming services are processed on a first-come, first-served basis within each part of the resource, without a waiting area, thus eliminating queuing delays. The hybrid resource scheduling method for integrated computing provided by this invention has the following process: Figure 3 As shown, the specific steps are as follows.

[0105] S1. After generating computing or communication services, the terminal sends a request to the primary AP node. The request includes the estimated required communication and computing resources, priority information, maximum transmit power, and maximum upload latency. Each terminal sorts the AP nodes in descending order according to the gain provided by each AP node, selects the AP node with the largest gain as the primary AP node and maintains the connection, and the others are secondary AP nodes.

[0106] S2. Each AP node sends its own resource load information to the CPU in real time.

[0107] S3. Each AP node will forward the received service request information to the CPU;

[0108] S4. The CPU determines a resource scheduling scheme based on the service request information of each AP node and the load information of the link, and feeds it back to each AP node through a hybrid resource scheduling algorithm.

[0109] Communication services: The main considerations are the impact on the limited fronthaul links of each node and the interference caused by multiple users.

[0110] Computing services: After retaining a portion of the computing tasks locally, the remaining tasks are offloaded in parallel to AP nodes and CPUs with computing capabilities using OMA frequency division multiplexing. In this process, considering the distance between the CPU and the terminal and the communication load of the computing nodes, AP nodes near the terminal that only support communication are used to offload the tasks to the CPU in a joint transmission manner, thereby increasing the transmission rate to resist the latency caused by the longer transmission distance.

[0111] S5. The main AP node notifies the terminal to uninstall the secondary AP node and the task uninstallation ratio.

[0112] S6. The terminal splits the task according to the resource scheduling scheme and unloads it to each AP node in parallel.

[0113] S7. Each AP node updates its load information and forwards the task to the CPU;

[0114] S8, CPU returns the business processing results that were offloaded to CPU as required;

[0115] S9. Each AP node returns the business processing results that were unloaded to itself as needed;

[0116] S10. Each AP node sends updated load information to the CPU as it releases occupied resources.

[0117] In step S4, the hybrid resource scheduling algorithm executed in the CPU is designed as follows: The CPU collects the maximum transmission power p uploaded by terminal n. n Information such as task size and maximum acceptable latency, combined with real-time load information of each AP node, are used as known inputs to the hybrid resource scheduling algorithm. The CPU invokes the hybrid resource scheduling algorithm to complete the decision-making process. The CPU processes the task processing strategy parameters {D} m,n},{B m,n},{α m,n},{η n The data is sent to the requesting terminal and each AP node providing the service. All terminals implement the optimal unloading scheme described above, and all service nodes complete the calculations and return the results after receiving the task data. Each AP node updates its load information to the CPU after releasing the corresponding occupied resources.

[0118] Specifically, in step S4 above, the process of determining the resource scheduling scheme using the hybrid resource scheduling algorithm is as follows:

[0119] S401, The CPU calculates the signal-to-noise ratio of the received data;

[0120] S402, CPU calculates the overall energy consumption and latency consumption of the network;

[0121] S403, CPU calculates channel gain, load information of each network node and link;

[0122] S404 and CPU form a decision problem model based on the processed data. The objective function is expressed as a comprehensive evaluation index of network energy consumption and latency. The optimal decision is obtained by solving the model.

[0123] Specifically, in step S401, the process of the CPU calculating the signal-to-noise ratio of the received data includes:

[0124] Use g m,n Let β represent the channel coefficient between terminal n and the m-th AP node. m,n and h m,n Let represent the large-scale fading coefficient and the small-scale fading coefficient between the nth terminal and the mth AP node, respectively. The small-scale fading coefficient consists of independent and identically distributed complex normal random variables. Then, the channel coefficient is expressed as:

[0125] g m,n =β m,n h m,n

[0126] Let Φ n The pilot sequence of terminal n, and the pilot vector received at the m-th AP node are represented as:

[0127]

[0128] in, Indicates the pilot transmission power. Represents τ p An additive noise vector that satisfies

[0129] Based on the received pilot vector, the channel estimate obtained using least squares is:

[0130]

[0131] For the transpose and conjugate of the terminal n pilot sequence;

[0132] Use x n p represents the data sent by terminal n. n p represents the power of transmitted data. n Satisfy p n =η n p n,max p n,max η represents the maximum transmit power of terminal n. n Let n represent the power transmit constraint factor of terminal n. Then the data received at the m-th AP node is:

[0133]

[0134] γ m Let be the additive noise vector received at the m-th AP node;

[0135] The data obtained by merging at the CPU is:

[0136]

[0137] M is the adjoint matrix of the channel estimation matrix. n To represent a set, terminal n is served by APs in this set;

[0138] The expression for the signal-to-noise ratio (SINR) of the uplink from terminal n is:

[0139]

[0140] p n' Let g be the power between the n'-th user and each AP node. m,n' Let σ be the channel coefficient between the n'th user and the m'th AP node, and σ be the noise power coefficient, which is equal to the variables mentioned above. Satisfied σ in.

[0141] In step S402, the process of the CPU calculating the overall network energy consumption and latency consumption includes:

[0142] Based on the SINR expression above, the transmission rate between the terminal and the CPU is expressed as:

[0143]

[0144] τ c For the duration of each coherent interval, B CPU，n This refers to the bandwidth allocated to the nth user for transmitting data with the CPU.

[0145] Similarly, the signal-to-noise ratio between terminal n and the m-th AP node is expressed as:

[0146]

[0147] The transmission rate between terminal n and the m-th AP node is expressed as:

[0148]

[0149] B m,n The bandwidth allocated to the m-th AP is the transmission bandwidth between the n-th user and the m-th node. The bandwidth allocated to the m-th AP cannot exceed its total available bandwidth resource B. m ;;

[0150] The tasks are independent and fine-grained, meaning they can be broken down into arbitrary parts that can participate in parallel offloading between edge servers and the CPU, using ν. n This indicates the number of processor clock cycles required to process 1 bit of data, and specifies the computing server clock speed as f. m,n C n α is the size of the data to be transmitted. m,nLet n represent the parallel offloading coefficient allocated to computing server m by terminal n. Then, its computation latency is:

[0151]

[0152] The energy consumption required by the computing server to process tasks is expressed as:

[0153]

[0154] Among them, the κ parameter depends on the architecture design of each terminal chip;

[0155] The total energy consumed is expressed as follows:

[0156]

[0157] The ω parameter describes the energy consumed in executing one clock cycle.

[0158] Using R link To describe the maximum transmission rate of the fronthaul link, the transmission delay of each link is expressed as:

[0159]

[0160] Because parallel offloading is used, the total delay is equal to the sum of the maximum delays for transmission and computation on a single link, that is:

[0161]

[0162] The transmission energy consumption is then expressed as:

[0163]

[0164] Total energy consumption is expressed as:

[0165]

[0166] In step S403, the process by which the CPU calculates the channel gain and the load information of each network node and link includes:

[0167] In this invention, each terminal sorts the APs in descending order based on the gain provided by each AP, and selects the AP with the highest gain as the primary AP to maintain the connection. The other APs in the AP cluster serving the terminal are determined by a resource scheduling algorithm. The link gain is measured by the large-scale fading coefficient.

[0168]

[0169] Use M nLet m be a set, where terminal n is served by APs in this set, and define an L×L binary diagonal matrix to describe the association between terminal n and the m-th AP:

[0170]

[0171] Use L m Let represent the forward link bandwidth of the m-th AP. Then, the link congestion level is expressed as:

[0172]

[0173] Similarly, using F m Let m represent the total computing resources at computing server m. Then, the computing resource utilization rate is expressed as:

[0174]

[0175] In step S404, considering network energy consumption, time consumption, and network load balancing, a decision problem is formed. The objective function is expressed as a comprehensive evaluation index of network energy consumption and latency, where ε is the weighting factor for energy consumption and time. In practice, this can be flexibly adjusted according to specific network requirements. The objective function is expressed as:

[0176]

[0177]

[0178] Where ε is the weighting factor for energy consumption and time, N1 and N2 represent the set of terminals sending computing services and the set of terminals sending communication services, respectively; C1 constrains the range of task offloading ratio; C2 constraint reflects the latency requirement of each computing service; C3 constraint reflects the transmission rate requirement of each communication service; C4 constraint reflects that the channel gain obtained by the terminal connection must be greater than the set minimum channel gain score. C5 constraint limits the frequency resource allocation range; C6 constraint reflects the usage of communication and computing resources by each node; C7 constraint limits the size of data generated by the terminal; C8 constraint adjusts the maximum number of connections for parallel offloading in a timely manner according to network load; C9 constraint limits the maximum number of connections for each AP node, making the network scalable; C10 constraint is a weight factor range constraint.

[0179] In step S404, reinforcement learning is used to obtain the optimal decision. The process includes:

[0180] The objective function of the above problem is non-convex, making it difficult to directly obtain the optimal solution. Usually, a suboptimal solution can be obtained based on optimization algorithms. This invention uses reinforcement learning to construct an action space and a state space, and by setting a reasonable reward function, an optimal unloading scheme is obtained.

[0181] Specifically, this invention uses (S,A,P,R,G) to describe a discrete-time Markov decision process, where S represents the state space, A represents the action space, P(a|s) is the probability of taking action a in state s, R(s,a,s') represents the immediate reward for taking action a to change state from s to s', and G represents the long-term reward.

[0182] Each element in the state space includes: χ m τ m , Task n Task n This indicates the task request information sent to the CPU by each terminal at the current moment, including the task data size C. n Maximum transmit power p n and maximum task latency

[0183] Each element in the action space includes: {D m,n},{B m,n},{α m,n},{η n};

[0184] Since the objective function is minimized, the reward is set to: -[εT] n +(1-ε)E n ].

[0185] The Actor-Critic algorithm is a hybrid of policy-based and value-based algorithms. Actor and Critic are the two components of Actor-Critic. The policy-based part can be approximated as:

[0186] π θ (s,a)=P(a∣s,θ)≈π(a∣s)

[0187] The value-based component can be approximated as:

[0188]

[0189]

[0190] The Monte Carlo policy gradient reinforcement algorithm can be modified into the Actor-Critic algorithm. In the Monte Carlo policy gradient reinforcement algorithm, the update formula for the Actor network model parameter θ is:

[0191]

[0192] Where α is the iteration step size variable, The gradient of the derivative with respect to θ, π θ The policy function containing the parameter θ, the score function The Gaussian score function can be used as an option. t The state value at each time position t;

[0193] Critic calculates the current state v using a Q-network. t The optimal value, and the actor utilizes v t The policy function parameters θ are iteratively updated, then an action is selected, and feedback and a new state are obtained. The critic uses the feedback and the new state to update the Q-network parameters ω. The critic then uses the new network parameters ω to help the actor calculate the state v. t The optimal value.

[0194] For the model parameters ω of the critic itself, the mean squared error loss function is typically used for iterative updates. A linear Q-function is used, such as Q(s,a,w)=φ(s,a). T If ω is constant, then the update formula for the critic network model parameter ω is:

[0195] ω=ω+βδφ(s,a)

[0196] Where β is the iteration step size variable, φ is the feature describing state s and action a, and δ is the temporal difference error, expressed as: δ = R t+1 +uQ(S t+1 A t+1 )-Q(S t A t ), R t+1 Let represent the instantaneous reward at time t+1, u be the decay factor variable, and Q be the linear Q-function.

[0197] The reinforcement learning algorithm based on Actor-Critic in this invention is as follows:

[0198] Input: Action set K, step size α, β, decay factor u, exploration rate r, number of iterations T, state feature dimension Num, Critic network structure and Actor network structure

[0199] Output: network parameters θ for the actor, and network parameters ω for the critic;

[0200] Initialization: Randomly initialize the Q-values ​​for all states and actions.

[0201] For i = 1 to T, do

[0202] (a) Initialize S to the first state of the current state sequence and obtain its eigenvector φ(S);

[0203] (b) In the actor network, the input is φ(S), the output is action A, and the feedback is R based on the new state S' obtained from A;

[0204] (c) In the critic network, input φ(S) and φ(S') to obtain Q-value outputs V(S) and V(S');

[0205] (d) Calculate the time difference error TD: δ=R+uV(S')-V(S);

[0206] (e) Update the gradient of the critic network parameter ω using the mean-variance loss function: ∑(R+uV(S')-V(S,W)) 2 ;

[0207] (f) Update actor network parameters θ:

[0208] In summary, this invention provides a hybrid resource scheduling method for flexibly scheduling communication and computing resources across multiple access points (APs). This avoids the process of APs uploading tasks to the CPU when computing resources are excessively occupied, thus preventing the complete depletion of node resources due to excessive communication resource consumption. It schedules nodes with relatively abundant communication resources to complete the uploading tasks. This also alleviates the problem of increased resource contention and service latency caused by parallel offloading under high task request rates. The parallel offloading mechanism is adjusted promptly based on real-time load trends; the maximum number of parallel offloads is reduced when the task volume increases. Reinforcement learning is used to dynamically adjust the current resource scheduling strategy by interacting with the environment in real time to obtain the load changes of nodes and links. This achieves maximum utilization of communication and computing resources within the UCcell-free framework, meeting the service quality requirements of different service types while ensuring load balancing and improving overall network efficiency.

[0209] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. However, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hybrid resource scheduling method integrating general computing and data processing, characterized in that, Includes the following steps: S1. After generating computing or communication services, the terminal sends a request to the main AP node. The request includes an estimated amount of communication and computing resources required, priority information, maximum transmit power, and maximum upload latency; Each terminal sorts the AP nodes in descending order according to the gain provided by each AP node, selects the AP node with the largest gain as the primary AP node and maintains the connection, and the others are secondary AP nodes; S2. Each AP node sends its own resource load information to the CPU in real time. S3. Each AP node will forward the received service request information to the CPU; S4. The CPU determines a resource scheduling scheme based on the service request information of each AP node and the load information of the link, and feeds it back to each AP node through a hybrid resource scheduling algorithm. S5. The main AP node notifies the terminal to uninstall the secondary AP node and the task uninstallation ratio. S6. The terminal splits the task according to the resource scheduling scheme and unloads it to each AP node in parallel. S7. Each AP node updates its load information and forwards the task to the CPU; S8, CPU returns the business processing results that were offloaded to CPU as required; S9. Each AP node returns the business processing results that were unloaded to itself as needed; S10. Each AP node sends updated load information to the CPU as it releases occupied resources.

2. The hybrid resource scheduling method integrating general computing and data processing according to claim 1, characterized in that, In step S4, the process of determining the resource scheduling scheme using the hybrid resource scheduling algorithm is as follows: S401, The CPU calculates the signal-to-noise ratio of the received data; S402, CPU calculates the overall energy consumption and latency consumption of the network; S403, CPU calculates channel gain, load information of each network node and link; S404 and CPU form a decision problem model based on the processed data. The objective function is expressed as a comprehensive evaluation index of network energy consumption and latency. The optimal decision is obtained by solving the model.

3. The hybrid resource scheduling method with integrated computing as described in claim 2, characterized in that, In step S401, the process of the CPU calculating the signal-to-noise ratio of the received data includes: Use g m,n Let β represent the channel coefficient between terminal n and the m-th AP node. m,n and h m,n Let represent the large-scale fading coefficient and the small-scale fading coefficient between the nth terminal and the mth AP node, respectively. The small-scale fading coefficient consists of independent and identically distributed complex normal random variables. Then, the channel coefficient is expressed as: g m，n =b m，n h m，n Let Φ n The pilot sequence of terminal n, and the pilot vector received at the m-th AP node are represented as: in, Indicates the pilot transmission power. Represents τ p An additive noise vector that satisfies σ is the noise power factor; Based on the received pilot vector, the channel estimate obtained using least squares is: For the transpose and conjugate of the terminal n pilot sequence; Use x n p represents the data sent by terminal n. n p represents the power of transmitted data. n Satisfy p n =η n p n，max p n，max η represents the maximum transmit power of terminal n. n Let n represent the power transmit constraint factor of terminal n. Then the data received at the m-th AP node is: γ m Let be the additive noise vector received at the m-th AP node; The data obtained by merging at the CPU is: M is the adjoint matrix of the channel estimation matrix. n To represent a set, terminal n is served by APs in this set; The expression for the uplink signal-to-noise ratio of terminal n is: p n′ Let g be the power between the n′-th user and each AP node. m，n′ Let be the channel coefficient between the n′-th user and the m-th AP node.

4. The hybrid resource scheduling method integrating general computing and data processing according to claim 3, characterized in that, In step S402, the process of the CPU calculating the overall network energy consumption and latency consumption includes: The transmission rate between the terminal and the CPU is expressed as: τ c For the duration of each coherent interval, B CPU，n This refers to the bandwidth allocated to the nth user for transmitting data with the CPU. The signal-to-noise ratio between terminal n and the m-th AP node is expressed as: The transmission rate between terminal n and the m-th AP node is expressed as: B m，n The bandwidth allocated to the m-th AP is the transmission bandwidth between the n-th user and the m-th node. The bandwidth allocated to the m-th AP cannot exceed its total available bandwidth resource B. m ; Use v n This indicates the number of processor clock cycles required to process 1 bit of data, and specifies the computing server clock speed as f. m，n C n α is the size of the data to be transmitted. m，n Let n represent the parallel offloading coefficient allocated to computing server m by terminal n. Then, its computation latency is: The energy consumption required by the computing server to process tasks is expressed as: Among them, the κ parameter depends on the architecture design of each terminal chip; The total energy consumed is expressed as follows: The ω parameter describes the energy consumed in executing one clock cycle. Using R link To describe the maximum transmission rate of the fronthaul link, the transmission delay of each link is expressed as: The total delay equals the sum of the maximum delay for transmission and computation on a single link, that is: The transmission energy consumption is then expressed as: Total energy consumption is expressed as:

5. The hybrid resource scheduling method integrating general computing and data processing according to claim 4, characterized in that, In step S403, the process by which the CPU calculates the channel gain and the load information of each network node and link includes: Link gain score between the nth user and the mth AP node It is measured by a function of the corresponding channel coefficients: Define an L×L binary diagonal matrix to describe the association between terminal n and the m-th AP: Use L m Let represent the forward link bandwidth of the m-th AP. Then, the link congestion level is expressed as: Use F m Let m represent the total computing resources at computing server m. Then, the computing resource utilization rate is expressed as:

6. The hybrid resource scheduling method integrating general computing and data processing according to claim 5, characterized in that, In step S404, the objective function is expressed as: Where ε is the weighting factor for energy consumption and time, N1 and N2 represent the set of terminals sending computing services and the set of terminals sending communication services, respectively; C1 constrains the range of task offloading ratio; C2 constraint reflects the latency requirement of each computing service; C3 constraint reflects the transmission rate requirement of each communication service; C4 constraint reflects that the channel gain obtained by the terminal connection must be greater than the set minimum channel gain score. C5 constraint limits the frequency resource allocation range; C6 constraint reflects the usage of communication and computing resources by each node; C7 constraint limits the size of data generated by the terminal; C8 constraint adjusts the maximum number of connections for parallel offloading in a timely manner according to network load; C9 constraint limits the maximum number of connections for each AP node, making the network scalable; C10 constraint is a weight factor range constraint.

7. The hybrid resource scheduling method integrating general computing and data processing according to claim 6, characterized in that, In step S404, reinforcement learning is used to obtain the optimal decision. The process includes: Let (S, A, P, R, G) describe a discrete-time Markov decision process, where S represents the state space, A represents the action space, P(a|s) is the probability of taking action a in state s, R(s, a, s') represents the immediate reward of taking action a to change state from s to s', and G represents the long-term reward. Each element in the state space includes: χ m τ m , Task n Task n This indicates the task request information sent to the CPU by each terminal at the current moment, including the task data size C. n Maximum transmit power p n and maximum task latency Each element in the action space includes: {D m，n }, {B m，n }, {α m，n },{η n }; The reward is set to: -[εT n +(1-ε)E n ]; The formula for updating the Actor network model parameter θ is: Where α is the iteration step size variable, The gradient of the derivative with respect to θ, π θ The policy function containing the parameter θ, the score function You can choose to use the Gaussian score function, v t The state value at each time position t; The formula for updating the parameter ω of the critic network model is: ω=ω+βδφ(s,a) Where β is the iteration step size variable, φ is the feature describing state s and action a, and δ is the temporal difference error, expressed as: δ = R t+1 +uQ(S t+1 A t+1 )-Q(S t A t ), R t+1 Let represent the instantaneous reward at time t+1, u be the decay factor variable, and Q be the linear Q-function.

8. The hybrid resource scheduling method integrating general computing and data processing according to claim 6, characterized in that, In step S4, the CPU will assign the task processing strategy parameter {D} m，n }, {B m，n }, {α m，n },{η n } is sent to the requesting terminal and each AP node providing the service.

Images