Wireless resource scheduling method and system

Abstract

The application discloses a wireless resource scheduling method and system, and the method implementation comprises the following steps: acquiring resource supply nodes, communication users and schedulable resources in a wireless communication network; constructing user association constraints and resource allocation constraints; constructing a target optimization function with the maximum network overall transmission efficiency as a target under the boundary conditions of the user association constraints and the resource allocation constraints; configuring each resource supply node as a decision unit, and each decision unit sequentially executes user connection operations and resource allocation operations according to a preset decision sequence, and generates corresponding local network transmission efficiency based on the execution results, wherein a subsequent decision unit executes corresponding operations after updating the decision parameters of each decision unit with the local network transmission efficiency corresponding to a previous decision unit as a reward signal, and the target optimization function is solved based on the execution results of all resource supply nodes to obtain the maximum overall network transmission efficiency under the constraint boundary. Therefore, the service quality of a large-scale heterogeneous network is significantly improved.

Description

Wireless Resource Scheduling Methods and Systems Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a wireless resource scheduling method and system. Background Technology

[0002] With the rapid development of mobile internet, wireless data traffic has experienced explosive growth, and users' demands for Quality of Service (QoS) continue to rise. Traditional cellular networks are struggling to handle the increasing workload. Heterogeneous networks (HetNets), as an important evolutionary structure for next-generation wireless networks, achieve efficient reuse of spatial spectrum by introducing various types of resource provisioning nodes. This provides significant potential for improving network service quality and capacity, making it a key technological direction for addressing the challenges of traffic growth.

[0003] In large-scale heterogeneous networks, the joint optimization problem of user association (UA) and resource allocation (RA) (UARA) is the core of ensuring network performance. Taking a traditional two-layer heterogeneous network as an example, this problem involves the matching of a large number of resource supply nodes, communication users and schedulable resources, and the solution space grows exponentially. At the same time, real networks have high requirements for real-time decision-making, making it extremely challenging to obtain a joint scheduling strategy that balances optimality and real-time performance.

[0004] Currently, existing techniques for solving the UARA joint optimization problem can be mainly divided into two categories: traditional methods and reinforcement learning methods, but both have significant shortcomings:

[0005] Traditional methods include heuristic techniques, optimization-based methods, and game theory methods. Heuristic techniques can only provide suboptimal solutions and lack explicit performance guarantees. Optimization-based methods are often centralized architectures that require instantaneous global channel state information (CSI). However, in large-scale, ultra-dense heterogeneous networks, accurately acquiring such complete wireless environment knowledge is extremely difficult or even impossible, and their computational complexity increases proportionally with network size, making them unsuitable for the real-time scheduling needs of large-scale networks. Game theory methods seek Nash equilibrium solutions by modeling non-cooperative games between network entities, but these methods heavily rely on fixed system configurations and have poor adaptability to dynamically changing wireless environments, requiring reconfiguration to match new system settings.

[0006] Reinforcement learning methods offer new solutions to the UARA problem. For example, the DQN (Deep Q-Network) algorithm treats the joint allocation problem as a multi-dimensional policy exploration problem, which can dynamically adjust resource allocation based on user information. However, this type of algorithm suffers from low sampling efficiency and unstable training process, and the design of the reward function is sensitive to the impact on experimental results, making it difficult to reproduce the optimization results. In addition, although existing reinforcement learning-based schemes (such as PPO proximal policy optimization) attempt to build a constrained optimization framework, they do not explicitly explain the constraint handling mechanism, making it difficult to guarantee that the output policy fully meets the preset constraints of user association and resource allocation, thus affecting the feasibility of the policy.

[0007] Meanwhile, deep reinforcement learning faces common challenges in wireless resource allocation applications, such as high training complexity and difficulty in system integration, which further limits its practical deployment effectiveness in large-scale heterogeneous networks. Summary of the Invention

[0008] Therefore, it is necessary to provide a wireless resource scheduling method and system to address the above-mentioned technical problems and solve at least one of the problems existing in the prior art.

[0009] Firstly, a wireless resource scheduling method is provided, including:

[0010] Acquire information on resource supply nodes, communication users, and schedulable resources in a wireless communication network;

[0011] Based on network operation requirements, resource supply nodes, communication users, and schedulable resources, user association constraints and resource allocation constraints are constructed.

[0012] Using the aforementioned user association constraints and resource allocation constraints as boundary conditions, a target optimization function is constructed with the goal of maximizing the overall network transmission efficiency.

[0013] Each resource supply node is configured as a decision unit. Each decision unit executes user connection operations and resource allocation operations in sequence according to a preset decision sequence, and generates the corresponding local network transmission performance based on the execution results. Subsequently, each decision unit updates its own decision parameters with the local network transmission performance of the preceding resource supply node as a reward signal and then executes the corresponding operation. Based on the execution results of all resource supply nodes, the objective optimization function is solved to obtain the maximum overall network transmission performance under the constraint boundary.

[0014] Secondly, a wireless resource scheduling system is provided, comprising:

[0015] The data acquisition unit is used to acquire information on resource supply nodes, communication users, and schedulable resources in the wireless communication network.

[0016] The constraint construction unit is used to construct user-related constraints and resource allocation constraints based on network operation requirements, resource supply nodes, communication users, and schedulable resources.

[0017] The objective optimization function construction unit is used to construct an objective optimization function with the user association constraints and resource allocation constraints as boundary conditions, aiming to maximize the overall network transmission efficiency.

[0018] The objective optimization function solving unit is used to configure each resource supply node as a decision unit. Each decision unit executes user connection operations and resource allocation operations in sequence according to a preset decision sequence, and generates the corresponding local network transmission efficiency based on the execution results. Subsequently, each decision unit updates its own decision parameters as a reward signal based on the local network transmission efficiency of the preceding resource supply node and then executes the corresponding operation. Based on the execution results of all resource supply nodes, the objective optimization function is solved to obtain the maximum overall network transmission efficiency under the constraint boundary.

[0019] This application provides a wireless resource scheduling method and system. The method includes: acquiring resource supply nodes, communication users, and schedulable resources in a wireless communication network; constructing user association constraints and resource allocation constraints based on network operation requirements, resource supply nodes, communication users, and schedulable resources; constructing a target optimization function with the goal of maximizing the overall network transmission efficiency using the user association constraints and resource allocation constraints as boundary conditions; configuring each resource supply node as a decision unit, and each decision unit sequentially executing user connection operations and resource allocation operations according to a preset decision sequence, and generating corresponding local network transmission efficiency based on the execution results. Each subsequent decision unit updates its decision parameters using the local network transmission efficiency corresponding to the preceding resource supply node as a reward signal before executing the corresponding operation. Based on the execution results of all resource supply nodes, the target optimization function is solved to obtain the maximized overall network transmission efficiency under the constraint boundaries. In this embodiment, by configuring each resource supply node as an independent intelligent decision-making unit to perform user connection and resource allocation operations sequentially, the exponential solution space of the traditional UARA problem is transformed into a linear decision-making process, significantly reducing computational complexity to adapt to large-scale heterogeneous network scenarios. Simultaneously, a target optimization function is constructed using preset user association constraints and resource allocation constraints as boundaries. Combined with the natural satisfaction of constraints by subsequent decision units' reference to previous results, this solves the problem of fuzzy constraint handling in existing reinforcement learning methods. A local search algorithm is used to refine the initial policy's neighborhood and mine better solutions. The parameters are iteratively optimized using the overall network transmission performance as a reward signal, overcoming the suboptimal nature of traditional heuristic methods and the instability of reinforcement learning training. Furthermore, it does not rely on complete wireless environment knowledge, can dynamically adapt to network changes, and ultimately maximizes network transmission performance within the constraint boundaries. It stably outputs a scheduling scheme that balances real-time performance, feasibility, and optimality, significantly improving the service quality of large-scale heterogeneous networks. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 is a flowchart illustrating a wireless resource scheduling method according to an embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of a model structure of a multi-agent model in one embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of the distribution of multiple base stations in one embodiment of the present invention;

[0024] Figure 4 is a schematic diagram comparing the rewards of the MAT+Local Search algorithm and the MAX_SINR algorithm in one embodiment of the present invention;

[0025] Figure 5 is a structural schematic diagram of a wireless resource scheduling system according to an embodiment of the present invention;

[0026] Figure 6 is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] In one embodiment, as shown in FIG1, a wireless resource scheduling method is provided, comprising the following steps:

[0029] In step S110, the resource supply nodes, communication users, and schedulable resources in the wireless communication network are obtained;

[0030] Optionally, several resource supply nodes, several communication users, and several schedulable resources corresponding to each resource supply node in a wireless communication network (such as a heterogeneous network with an Orthogonal Frequency Division Multiple Access (OFDMA) architecture) are identified and acquired. It should be noted that a resource supply node refers to a network infrastructure entity with wireless signal transmission capabilities, capable of providing network access services to communication users, and able to allocate schedulable resources. It is the resource supplier and scheduling execution entity in the wireless communication network. For example, various base station devices in heterogeneous networks such as macro base stations and micro base stations, or network nodes with resource allocation capabilities such as gateway devices in the Internet of Things (IoT). A communication user refers to a terminal-side entity that needs to access the wireless communication network and obtain communication services (such as data transmission, voice calls, etc.), which may include various terminal devices with wireless communication functions such as smartphones, tablets, laptops, IoT terminals (such as sensors, smart devices), and industrial control terminals. Schedulable resources refer to scarce network resources that resource supply nodes can allocate to communication users to support them in completing communication services, including physical resource blocks (PRBs), channel resources, transmission time slots, and transmit power resources in wireless communication.

[0031] In step S120, user association constraints and resource allocation constraints are constructed based on network operation requirements, resource supply nodes, communication users and schedulable resources;

[0032] Optionally, based on the core operational requirements of the wireless communication network (such as service quality assurance, resource utilization improvement, load balancing control, etc.), resource supply nodes, communication users and the maximum access capacity of schedulable resources, service requirements and total quantity limits, user association constraints and resource allocation constraints are constructed to clearly define the association rules between a single communication user and a resource supply node and the allocation rules between a single schedulable resource and a communication user, so as to provide compliance boundaries for the subsequent construction of the objective optimization function and decision execution.

[0033] The user association constraint limits a single communication user to establish a connection with only one resource supply node and the maximum number of connected users of a single resource supply node does not exceed a preset threshold; the resource allocation constraint limits a single schedulable resource to be allocated only to a single communication user and the maximum amount of resources acquired by a single communication user does not exceed a preset threshold.

[0034] For example, taking the resource supply node as a base station, it is possible to obtain information about multiple base stations currently used by the wireless communication network. Communication users Resource blocks used by the base station for allocation .use Represents the set of all base stations. Represents the set of all communication users, in particular, Indicates base station Users within signal range, This represents the set of resource blocks owned by a base station. Assuming each base station owns the same number of resource blocks, that is, ... One resource block. Considering that different base stations may have different user sets, the binary user association vector is... Given, if the base station Connect users Then there is ,otherwise ,use This represents the potential base stations that user i can connect to. Furthermore, the spectrum resources k of each base station are allocated to its associated users. For user i... Its allocation is determined by 0-1 decision variables. Provided. If the base station spectrum resources If assigned to user u, then... ,otherwise , This represents the set of users connected to base stations. Each communication user can only choose one base station to connect to, and a single base station can have a maximum of [number] connections. For each communication user, the user association constraint is... It can be represented as:

[0035] ;

[0036] ;

[0037] Regarding subcarrier limitations, assuming each resource block It can be assigned to at most one user, and each user can be assigned a maximum of [number missing] [units missing]. One resource block Then the resource allocation constraint can be expressed as:

[0038] ;

[0039] .

[0040] In step S130, using the user association constraints and resource allocation constraints as boundary conditions, a target optimization function is constructed with the goal of maximizing the overall network transmission efficiency.

[0041] Optionally, in the joint optimization problem of user association and resource allocation, maximizing the achievable network rate is the core requirement. By using pre-constructed user association constraints (such as a single communication user establishing a connection only with a single resource supply node, and the maximum number of connected users of a single resource supply node not exceeding a preset threshold) and resource allocation constraints (such as a single schedulable resource being allocated only to a single communication user, and the maximum resource acquisition amount of a single communication user not exceeding a preset threshold) as rigid boundary conditions, the legitimate scope of decision optimization is clearly defined. The objective optimization function aims to maximize the overall network transmission efficiency (such as maximizing the total throughput of all resource supply nodes, which can be the sum of the throughput of all resource supply nodes and communication user pairs). By integrating the transmission characteristics of communication links, resource supply and demand matching logic, and network operation requirements, an objective optimization function that combines constraint compliance and objective orientation is constructed. By solving this function, the overall network transmission efficiency is maximized.

[0042] In step S140, each resource supply node is configured as a decision unit. Each decision unit executes user connection operations and resource allocation operations sequentially according to a preset decision sequence, and generates corresponding local network transmission performance based on the execution results. Subsequently, each decision unit updates its decision parameters as a reward signal based on the local network transmission performance of the preceding resource supply node before executing the corresponding operation. Based on the execution results of all resource supply nodes, the objective optimization function is solved to obtain the maximum overall network transmission performance under the constraint boundary.

[0043] It should be noted that the User Association and Resource Allocation Joint Optimization (UARA) problem corresponding to the objective optimization function is a non-convex optimization problem. Solving it using traditional methods suffers from drawbacks such as low optimization efficiency, sparse rewards, and high computational complexity. Therefore, this embodiment introduces Multi-Agent Reinforcement Learning (MARL) technology, configuring each resource supply node (such as macro base stations, micro base stations, etc.) in the wireless communication network as an independent intelligent decision-making unit. This endows each node with autonomous decision-making capabilities, enabling it to independently complete user connection operations (i.e., establishing or adjusting the association relationship with communication users according to constraints and network status) and schedulable resource allocation operations (i.e., dynamically allocating its own schedulable resources to associated communication users) based on its own learned and trained decision-making strategies (covering user association priority judgment, resource allocation rule adaptation, etc.).

[0044] After each decision-making unit completes a set of operations, it generates a local network transmission performance reflecting the local scheduling quality of that node based on the actual execution effect. During the decision-making process, subsequent decision-making units in the decision sequence use the local network transmission performance of the preceding decision-making units as a quantified reward signal, adjust their own decision logic through parameter update algorithms, and then execute the corresponding decision operation, achieving iterative optimization of the strategy between sequences. After all decision-making units have completed their operations, the user connection state and resource allocation state of the entire network are summarized and substituted into the objective optimization function with user association constraints and resource allocation constraints as boundary conditions to solve for the scheduling result that maximizes the overall network transmission performance within the constraints. The distributed decision-making model avoids the computational bottleneck of the centralized architecture while ensuring that the decisions of each decision-making unit are consistent with the overall network optimization goal.

[0045] In one embodiment of this application, the step of constructing a target optimization function with the user association constraints and resource allocation constraints as boundary conditions, aiming to maximize the overall network transmission performance, includes:

[0046] Based on the transmission power of the communication link, the channel transmission gain, and the environmental noise level, calculate the signal-to-interference ratio of each communication user on each potential allocated resource;

[0047] Based on the signal-to-interference ratio and the preset channel gain coefficient, the signal transmission rate of each communication user on the corresponding potential allocated resources is calculated.

[0048] Using the user association constraints and resource allocation constraints as boundary conditions, and combining the preset transmission bandwidth and the signal transmission rate, a target optimization function is constructed with the goal of maximizing the overall network transmission efficiency.

[0049] The user association constraint limits a single communication user to establish a connection with only one resource supply node and the maximum number of connected users of a single resource supply node does not exceed a preset threshold; the resource allocation constraint limits a single schedulable resource to be allocated only to a single communication user and the maximum amount of resources a single communication user can acquire does not exceed a preset threshold.

[0050] Optionally, firstly, by combining the transmit power of the resource supply nodes, the channel transmission gain of the communication link, and the noise level in the network environment, the signal-to-interference ratio (SINR) of each communication user on each potential allocated resource (such as physical resource blocks, channel resources, etc.) is accurately calculated using a signal propagation model. This ratio directly reflects the anti-interference capability and transmission quality foundation of the communication link. Secondly, based on the SINR calculated above, and combined with a preset channel gain coefficient (calibrated parameters pre-configured according to network scenarios, communication protocols, etc.), the actual signal transmission rate of each communication user on the corresponding potential allocated resource is further derived using a rate calculation formula, providing a basis for the quantitative evaluation of network transmission performance. The system provides core metrics; finally, it uses preset user association constraints and resource allocation constraints as rigid boundary conditions (wherein, the user association constraint explicitly limits a single communication user to only one resource-supplying node, and the maximum number of connected users on a single resource-supplying node cannot exceed a preset threshold, avoiding node overload; the resource allocation constraint limits a single schedulable resource to only one communication user, and the maximum resource acquisition amount for a single communication user cannot exceed a set threshold, ensuring the fairness and rationality of resource allocation), and integrates preset transmission bandwidth parameters with the aforementioned calculated signal transmission rates for each user, to construct a target optimization function aimed at maximizing the overall network transmission performance. This target optimization function can be expressed as:

[0051] ;

[0052] Among them, (b) and (c) are user association constraints, which state that each resource supply node can be associated with at most one user. There are a number of communication users, and each user can only connect to one resource provider node. (d) and (e) are resource allocation constraints, which define the schedulable resource blocks within any resource provider node. It can only be assigned to one communication user, and each communication user can be assigned [amount]. One schedulable resource block , Indicates the signal-to-interference ratio. Represents the channel gain coefficient. This represents the weighting coefficient.

[0053] In one embodiment of this application, calculating the signal-to-interference ratio for each communication user on each potential allocated resource based on the transmission power of the communication link, channel transmission gain, and ambient noise level includes:

[0054] Determine the spatial distance between each resource supply node and each communication user, the carrier frequency for signal transmission, and the altitude parameters of the communication users;

[0055] Based on the numerical range of the spatial distance, the propagation scenario type is determined;

[0056] A standardized path loss model is used to determine the attenuation calculation rules matching the propagation scenario type, and the signal attenuation is determined based on the attenuation calculation rules, spatial distance, carrier frequency, and the altitude parameter of the communication user.

[0057] The transmit power, initial channel transmission gain, and ambient noise level parameters of the communication link are obtained, and the initial channel transmission gain is calibrated in conjunction with the signal attenuation to obtain the effective signal power.

[0058] The signal-to-interference ratio of a communication user under the corresponding potential allocation resources and corresponding resource supply nodes is calculated by using the ratio of the effective signal power to the interference plus noise power.

[0059] Optionally, the three-dimensional spatial distance between each resource supply node and each communication user, the carrier frequency used in signal transmission, and the actual deployment altitude parameters of the communication user terminals (such as the altitude of ground terminals and the flight altitude of air terminals) can be obtained through preset methods (such as network-side positioning modules, terminal-reported data, or wireless signal detection technology). Then, based on preset thresholds for near and far spatial distances, and according to the numerical range of the spatial distance between each resource supply node and the communication user, the corresponding propagation scenario type (line-of-sight propagation scenario or non-line-of-sight propagation scenario) can be determined. Furthermore, the industry-standardized path loss model Path can be used. Loss is calculated by matching the corresponding signal attenuation calculation rule based on the determined propagation scenario type. Then, the previously obtained spatial distance, carrier frequency, and communication user height parameters are substituted into the attenuation calculation rule. The path attenuation of the signal during transmission is obtained through quantization calculation, fully considering the impact of factors such as terrain and obstructions on signal propagation. Subsequently, the transmit power of the communication link (the signal transmit power setting value of the resource supply node), the initial channel transmission gain (the ideal channel gain under no attenuation condition), and environmental noise level parameters (such as thermal noise, background interference, etc.) are extracted from the network communication protocol configuration or link real-time monitoring data. The initial channel transmission gain is dynamically calibrated using the calculated signal attenuation to eliminate the impact of path attenuation and obtain the effective signal power at the communication user's receiving end. Finally, by calculating the ratio of this effective signal power to the sum of interference and noise power (including the co-frequency interference power of other resource supply node signals, the sum of environmental noise power, etc.), the signal interference ratio of each communication user on each potential allocated resource is accurately determined.

[0060] For example, since the radio coverage areas of the deployed resource provisioning nodes overlap, there is a co-channel interference problem. Assuming... It is a base station In resource block The transmission power of the base station Used in RB The first The signal-to-interference-plus-noise ratio (SINR) at each user can be expressed as:

[0061] ;

[0062] in For base stations With RB Assigned to users The channel gain matrix between This represents the noise power spectral density.

[0063] Based on this interference-to-noise ratio, the maximum achievable rate can be further calculated, specifically as follows:

[0064] ;

[0065] Wherein, BW is the bandwidth transmission rate in the base station system, usually measured in MHz.

[0066] In one embodiment of this application, the propagation scenario types include line-of-sight propagation scenarios and non-line-of-sight propagation scenarios. A line-of-sight propagation scenario refers to a spatial distance within a short-range interval, while a non-line-of-sight propagation scenario refers to a spatial distance within a long-range interval. Determining the signal attenuation amount based on the attenuation calculation rules, spatial distance, carrier frequency, and the altitude parameter of the communication user includes:

[0067] In the case of line-of-sight propagation, the signal attenuation is calculated based on the spatial distance and carrier frequency using the attenuation calculation rules corresponding to the line-of-sight scenario.

[0068] In the case of non-line-of-sight propagation scenarios, the signal attenuation is calculated based on the three-dimensional spatial distance, carrier frequency, and the difference between the actual height of the terminal and the reference height, using the attenuation calculation rules corresponding to non-line-of-sight scenarios.

[0069] It should be noted that in practice, the channel gain H may be affected by path loss during channel transmission. Therefore, a path loss model can be used to model the channel. For example, the 3GPP UMA (UrbanMacro) model can be used to calculate the signal attenuation in line-of-sight (LOS) and non-line-of-sight (NLOS) propagation scenarios respectively.

[0070] The path loss calculation formula for line-of-sight (LOS) propagation scenarios is as follows:

[0071] ;

[0072] It can be used to calculate signal attenuation when there is a direct, unobstructed visual path between the transmitter and receiver. This is the three-dimensional straight-line distance between the transmitter and receiver. The coefficient 22 indicates that the path loss exponent is equivalent to 2.2, meaning that for every tenfold increase in distance, the path loss increases by 22 dB. This is the carrier frequency (unit: Hz, Hertz). A coefficient of 20 is a typical value in free-space path loss, meaning that for every 10-fold increase in frequency, the path loss increases by 20 dB.

[0073] The formula for calculating path loss in non-line-of-sight (NLOS) propagation scenarios is as follows:

[0074] ;

[0075] It is used to calculate signal attenuation when there is no direct path between the transmitter and receiver due to obstacles such as buildings and trees. Attenuation is typically more severe in NLOS environments than in LOS environments. The height of the user terminal (in meters) typically refers to the height of the mobile phone or receiving device. 1.5 meters is a reference height (in meters). This correction term means that for every 1 meter increase in the user terminal's height relative to the 1.5-meter reference height, the path loss decreases by 0.6 dB (because the sign is negative). Conversely, for every 1 meter decrease in height, the path loss increases by 0.6 dB. This is because a higher terminal may more easily avoid some ground obstacles, thus improving channel conditions.

[0076] In the simulation environment, a probabilistic formula is set up to determine whether the signal transmission is line-of-sight transmission:

[0077] ;

[0078] When the distance is less than 18 meters, it is definitely visual distance (LOS) with a probability of 1; when the distance is between 18 and 1000 meters, the probability of LOS decreases as the distance increases, as given by the piecewise formula above; when the distance is greater than 1000 meters, the probability of visual distance is 0, which means it is definitely non-visual distance (NLOS).

[0079] Finally, the received power after path loss can be expressed by the following formula:

[0080] .

[0081] in, For transmitting power, This refers to path loss.

[0082] In one embodiment of this application, configuring each resource supply node as a decision-making unit, and having each decision-making unit sequentially execute user connection operations and resource allocation operations according to a preset decision sequence, includes:

[0083] According to the preset sorting rules, the execution order of each resource supply node is determined, forming a decision sequence;

[0084] According to the order of each resource supply node in the decision sequence, user connection operations and resource allocation operations are executed sequentially based on their own decision strategies. When the subsequent decision unit executes a decision, it observes the decision strategies of the preceding resource supply nodes and the corresponding local network transmission performance, and combines the environmental coding data output by the multi-agent model to generate its current decision strategy, so as to execute user connection operations and resource allocation operations based on the current decision strategy.

[0085] Optionally, all resource supply nodes are ordered according to a preset sorting rule (such as based on the coverage area, current load, hardware performance level, or random sorting of resource supply nodes), clarifying the execution priority of each node, thereby forming a fixed decision sequence. According to the order of each resource supply node in the decision sequence, each node performs user connection operations (such as establishing, maintaining, or adjusting the association with communication users) and resource allocation operations (such as allocating schedulable resources such as physical resource blocks and channel resources to associated users) based on its initial decision strategy or an iteratively optimized decision strategy. After each resource supply node completes its corresponding operation, it generates a local network transmission performance reflecting its scheduling effect based on local performance indicators such as the signal transmission rate and resource utilization of the users it serves.

[0086] It should be noted that before executing a decision, the subsequent decision-making unit observes the decision-making strategies of the preceding resource supply nodes (including core decision-making information such as user association schemes and resource allocation vectors) and their corresponding local network transmission performance. This local network transmission performance is used as a quantified reward signal. Simultaneously, it combines the environmental encoding data output by the multi-agent model (this data is generated by the encoder of the multi-agent model after encoding the local observation information of each resource supply node, the interference relationship between nodes, the network load distribution, and other global states, which can comprehensively reflect the current network operating environment). Through the mask self-attention layer of the model decoder, it associates the preceding decision-making actions and fuses the environmental encoding features through the mask cross-attention layer, dynamically adjusting its own decision parameters to generate the optimal decision-making strategy adapted to the current network scenario. Then, based on this strategy, it executes user connection and resource allocation operations, ensuring that the decisions of the subsequent decision-making unit can fully draw on the scheduling experience of the preceding decision-making unit, realizing iterative optimization of the strategy in the sequential decision-making process, and improving the coordination and overall efficiency of the entire network resource scheduling.

[0087] In the decision sequence, when each resource supply node executes the user association and resource block allocation strategy, it observes the allocation strategy of the preceding resource supply node (base station) and the corresponding local network transmission performance (reward). In this case, the overall system advantage function is equivalent to the sum of the local advantage functions of all resource supply nodes that have observed the strategies of their preceding nodes, i.e.:

[0088] ;

[0089] ;

[0090] in, This represents the policy function of a multi-agent model, where o represents the observation information. Represents the overall advantage function. Represents the local dominance function. This represents the action of the m-th resource supply node. This represents the sequence of actions of all nodes preceding the m-th resource supply node. This represents the sequence of actions of the first n resource supply nodes in the decision sequence. The baseline state value function (based only on the expected cumulative reward of the actions of the first h nodes). Representation Strategy The state-action value function under the following conditions The local advantage function is defined by the difference between two value functions.

[0091] Each resource supply node only needs to select a strategy that maximizes the overall reward of the system to avoid the "lazy agent" phenomenon and achieve effective optimization of the allocation strategy.

[0092] Meanwhile, the sequential decision-making architecture significantly improves the computational efficiency of the inference process. This is due to the limited action space of a single resource-supplying node. It only includes the allocation combination of communication users and corresponding resource blocks within its observation range, thus reducing the number of searches in a single inference. This is the cumulative value of the action space for all resource-supplying nodes; in contrast, in traditional reinforcement learning joint optimization schemes, the number of searches per inference... The product of the action space of each resource supply node is easily caused by the multiplicative expansion of the search space, which leads to the dimensionality explosion problem, resulting in the reasoning and computation efficiency being far lower than that of the sequential decision-making scheme in this application.

[0093] In one embodiment of this application, when the subsequent decision-making unit executes a decision, it observes the decision-making strategy of the preceding resource supply node and the corresponding local network transmission performance, and combines it with environmental coding data to generate its own current decision-making strategy. Based on this current decision-making strategy, it executes user connection operations and resource allocation operations, including:

[0094] Deploy the multi-agent model, which includes an encoder and a decoder, wherein the decoder is configured with a masked self-attention layer and a masked cross-attention layer;

[0095] The encoder encodes the local observation information of each resource supply node and the decision-making strategy of the preceding resource supply node to generate environmental coded data containing the local vision information of the resource supply node and the interference relationship between nodes.

[0096] The decoder uses a masked self-attention layer to associate the preceding resource supply node's decision-making and execution actions, and a masked cross-attention layer to fuse the environmental encoded data, thereby generating the current self-decision strategy corresponding to the subsequent decision unit.

[0097] Based on its current decision-making strategy, the system performs user connection operations and resource allocation operations, and records the current decision-making strategy and the corresponding local network transmission performance for observation by the next-level subsequent decision-making unit.

[0098] Optionally, as shown in Figure 2, a multi-agent model is deployed in the decision control layer of the wireless communication network. The preferred model is a multi-agent Transformer (MAT) framework based on the Transformer encoder-decoder architecture (a novel multi-agent learning framework whose multi-agent characteristics can be accurately adapted to the multi-resource supply node scenario in wireless resource scheduling, with each resource supply node corresponding to an agent with independent decision-making capabilities). The decoder of the model is specially configured with a masked self-attention layer and a masked cross-attention layer to meet the needs of action association and environmental information fusion in sequence decision-making. Specifically, the encoder of the multi-agent model receives local observation information from each resource supply node (including its own load status, communication user demand within its coverage area, and local schedulable resource reserves), as well as the decision-making strategies executed by preceding resource supply nodes (including user association schemes, resource allocation vectors, and other key decision-making information). The encoder performs deep encoding on this data using built-in self-attention layers, MLP layers, and residual connections, integrating the scattered observation and decision information into a unified feature vector. This ultimately generates environmental encoded data that includes both the local perspective information of each resource supply node and accurately reflects the signal interference relationship between nodes. Subsequently, the decoder initiates a dual-attention mechanism for collaborative computation: first, the first-layer masked self-attention layer calculates the attention value between the input action of the i-th resource supply node and the actions of the nodes preceding it, ensuring that the node does not consider subsequent nodes when making decisions. Then, the second-layer masked cross-attention layer calculates the cross-attention between the node's action and the observation encoding, allowing the node to consider the current environmental information when making decisions. This association enables nodes to directly bind their actions to environmental feedback, clearly identifying which observation features led to reward changes, thereby accurately locating their own contribution and ensuring a reasonable credit allocation mechanism. After updating the allocation strategy, the decoder adds the current node's action to the action sequence of the old node, preparing for updates by subsequent decision units. Finally, based on its current decision strategy, each subsequent decision unit executes corresponding user connection operations (establishing or adjusting the association with the target communication user) and resource allocation operations (precisely allocating schedulable resources to associated users), and simultaneously records the specific strategy content of this decision and the resulting local network transmission performance. This provides observable decision references and reward signals for the next-level subsequent decision unit, forming a closed-loop optimization chain for sequence decision-making. Specifically, the encoder first receives the observation sequence in any order. The observation information is input into multiple computational modules for encoding. These modules include a self-attention layer, an MLP layer, and residual connections. The encoded observations... This includes both the field-of-view information of the base stations and the interference relationships between the base stations.

[0099] While traditional Transformers can learn observation information and calculate allocation strategies through attention mechanisms and neural network characteristics, they are not optimized for multi-agent collaborative scenarios. The MAT framework, on the other hand, sets a masking mechanism for the self-attention layer of the decoder, which can accurately calculate the action advantage function of each resource supply node (agent) under sequential decision-making conditions. The action advantage functions of each node are accumulated and used for system reward calculation. This ensures the independence and specificity of individual node decisions, while also achieving collaborative optimization among multiple nodes. It perfectly matches the core requirement of parallel decision-making by multiple resource supply nodes in wireless resource scheduling while taking into account overall performance.

[0100] In one embodiment of this application, the training process of the multi-agent model is as follows:

[0101] The multi-agent model is initialized using the initial model parameters generated by supervised learning pre-training;

[0102] In each training iteration, environmental coding data is generated based on the encoder's observation information of each resource supply node and the decision strategy encoding of the preceding decision unit. The decoder generates and executes the decision strategy of each resource supply node according to the decision sequence. The optimal decision strategy is determined by combining the local search algorithm. The observation information, the optimal decision strategy and the corresponding local reward are recorded and stored in the experience cache pool.

[0103] Batch data is sampled from the experience buffer pool, and the local advantage of each resource supply node is calculated based on the actions of the preceding decision-making units, its own decision-making actions, and local reward feedback observed by each resource supply node. The local advantage reflects the contribution of the corresponding resource supply node's own strategy to the overall network transmission efficiency.

[0104] Determine the new decision strategy and the old decision strategy for each resource supply node, wherein the new decision strategy is the decision strategy generated by the decoder in the current iteration, and the old decision strategy is the historical decision strategy that has been executed and verified in the previous iteration;

[0105] Calculate the ratio of the action probability of the new decision-making strategy to that of the old decision-making strategy in the same decision-making scenario;

[0106] Based on the local dominance function and the action probability ratio, the value parameters of the encoder and the decision parameters of the decoder are iteratively optimized by limiting the policy update magnitude through a preset algorithm.

[0107] Optionally, initial model parameters are generated through supervised learning pre-training. These parameters are then used to initialize the multi-agent model. In each training iteration, the model encoder first jointly encodes the local observation information of each resource supply node and the decision-making strategy of the preceding decision-making unit to generate environmental encoded data containing the local vision of the node and the interference relationship between nodes. Then, the decoder generates corresponding decision-making strategies for each resource supply node according to a preset decision sequence and simulates their execution. At the same time, the generated initial decision-making strategy is refined and optimized by combining a local search algorithm to select the optimal decision-making strategy that is suitable for the current training scenario. Subsequently, the observation information, optimal decision-making strategy and corresponding local reward (obtained based on the quantification of local network transmission efficiency) in this training round are uniformly recorded and stored in the experience cache pool to form training data reserves.

[0108] Subsequently, batch training data is randomly sampled from the experience cache pool. Based on the actions of the preceding decision units observed by each resource supply node, the decision actions executed by itself, and the corresponding local reward feedback, the local advantage of each resource supply node is obtained through the advantage function calculation method. This local advantage can accurately quantify the contribution of the decision strategy of a single resource supply node to the overall transmission efficiency of the network. Then, the new decision strategy and the old decision strategy of each resource supply node are defined. The new decision strategy is the decision strategy generated by the decoder in the current iteration, and the old decision strategy is the historical decision strategy that has been executed and verified to be effective in the previous iteration. By calculating the ratio of the action probability of the two in the same decision scenario, the magnitude and direction of the strategy update are measured.

[0109] Finally, using local advantage and action probability ratio as core inputs, the Clipping PPO near-end policy optimization algorithm is employed to limit the step size of policy updates, avoiding parameter oscillations or gradient explosion during training. Through multiple iterations, the decision parameters of the decoder are continuously optimized, ultimately obtaining the optimal model parameters that balance decision independence and global synergy. This ensures that the model can adapt to the sequential decision-making scenario of wireless resource scheduling and outputs a decision policy that maximizes the overall transmission efficiency of the network.

[0110] During the training phase, observations can be calculated. Corresponding value function The loss function of the encoder, calculated by minimizing the Bellman error, can be specifically expressed as:

[0111] ;

[0112] in, The value function represents the value at time step t. The value function represents the value at time step t+1. This represents the reward discount factor. Indicates observation as Execute actions The immediate reward received (system throughput). represents the normalization coefficient, T is the time step of the training sequence, and n represents the number of agents.

[0113] The optimization process of the clipping PPO near-end policy optimization algorithm can be represented as follows: ;

[0114] ;

[0115] in, This represents the ratio of the probability of the new action to the probability of the old action. This represents the estimate of the dominance function at time step t. This represents the cutoff factor for PPO. This indicates the clipping operation (the ratio of this strategy) Limited to (within the range) Represents the trainable parameters of the decoder, and represents the normalization coefficients. Indicating a new strategy, This indicates the old strategy.

[0116] In one embodiment of this application, before initializing the multi-agent model with the initial model parameters generated by supervised learning pre-training, the method further includes:

[0117] For all the resource supply nodes, communication users and schedulable resources used for training, calculate the first signal-to-interference ratio without considering interference between users.

[0118] The first signal interference ratios are arranged to form a ternary combination matching priority list;

[0119] Based on the user association constraints and resource allocation constraints, the connection and allocation relationships corresponding to the highest priority ternary combination are temporarily activated in sequence. If the total signal transmission rate increases after activation, the current decision variable is overwritten with the old decision variable and the association matrix between the communication user and the resource supply node is updated. Otherwise, the process reverts to the temporary activation operation.

[0120] Based on the activation results, determine the joint action corresponding to the schedulable resource allocation strategy, and calculate the second signal-to-interference ratio considering inter-user interference for each activated ternary combination.

[0121] Based on the second signal interference ratio and the preset transmission bandwidth, calculate the current total signal transmission rate;

[0122] The joint action corresponding to the schedulable resource allocation strategy, the current observation information, and the reward corresponding to the current total signal transmission rate are stored together in the supervised learning dataset to train the initial model based on the supervised learning dataset and obtain the initial model parameters.

[0123] Optionally, a heuristic (Max_SINR) baseline algorithm is used to supervise learning the initial dataset for all base stations. ,user Resource blocks The set of data is optimized using a greedy algorithm: traversing potential link-RB combinations in descending order to greedily optimize the overall system rate. Associations are activated only when adding new associations strictly improves the overall system rate. Further optimization is then performed through neighborhood search, ultimately generating a pre-training set. The pre-training set is imported into a simulation environment to initialize a multi-agent network. In each iteration, pre-training set data is sampled, the advantage function and reward are calculated, and the policy update magnitude is limited. Through multiple iterations, the model parameters are optimized, thus solving the Wireless Resource Allocation and User Association (UARA) optimization problem.

[0124] Specifically, environmental parameters are initialized, and the first signal-to-interference ratio (SINR) of all resource supply nodes-communication users-schedulable resources is pre-calculated, ignoring interference from other users, and sorted in descending order of value. Simultaneously, a user-base station association matrix z, a resource block allocation matrix a, and the initial system throughput are created, and decision variables are initialized. Then, for all base stations... ,user Resource blocks The set of [variables]. The first signal-to-interference ratio can be expressed as:

[0125] ;

[0126] in, For base stations The transmission power, For base stations ,user Resource blocks Channel gain of triplet This represents the noise power in the environment. Finally, the SINR values ​​are obtained in descending order through iteration.

[0127] For all triples, a greedy matching process is performed. Each time, a set of associations for base station B, user U, and resource block K is selected from a table of examples sorted in descending order of SINR values. This set is activated only if adding a new association strictly increases the overall system rate. After satisfying the user association constraint and resource allocation constraint, the current user-base station-resource block is temporarily activated. If the activated system rate increases, the current decision variable overwrites the old variable, and the user-base station association matrix is ​​updated; otherwise, the modification is rolled back.

[0128] After obtaining the greedy matching result, for each active user-base station-resource block combination, the second signal-to-interference ratio (SINR) considering inter-user interference is calculated, which can be expressed as:

[0129] ;

[0130] The final calculated total system speed is:

[0131] ;

[0132] Running the joint action generated by MAX_SINR With system rewards It will be stored in the supervised learning dataset. The specific implementation process of the MAX_SINR algorithm is shown in Table 1 below:

[0133] Table 1: Detailed Implementation Process of the MAX_SINR Algorithm

[0134]

[0135] It should be noted that neural network models can be used when supervising learning on the dataset. Given the one-to-many constraint relationship between users and resource blocks (RBs) and base stations and users (i.e., a single user can be associated with multiple resource blocks and a single base station can be associated with multiple users, but a single resource block is only assigned to a single user and a single user is only associated with a single base station), a loss function suitable for multi-label classification problems should be selected. Specifically, the BCEWithLogitsLoss() function, which combines the Sigmoid activation function and binary cross-entropy, can be used. This function can effectively handle the association constraints between labels, accurately measure the difference between the model's prediction results and the true labels, and improve the training effect and model convergence accuracy of supervised learning.

[0136] In one embodiment of this application, the step of determining the optimal decision strategy by combining a local search algorithm includes:

[0137] The initial self-decision strategy of each resource supply node generated by the decoder is used as the baseline strategy. The user connection operation and resource allocation operation corresponding to the baseline strategy are executed, and the corresponding system reward is recorded as the baseline reward.

[0138] For the user connection vector in the baseline strategy, a neighborhood user association scheme is constructed by adjusting the values ​​of the components in the user connection vector;

[0139] Based on the neighborhood user association scheme, a corresponding schedulable resource allocation vector is generated. The user connection vector corresponding to the neighborhood user association scheme is concatenated with the schedulable resource allocation vector to form a new action sequence.

[0140] Execute the user connection operation and resource allocation operation corresponding to the new action sequence, and calculate the system reward corresponding to the new action sequence;

[0141] If the system reward is greater than the baseline reward, then the new action sequence is determined as the current optimal decision strategy;

[0142] If the system reward is not greater than the baseline reward, then the baseline strategy is retained as the current optimal decision strategy.

[0143] Optionally, a neighborhood search algorithm (MAT+Local Search) is combined with the multi-agent model to realize the batch generation of base station observation codes and action allocation strategies. Based on the multi-agent Transformer (MAT) model framework, the encoder batch encodes the local observation information of each base station (resource supply node), the decision strategies of preceding decision units, and the interference relationships between nodes to generate global environment coding data. Then, the decoder, configured with a masked self-attention layer and a masked cross-attention layer, outputs an initial action allocation strategy (including user association schemes and resource allocation vectors) for each base station according to a preset decision sequence. Based on this, a local search algorithm is introduced to analyze the action space of the base stations. The algorithm explores the neighborhood (e.g., fine-tuning user connection vectors, resource allocation vectors, etc.), and selects the optimal decision strategy to further improve the cumulative reward of the system by comparing the system rewards of the new action sequence with those of the baseline strategy. At the same time, the algorithm can iteratively update the model parameters by minimizing the loss functions of the encoder and decoder.

[0144] Specifically, the number of pre-set resource supply nodes Number of communication users U, number of schedulable resources K, bandwidth BW, environmental noise Parameters for the scenario, and for each resource supply node With communication users Calculate the Euclidean distance between the resource supply node and the communication user:

[0145] ;

[0146] in, Indicates resource supply nodes x-coordinate Indicates resource supply nodes The ordinate, Indicates communication user x-coordinate Indicates communication user The ordinate.

[0147] This information is then added to the distance matrix. When the distance between the resource provider node and the communication user is less than the coverage area of ​​the base station, the communication user is added to the observation list of the resource provider node, indicating that the communication user can be associated with the resource provider node.

[0148] Next, an action space is constructed for each resource supply node. This includes User Association (UA) and Scheduled Resource Allocation (RBG). The former is the probability of association between a resource provider node and each user, while the latter is the probability of allocating a scheduled resource to the user-user association. Each action component is a continuous real number in the interval [0,1], and can be represented as:

[0149] ;

[0150] ;

[0151] Before each round of environmental interaction, the allocation of UA and RBG for resource supply nodes can be modified to ensure that the constraints in the UARA problem are met.

[0152] During the training phase: The encoder first processes the observation information from each base station. Encode the observation information. The two self-attention layers that are input to the decoder are: the first layer is a masked attention layer, which uses a masking mechanism to shield the subsequent nodes of each resource supply node and only calculates the correlation and interference degree between the current resource supply node and the preceding decision unit; the second layer is a cross-attention layer, which uses the encoded observation information. The correlation between quantified node decisions and global environment observation is established. The decoder performs differential masking on each node according to the decision sequence, sequentially driving each node to complete user association and resource block allocation operations, and adding the current node's decision action to the preceding action sequence. This provides a reference for decision-making at the next stage.

[0153] In action sequence Perform a local search: based on the current action Using this as a baseline, record the system rewards, then select users within the observation range for resource block reallocation, and assign the association vectors of the target users. Set the value from 0 to 1 to construct a new action. and Then, a new action vector is concatenated and the reward is evaluated. ;

[0154] If reward Greater than the original baseline reward If the optimal action is updated, the search can be terminated early. Using Local Search allows MAT to search for a better solution in the neighborhood of the target assignment policy, thereby optimizing system throughput.

[0155] After the allocation strategy is finally determined, the current observations will be... ,action ,award The data is stored in the experience buffer for subsequent training sampling. During each training round, the system samples a batch of observations from the buffer. ,action ,award Triplet, using an encoder to process observations Encode and calculate the value function And calculate the encoder loss value. .

[0156] Calculate the loss value of the decoder need Corresponding joint advantage function This step uses the generalized advantage estimator (GAE) to derive the corresponding joint advantage function A, and then encodes the observations. With action set The input decoder obtains the action allocation strategy for each base station. Then, the decoder loss is calculated.

[0157] After obtaining the loss functions for the encoder and decoder, the system updates the encoder parameters using gradient descent. With decoder parameters To minimize the total loss function: This achieves optimized convergence of model parameters.

[0158] During the inference phase: the system locks the encoder parameters. With decoder parameters Reacquire observation information from each node And then encode it. The decoder uses the decision strategy obtained during training. The decision sequence is autoregressively decoded to generate the corresponding communication user and schedulable resource block allocation action for each node. Simultaneously calculate the local throughput corresponding to the actions of each base station. The total system throughput is obtained by summing the local throughput of all base stations. ,Right now:

[0159] ;

[0160] It should be noted that the specific implementation process of the MAT+Local Search algorithm is shown in Table 2 below:

[0161] Table 2. Detailed implementation process of the MAT+Local Search algorithm:

[0162]

[0163] In addition, the parameters used in the MAT training environment are shown in Table 3 below:

[0164] Table 3: Parameters used in the MAT training environment:

[0165]

[0166] The MAT+Local Search algorithm converged after 160,000 steps (approximately 6,400 episodes) in a 10 Mbps bandwidth, 7 base station environment (as shown in Figure 3), with a single-step output system throughput of 3160.37 Mbps. The traditional heuristic Max_SINR algorithm yielded 2038.92 Mbps, representing a performance improvement of 55.00%, as shown in Figure 4.

[0167] In this embodiment, by configuring each resource supply node as an independent intelligent decision-making unit to perform user connection and resource allocation operations sequentially, the exponential solution space of the traditional UARA problem is transformed into a linear decision-making process, significantly reducing computational complexity to adapt to large-scale heterogeneous network scenarios. Simultaneously, a target optimization function is constructed using preset user association constraints and resource allocation constraints as boundaries. Combined with the natural satisfaction of constraints by subsequent decision units' reference to previous results, this solves the problem of fuzzy constraint handling in existing reinforcement learning methods. A local search algorithm refines the initial policy through neighborhood refinement to discover better solutions, and iteratively optimizes parameters using the overall network transmission performance as a reward signal. This overcomes the suboptimal nature of traditional heuristic methods and the instability of reinforcement learning training. Furthermore, it does not rely on complete wireless environment knowledge, can dynamically adapt to network changes, and ultimately maximizes network transmission performance within the constraint boundaries. It stably outputs a scheduling scheme that balances real-time performance, feasibility, and optimality, significantly improving the service quality of large-scale heterogeneous networks.

[0168] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0169] In one embodiment, a wireless resource scheduling system is provided, which corresponds one-to-one with the wireless resource scheduling methods described in the above embodiments. As shown in Figure 5, the wireless resource scheduling system includes a data acquisition unit 10, a constraint construction unit 20, an objective optimization function construction unit 30, and an objective optimization function solving unit 40. Detailed descriptions of each functional module are as follows:

[0170] Data acquisition unit 10 is used to acquire resource supply nodes, communication users and schedulable resources in the wireless communication network;

[0171] Constraint construction unit 20 is used to construct user-related constraints and resource allocation constraints based on network operation requirements, resource supply nodes, communication users and schedulable resources.

[0172] The objective optimization function construction unit 30 is used to construct an objective optimization function with the user association constraints and resource allocation constraints as boundary conditions, aiming to maximize the overall network transmission efficiency.

[0173] The objective optimization function solving unit 40 is used to configure each resource supply node as a decision unit. Each decision unit executes user connection operations and resource allocation operations in sequence according to a preset decision sequence, and generates the corresponding local network transmission efficiency based on the execution results. The subsequent decision unit updates its own decision parameters as a reward signal based on the local network transmission efficiency corresponding to the preceding resource supply node and then executes the corresponding operation. Based on the execution results of all resource supply nodes, the objective optimization function is solved to obtain the maximum overall network transmission efficiency under the constraint boundary.

[0174] In one embodiment of this application, the objective optimization function construction unit 30 is further configured to:

[0175] Based on the transmission power of the communication link, the channel transmission gain, and the environmental noise level, calculate the signal-to-interference ratio of each communication user on each potential allocated resource;

[0176] Based on the signal-to-interference ratio and the preset channel gain coefficient, the signal transmission rate of each communication user on the corresponding potential allocated resources is calculated.

[0177] Using the user association constraints and resource allocation constraints as boundary conditions, and combining the preset transmission bandwidth and the signal transmission rate, a target optimization function is constructed with the goal of maximizing the overall network transmission efficiency.

[0178] The user association constraint limits a single communication user to establish a connection with only one resource supply node and the maximum number of connected users of a single resource supply node does not exceed a preset threshold; the resource allocation constraint limits a single schedulable resource to be allocated only to a single communication user and the maximum amount of resources a single communication user can acquire does not exceed a preset threshold.

[0179] In one embodiment of this application, the objective optimization function construction unit 30 is further configured to:

[0180] Determine the spatial distance between each resource supply node and each communication user, the carrier frequency for signal transmission, and the altitude parameters of the communication users;

[0181] Based on the numerical range of the spatial distance, the propagation scenario type is determined;

[0182] A standardized path loss model is used to determine the attenuation calculation rules matching the propagation scenario type, and the signal attenuation is determined based on the attenuation calculation rules, spatial distance, carrier frequency, and the altitude parameter of the communication user.

[0183] The transmit power, initial channel transmission gain, and ambient noise level parameters of the communication link are obtained, and the initial channel transmission gain is calibrated in conjunction with the signal attenuation to obtain the effective signal power.

[0184] The signal-to-interference ratio of a communication user under the corresponding potential allocation resources and corresponding resource supply nodes is calculated by using the ratio of the effective signal power to the interference plus noise power.

[0185] In one embodiment of this application, the propagation scenario type includes line-of-sight propagation scenarios and non-line-of-sight propagation scenarios. The line-of-sight propagation scenario refers to a spatial distance within a short-range interval, and the non-line-of-sight propagation scenario refers to a spatial distance within a long-range interval. The objective optimization function construction unit 30 is further configured to:

[0186] In the case of line-of-sight propagation, the signal attenuation is calculated based on the spatial distance and carrier frequency using the attenuation calculation rules corresponding to the line-of-sight scenario.

[0187] In the case of non-line-of-sight propagation scenarios, the signal attenuation is calculated based on the three-dimensional spatial distance, carrier frequency, and the difference between the actual height of the terminal and the reference height, using the attenuation calculation rules corresponding to non-line-of-sight scenarios.

[0188] In one embodiment of this application, the objective optimization function solving unit 40 is further configured to:

[0189] According to the preset sorting rules, the execution order of each resource supply node is determined, forming a decision sequence;

[0190] According to the order of each resource supply node in the decision sequence, user connection operations and resource allocation operations are executed sequentially based on their own decision strategies. When the subsequent decision unit executes a decision, it observes the decision strategies of the preceding resource supply nodes and the corresponding local network transmission performance, and combines the environmental coding data output by the multi-agent model to generate its current decision strategy, so as to execute user connection operations and resource allocation operations based on the current decision strategy.

[0191] In one embodiment of this application, the objective optimization function solving unit 40 is further configured to: deploy the multi-agent model, the multi-agent model including an encoder and a decoder, wherein the decoder is configured with a masked self-attention layer and a masked cross-attention layer;

[0192] The encoder encodes the local observation information of each resource supply node and the decision-making strategy of the preceding resource supply node to generate environmental coded data containing the local vision information of the resource supply node and the interference relationship between nodes.

[0193] The decoder uses a masked self-attention layer to associate the preceding resource supply node's decision-making and execution actions, and a masked cross-attention layer to fuse the environmental encoded data, thereby generating the current self-decision strategy corresponding to the subsequent decision unit.

[0194] Based on its current decision-making strategy, the system performs user connection operations and resource allocation operations, and records the current decision-making strategy and the corresponding local network transmission performance for observation by the next-level subsequent decision-making unit.

[0195] In one embodiment of this application, the system further includes a model training unit, used for:

[0196] The multi-agent model is initialized using the initial model parameters generated by supervised learning pre-training;

[0197] In each training iteration, environmental coding data is generated based on the encoder's observation information of each resource supply node and the decision strategy encoding of the preceding decision unit. The decoder generates and executes the decision strategy of each resource supply node according to the decision sequence. The optimal decision strategy is determined by combining the local search algorithm. The observation information, the optimal decision strategy and the corresponding local reward are recorded and stored in the experience cache pool.

[0198] Batch data is sampled from the experience buffer pool, and the local advantage of each resource supply node is calculated based on the actions of the preceding decision-making units, its own decision-making actions, and local reward feedback observed by each resource supply node. The local advantage reflects the contribution of the corresponding resource supply node's own strategy to the overall network transmission efficiency.

[0199] Determine the new decision strategy and the old decision strategy for each resource supply node, wherein the new decision strategy is the decision strategy generated by the decoder in the current iteration, and the old decision strategy is the historical decision strategy that has been executed and verified in the previous iteration;

[0200] Calculate the ratio of the action probability of the new decision-making strategy to that of the old decision-making strategy in the same decision-making scenario;

[0201] Based on the local dominance function and the action probability ratio, the value parameters of the encoder and the decision parameters of the decoder are iteratively optimized by limiting the policy update magnitude through a preset algorithm.

[0202] In one embodiment of this application, the model training unit is further configured to:

[0203] For all the resource supply nodes, communication users and schedulable resources used for training, calculate the first signal-to-interference ratio without considering interference between users.

[0204] The first signal interference ratios are arranged to form a ternary combination matching priority list;

[0205] Based on the user association constraints and resource allocation constraints, the connection and allocation relationships corresponding to the highest priority ternary combination are temporarily activated in sequence. If the total signal transmission rate increases after activation, the current decision variable is overwritten with the old decision variable and the association matrix between the communication user and the resource supply node is updated. Otherwise, the process reverts to the temporary activation operation.

[0206] Based on the activation results, determine the joint action corresponding to the schedulable resource allocation strategy, and calculate the second signal-to-interference ratio considering inter-user interference for each activated ternary combination.

[0207] Based on the second signal interference ratio and the preset transmission bandwidth, calculate the current total signal transmission rate;

[0208] The joint action corresponding to the schedulable resource allocation strategy, the current observation information, and the reward corresponding to the current total signal transmission rate are stored together in the supervised learning dataset to train the initial model based on the supervised learning dataset and obtain the initial model parameters.

[0209] In one embodiment of this application, the model training unit is further configured to:

[0210] The initial self-decision strategy of each resource supply node generated by the decoder is used as the baseline strategy. The user connection operation and resource allocation operation corresponding to the baseline strategy are executed, and the corresponding system reward is recorded as the baseline reward.

[0211] For the user connection vector in the baseline strategy, a neighborhood user association scheme is constructed by adjusting the values ​​of the components in the user connection vector;

[0212] Based on the neighborhood user association scheme, a corresponding schedulable resource allocation vector is generated. The user connection vector corresponding to the neighborhood user association scheme is concatenated with the schedulable resource allocation vector to form a new action sequence.

[0213] Execute the user connection operation and resource allocation operation corresponding to the new action sequence, and calculate the system reward corresponding to the new action sequence;

[0214] If the system reward is greater than the baseline reward, then the new action sequence is determined as the current optimal decision strategy;

[0215] If the system reward is not greater than the baseline reward, then the baseline strategy is retained as the current optimal decision strategy.

[0216] In this embodiment, by configuring each resource supply node as an independent intelligent decision-making unit to perform user connection and resource allocation operations sequentially, the exponential solution space of the traditional UARA problem is transformed into a linear decision-making process, significantly reducing computational complexity to adapt to large-scale heterogeneous network scenarios. Simultaneously, a target optimization function is constructed using preset user association constraints and resource allocation constraints as boundaries. Combined with the natural satisfaction of constraints by subsequent decision units' reference to previous results, this solves the problem of fuzzy constraint handling in existing reinforcement learning methods. A local search algorithm refines the initial policy through neighborhood refinement to discover better solutions, and iteratively optimizes parameters using the overall network transmission performance as a reward signal. This overcomes the suboptimal nature of traditional heuristic methods and the instability of reinforcement learning training. Furthermore, it does not rely on complete wireless environment knowledge, can dynamically adapt to network changes, and ultimately maximizes network transmission performance within the constraint boundaries. It stably outputs a scheduling scheme that balances real-time performance, feasibility, and optimality, significantly improving the service quality of large-scale heterogeneous networks.

[0217] Specific limitations regarding the wireless resource scheduling system can be found in the limitations of the wireless resource scheduling method described above, and will not be repeated here. Each module in the aforementioned wireless resource scheduling system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can call and execute the corresponding operations of each module.

[0218] In one embodiment, a computer device is provided, which may be a terminal device, and its internal structure diagram is shown in Figure 6. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a readable storage medium. The readable storage medium stores computer-readable instructions. The network interface of the computer device is used to communicate with external terminals via a network connection. When the computer-readable instructions are executed by the processor, they implement a wireless resource scheduling method. The readable storage medium provided in this embodiment includes both non-volatile readable storage media and volatile readable storage media.

[0219] In this application embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, it implements the steps of the wireless resource scheduling method described above.

[0220] In one embodiment of the application, a readable storage medium is provided, which stores computer-readable instructions. When the computer-readable instructions are executed by a processor, they implement the steps of the wireless resource scheduling method described above.

[0221] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0222] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0223] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A wireless resource scheduling method, characterized in that, The method includes: acquiring resource supply nodes, communication users, and schedulable resources in a wireless communication network; constructing user association constraints and resource allocation constraints based on network operation requirements, resource supply nodes, communication users, and schedulable resources; constructing a target optimization function with the user association constraints and resource allocation constraints as boundary conditions, aiming to maximize the overall network transmission efficiency; configuring each resource supply node as a decision unit, and each decision unit sequentially executing user connection operations and resource allocation operations according to a preset decision sequence, and generating corresponding local network transmission efficiency based on the execution results, wherein subsequent decision units update their respective decision parameters as reward signals using the local network transmission efficiency corresponding to the preceding resource supply node before executing corresponding operations; and solving the target optimization function based on the execution results of all resource supply nodes to obtain the overall network transmission efficiency maximized under the boundary conditions.

2. The wireless resource scheduling method as described in claim 1, characterized in that, The step of constructing a target optimization function with the user association constraint and resource allocation constraint as boundary conditions to maximize the overall network transmission performance includes: calculating the signal-to-interference ratio (SIR) of each communication user on each potential allocated resource based on the transmission power of the communication link, channel transmission gain, and environmental noise level; calculating the signal transmission rate of each communication user on the corresponding potential allocated resource based on the SIR and a preset channel gain coefficient; and constructing the target optimization function with the user association constraint and resource allocation constraint as boundary conditions, combined with a preset transmission bandwidth and the signal transmission rate. The user association constraint limits a single communication user to only establish a connection with a single resource supply node, and the maximum number of connected users on a single resource supply node does not exceed a preset threshold. The resource allocation constraint limits a single schedulable resource to only be allocated to a single communication user, and the maximum resource acquisition amount for a single communication user does not exceed a preset threshold.

3. The wireless resource scheduling method as described in claim 2, characterized in that, The calculation of the signal-to-interference ratio (SIR) for each communication user on each potential allocation resource, based on the transmission power of the communication link, channel transmission gain, and ambient noise level, includes: determining the spatial distance between each resource supply node and each communication user, the carrier frequency of signal transmission, and the altitude parameter of the communication user; determining the propagation scenario type based on the numerical range of the spatial distance; determining the attenuation calculation rule matching the propagation scenario type using a standardized path loss model, and determining the signal attenuation amount based on the attenuation calculation rule, spatial distance, carrier frequency, and the altitude parameter of the communication user; obtaining the transmission power, initial channel transmission gain, and ambient noise level parameters of the communication link, calibrating the initial channel transmission gain in conjunction with the signal attenuation amount to obtain the effective signal power; and calculating the SIR for the communication user under the corresponding potential allocation resource and the corresponding resource supply node by using the ratio of the effective signal power to the interference plus noise power.

4. The wireless resource scheduling method as described in claim 3, characterized in that, The propagation scenario types include line-of-sight (LAS) propagation scenarios and non-line-of-sight (NOS) propagation scenarios. A LAS propagation scenario refers to a spatial distance within the short-range range, while a NOS propagation scenario refers to a spatial distance within the long-range range. The determination of signal attenuation based on the attenuation calculation rules, spatial distance, carrier frequency, and the communication user's height parameter includes: for a LAS propagation scenario, calculating the signal attenuation based on the spatial distance and carrier frequency using the attenuation calculation rules corresponding to the LAS scenario; for a NOS propagation scenario, calculating the signal attenuation based on the spatial distance, carrier frequency, and the difference between the terminal's actual height and a reference height using the attenuation calculation rules corresponding to the NOS scenario.

5. The wireless resource scheduling method as described in claim 1, characterized in that, The step of configuring each resource supply node as a decision unit, with each decision unit sequentially executing user connection operations and resource allocation operations according to a preset decision sequence, includes: determining the execution order of each resource supply node according to a preset sorting rule to form a decision sequence; and sequentially executing user connection operations and resource allocation operations based on its own decision strategy according to the sorting of each resource supply node in the decision sequence. Specifically, when a subsequent decision unit executes a decision, it observes the decision strategy of the preceding resource supply node and the corresponding local network transmission performance, and combines this with the environmental coding data output by the multi-agent model to generate its current decision strategy, thereby executing user connection operations and resource allocation operations based on the current decision strategy.

6. The wireless resource scheduling method as described in claim 5, characterized in that, When executing a decision, the subsequent decision unit observes the decision strategies of the preceding resource supply nodes and their corresponding local network transmission performance, and combines this with environmental coding data to generate its own current decision strategy. Based on this current decision strategy, it executes user connection operations and resource allocation operations. This includes: deploying the multi-agent model, which includes an encoder and a decoder, wherein the decoder is configured with a masked self-attention layer and a masked cross-attention layer; encoding the local observation information of each resource supply node and the decision strategies of the preceding resource supply nodes through the encoder to generate environmental coding data containing the local vision information of the resource supply nodes and the interference relationships between nodes; associating the decision execution actions of the preceding resource supply nodes through the masked self-attention layer of the decoder and fusing the environmental coding data through the masked cross-attention layer to generate the current decision strategy corresponding to the subsequent decision unit; executing user connection operations and resource allocation operations based on the current decision strategy, and recording the current decision strategy and its corresponding local network transmission performance for observation by the next-level subsequent decision unit.

7. The wireless resource scheduling method as described in claim 6, characterized in that, The training process of the multi-agent model is as follows: the multi-agent model is initialized using the initial model parameters generated by supervised learning pre-training; in each training iteration, environmental encoding data is generated based on the observation information of each resource supply node and the decision strategy encoding of the preceding decision unit by the encoder; the decision strategy of each resource supply node is generated and executed by the decoder according to the decision sequence; the optimal decision strategy is determined by combining the local search algorithm; the observation information, the optimal decision strategy and the corresponding local reward are recorded and stored in the experience cache pool. Batch data is sampled from the experience cache pool. Based on the actions of the preceding decision units, the decision actions of each resource supply node, and the local reward feedback observed by each resource supply node, the local advantage corresponding to each resource supply node is calculated. The local advantage reflects the contribution of the corresponding resource supply node's own strategy to the overall network transmission efficiency. The new decision strategy and the old decision strategy of each resource supply node are determined. The new decision strategy is the decision strategy generated by the decoder in the current iteration, and the old decision strategy is the historical decision strategy that has been executed and verified in the previous iteration. The action probability ratio of the new decision strategy and the old decision strategy under the same decision scenario is calculated. Based on the local advantage function and the action probability ratio, the policy update range is limited by a preset algorithm, and the value parameters of the encoder and the decision parameters of the decoder are iteratively optimized.

8. The wireless resource scheduling method as described in claim 7, characterized in that, Before initializing the multi-agent model using the initial model parameters generated by supervised learning pre-training, the method further includes: calculating a first signal-to-interference ratio (SIR) without considering inter-user interference for all tripartite combinations consisting of resource supply nodes, communication users, and schedulable resources used for training; arranging the first SIR to form a tripartite combination matching priority list; based on the user association constraints and resource allocation constraints, temporarily activating the connection and allocation relationships corresponding to the tripartite combination with the highest current priority in sequence; if the total signal transmission rate increases after activation, the current decision variable overwrites the old decision variable and updates the association matrix of the communication user and resource supply node; otherwise, reverting to the temporary activation operation; determining the joint action corresponding to the schedulable resource allocation strategy based on the activation result, and calculating a second SIR considering inter-user interference for each activated tripartite combination; calculating the current total signal transmission rate based on the second SIR and a preset transmission bandwidth; storing the joint action corresponding to the schedulable resource allocation strategy, the current observation information, and the reward corresponding to the current total signal transmission rate together in a supervised learning dataset to train the initial model based on the supervised learning dataset and obtain the initial model parameters.

9. The wireless resource scheduling method as described in claim 7, characterized in that, The step of determining the optimal decision strategy using a local search algorithm includes: taking the initial self-decision strategy of each resource supply node generated by the decoder as a baseline strategy; executing the user connection operation and resource allocation operation corresponding to the baseline strategy; and recording the corresponding system reward as the baseline reward; constructing a neighborhood user association scheme by adjusting the values ​​of the components in the user connection vector in the baseline strategy; generating a corresponding schedulable resource allocation vector based on the neighborhood user association scheme; concatenating the user connection vector corresponding to the neighborhood user association scheme with the schedulable resource allocation vector to form a new action sequence; executing the user connection operation and resource allocation operation corresponding to the new action sequence; calculating the system reward corresponding to the new action sequence; if the system reward is greater than the baseline reward, then the new action sequence is determined as the current optimal decision strategy; if the system reward is not greater than the baseline reward, then the baseline strategy is retained as the current optimal decision strategy.

10. A wireless resource scheduling system, characterized in that, The system includes: a data acquisition unit for acquiring resource supply nodes, communication users, and schedulable resources in a wireless communication network; a constraint construction unit for constructing user association constraints and resource allocation constraints based on network operation requirements, resource supply nodes, communication users, and schedulable resources; a target optimization function construction unit for constructing a target optimization function with the user association constraints and resource allocation constraints as boundary conditions, aiming to maximize the overall network transmission efficiency; and a target optimization function solving unit for configuring each resource supply node as a decision unit, with each decision unit sequentially executing user connection operations and resource allocation operations according to a preset decision sequence, and generating corresponding local network transmission efficiency based on the execution results. Each subsequent decision unit updates its decision parameters using the local network transmission efficiency corresponding to the preceding resource supply node as a reward signal before executing its corresponding operation. Based on the execution results of all resource supply nodes, the target optimization function is solved to obtain the maximum overall network transmission efficiency under the boundary conditions.

Images