Resource allocation method and system of cellular-free large-scale system, and storage medium

Abstract

The invention discloses a resource allocation method and system for a non-cellular large-scale system, and a storage medium. The method comprises the following steps: constructing a non-orthogonal multiple access-based non-cellular large-scale MIMO system model; determining a downlink data transmission rate model of the cellular-free large-scale MIMO-NOMA system; determining an optimization problem model based on the downlink data transmission rate model, wherein the objective of the optimization problem model is to maximize the rate of the cellular-free large-scale MIMO-NOMA system; based on the optimization problem model, obtaining a layered depth deterministic strategy gradient framework; according to a hierarchical depth deterministic strategy gradient framework, the networks are iteratively and mutually updated to obtain an optimal strategy; and carrying out resource allocation on the non-cellular large-scale MIMO-NOMA system based on the optimal strategy. By adopting the technical scheme provided by the invention, joint optimization is carried out on user pairing and power distribution in the cellular-free large-scale MIMO-NOMA system.

Description

Technical Field

[0001] The present invention belongs to the field of wireless communication technology, and in particular relates to a resource allocation method and system, and a storage medium for a non-cellular large-scale MIMO-NOMA system. Background Art

[0002] Mobile communication is one of the most widely used fields of wireless communication technology. It contains rich technical and social backgrounds, and the research on mobile communication is also deepening and expanding with the progress of the times.

[0003] With the continuous development of mobile communication technology, massive MIMO, as an important communication technology, has been widely used in modern communication systems. Compared with traditional small-scale MIMO systems, massive MIMO systems have shown obvious advantages in many aspects. It has higher spectrum efficiency, wider coverage and lower interference level. This enables massive MIMO systems to effectively meet the needs of future mobile communication systems for high speed, low latency and massive number of connections.

[0004] As an important form of massive MIMO technology, the advantages of non-cellular massive MIMO system are simplicity, flexibility, low cost, etc. It has broad application prospects in the future 5G and subsequent communication networks.

[0005] As an important technology to improve spectrum efficiency, Non-Orthogonal Multiple Access (NOMA) has attracted much attention in academia and industry. Its core idea is to use multi-user channel gain to superimpose multi-user data signals in the power domain to provide services to multiple users in the same spectrum block. Compared with orthogonal multiple access, NOMA can support more users on limited spectrum resources, but it will introduce intra-cluster interference. This interference can be offset by continuous interference cancellation at the receiving end.

[0006] Recent studies have shown that it is crucial to combine NOMA technology with cell-free massive MIMO systems to fully utilize the advantages of both and meet the conflicting requirements of future wireless networks for high spectral efficiency, massive connections with low latency, high reliability, and user fairness. However, in related technologies, there is little research on the joint optimization of user pairing and power allocation in cell-free massive MIMO-NOMA systems, and the application of DRL as a tool for optimization is relatively insufficient. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a resource allocation method and system and a storage medium for a non-cellular large-scale MIMO-NOMA system. Aiming at the user pairing and power allocation problems in the downlink of the system, a hierarchical deep deterministic policy gradient (H-DDPG) framework that can jointly optimize the two problems is proposed; the framework adopts a two-layer control network running on different time scales, and by collaboratively optimizing user pairing and power allocation, the system rate is improved while satisfying the corresponding user pairing constraints and the maximum transmission power constraints of each AP; specifically, through hierarchical design, the system-level problem, i.e., user pairing, is processed at the high level. The link-level problem, i.e., power allocation, is solved at the low level; and two corresponding DDPG agents are trained respectively, so that they learn from the environment and gradually improve the strategy to maximize the system rate.

[0008] To achieve the above object, the present invention adopts the following technical solution:

[0009] A resource allocation method for a non-cellular large-scale system, comprising:

[0010] Construct a non-cellular massive MIMO system model based on non-orthogonal multiple access;

[0011] Determine the downlink data transmission rate model for the cell-free massive MIMO-NOMA system;

[0012] Determine the optimization problem model based on the downlink data transmission rate model, the goal of the optimization problem model is to maximize the rate of the non-cellular massive MIMO-NOMA system;

[0013] Based on the optimization problem model, a hierarchical deep deterministic policy gradient framework is obtained;

[0014] Based on the hierarchical deep deterministic policy gradient framework, the networks are iteratively updated to obtain the optimal strategy.

[0015] Resource allocation for cell-free massive MIMO-NOMA systems based on optimal strategies.

[0016] Preferably, the hierarchical deep deterministic policy gradient framework includes: a first DDPG agent and a second DDPG agent, wherein the first DDPG agent is used to optimize user pairing and the second DDPG agent is used to optimize power allocation.

[0017] Preferably, the first DDPG agent and the second DDPG agent utilize a strategy with maximum reward to pair users and allocate power to each user in the network, and update their strategies at the end of each training.

[0018] The present invention also provides a resource allocation system for a non-cellular large-scale system, comprising:

[0019] A first processing device is used to construct a non-cellular massive MIMO system model based on non-orthogonal multiple access;

[0020] a second processing device for determining a downlink data transmission rate model for a non-cellular massive MIMO-NOMA system;

[0021] A third processing device is used to determine an optimization problem model based on a downlink data transmission rate model, wherein a goal of the optimization problem model is to maximize a non-cellular massive MIMO-NOMA system rate;

[0022] A fourth processing device is used to obtain a hierarchical deep deterministic policy gradient framework based on the optimization problem model;

[0023] a fifth processing means for iteratively updating the networks with each other to obtain an optimal policy according to a hierarchical deep deterministic policy gradient framework;

[0024] The sixth processing device is used to allocate resources for the non-cellular massive MIMO-NOMA system based on the optimal strategy.

[0025] Preferably, the hierarchical deep deterministic policy gradient framework includes: a first DDPG agent and a second DDPG agent, wherein the first DDPG agent is used to optimize user pairing and the second DDPG agent is used to optimize power allocation.

[0026] Preferably, the fifth processing device uses the strategy with the maximum reward through the first DDPG agent and the second DDPG agent to perform user pairing and allocate power to each user in the network, and updates their strategies at the end of each training.

[0027] The present invention also provides a storage medium, on which a computer program is stored, and the computer program executes the resource allocation method for a non-cellular large-scale system when running.

[0028] The present invention proposes a joint optimization problem considering user pairing and power allocation for the downlink of a non-cellular massive MIMO-NOMA system. Under the corresponding user pairing constraints and the maximum transmit power constraints of each AP, unnecessary interference is minimized to maximize the system rate. In order to solve the proposed joint optimization problem, the H-DDPG algorithm is proposed, and a collaborative optimization framework combining user pairing and power allocation is created. By running two-layer control networks on different time scales, the decomposition and processing of system-level and link-level problems are achieved, with the goal of maximizing the system rate. For the system-level problem, i.e., user pairing, DDPG is used on a large time scale to obtain appropriate user pairing, which helps to improve the overall performance of the network. For the link-level problem, i.e., power allocation, DDPG is used on a small time scale to reduce interference between users and improve transmission performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0030] Figure 1 It is a flowchart of a resource allocation method for a non-cellular massive MIMO-NOMA system based on deep reinforcement learning provided by an embodiment of the present invention;

[0031] Figure 2 It is a structural schematic diagram of a non-cellular massive MIMO-NOMA system model provided by an embodiment of the present invention;

[0032] Figure 3 It is a structural diagram of the HDRL algorithm framework provided by an embodiment of the present invention;

[0033] Figure 4 is a schematic diagram of the variation trend of the average sum rate with the number of iterations under different schemes provided in the embodiments of the present invention;

[0034] Figure 5 It is a schematic diagram of the change trend of the cumulative distribution function of the sum rate under different schemes provided by the embodiments of the present invention;

[0035] Figure 6 It is a schematic diagram of the variation trend of the sum rate with the relative position between the AP and the user provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] Embodiment 1:

[0039] like Figure 1 As shown, an embodiment of the present invention provides a resource allocation method for a non-cellular massive MIMO-NOMA system based on deep reinforcement learning, which may include:

[0040] Step 1: Construct a non-cellular massive MIMO system model based on non-orthogonal multiple access; Figure 2 As shown, the system model includes a central controller (CPU), an access point (AP), and a cluster after user pairing.

[0041] Consider the downlink of a cell-free massive MIMO-NOMA system, where L single-antenna APs serve N single-antenna users spatially distributed in the same time-frequency resource block. The N users are divided into M clusters, each cluster consists of K users, that is, N = MK. m,k To represent the kth user in the mth cluster. All APs are connected to the CPU through a high-speed fronthaul link network to exchange collaborative information.

[0042] The standard TDD and block fading model is adopted, where the time-frequency resource is divided into coherent blocks, so that the channel coefficients in each block can be assumed to be fixed. The coherent block is represented by τ c In this embodiment of the present invention, only the resource allocation of the downlink of the non-cellular massive MIMO-NOMA system is concerned to maximize the system rate. Therefore, assuming that the duration of uplink training is τ p , the remaining coherence interval τ c -τ p Used for downlink data transmission.

[0043] From the first AP to the UE m,k The channel coefficient vector between can be modeled as

[0044]

[0045] Among them, β l,m,kis the large-scale fading coefficient, reflecting the impact of path loss and shadow fading; h l,m,k ~CN(0,1) represents the small-scale fading vector and takes Rayleigh fading into account.

[0046] The AP estimates the uplink channel through the pilot sent by the user, and then obtains the downlink channel by using the reciprocity of the TDD channel. In order to minimize the channel estimation overhead, users in the same cluster are assigned the same pilot sequence. The M pilot sequences assigned to M clusters are mutually orthogonal, so τ p ≥M. The pilot sequence assigned to K users in the mth cluster is recorded as Satisfy ||φ m || 2 = 1 and for n≠i, |φ m || 2 =1. The pilot signal received by the lth AP can be written as:

[0047]

[0048] Among them, p p is the pilot transmission power, Represents the additive white Gaussian noise vector at the lth AP.

[0049] To estimate the channel g l,m,k , project the pilot signal received by the lth AP onto φ m Get

[0050]

[0051] According to the MMSE principle, g l,m,k The channel estimate can be expressed as

[0052]

[0053] use is a Gaussian distribution, we can Rewrite as

[0054]

[0055] Among them, P l,m ~CN(0,1) and

[0056]

[0057] The corresponding channel estimation error is

[0058] Step 2: Determine the downlink data transmission rate model of the cell-free massive MIMO-NOMA system.

[0059] In downlink data transmission, the AP uses a conjugate beamformer designed based on the locally estimated channel state information, which is obtained through channel reciprocity and uplink MMSE channel estimation. Assume that the data signal of the kth user in the mth cluster is Then the signal superposition from the lth AP to the user in the mth cluster is

[0060]

[0061] In the formula, ρ d is the transmission power corresponding to the data signal, η l,m,k represents the power control coefficient sent by the lth AP to the kth user in the mth cluster. In order to meet the transmission power constraint of each AP, η l,m,k Needs to be satisfied

[0062]

[0063] Therefore, the signal received by the kth user in the mth cluster is

[0064]

[0065] Among them, n m,k ~CN(0,1) is additive white Gaussian noise.

[0066] The spectral efficiency expression of the cell-free massive MIMO-NOMA system is derived by considering the effects of channel estimation error, pilot contamination, incomplete continuous interference cancellation and power optimization.

[0067] From the above formula, y m,k Rewrite as

[0068]

[0069] To apply power-domain NOMA, we sort the users in the m-th cluster according to the expected effective channel gain, which is

[0070]

[0071] In power domain NOMA, higher power is allocated to users with lower channel gain, so that Therefore, in the mth cluster, the kth user can always decode The signal of the i-th user is obtained if the k-th user can decode his own signal. The kth user can cancel the intra-cluster interference from the i-th user before decoding its own signal. The signal of the user is regarded as interference.

[0072] Therefore, the above formula can be rewritten as:

[0073]

[0074] in, for The estimated value of

[0075]

[0076] in, and Parameter c m,k' Reflects and The correlation between m,k' The smaller the value, the smaller the correlation. Note that c m,k' = 1 does not represent complete continuous interference cancellation. This is because when c m,k' = 1, the residual intra-cluster interference term is not equal to zero. Assuming that all users only know the channel statistics, it can be further written as

[0077]

[0078] in,

[0079]

[0080]

[0081]

[0082]

[0083]

[0084] They represent coherent beamforming gain, beamforming gain uncertainty, intra-cluster interference, residual interference caused by incomplete continuous interference cancellation, and inter-cluster interference, respectively. The achievable rate is

[0085]

[0086] in,

[0087]

[0088] Step 3: Determine the optimization problem model based on the downlink data transmission rate model. The goal of the optimization problem model is to maximize the rate of the non-cellular massive MIMO-NOMA system; the variables of the optimization problem model include user pairing variables and power control coefficients.

[0089] The present invention maximizes the CF-mMIMO-NOMA system rate by jointly optimizing user pairing and power allocation. The optimization problem is formulated as

[0090]

[0091] Among them, α n,n' is the user pairing variable, n=1,...,N, n'=1,...,N, when user n and user n' are paired, α ' =1, otherwise α ' =0.

[0092] n,nn,n

[0093] The above optimization problem is to maximize the system rate under the corresponding user pairing constraints and the maximum transmit power constraints of each AP. Constraints C1-C2 and C8 are necessary conditions for user pairing and power control coefficients, respectively. Constraints C3-C6 ensure that each user can be paired with at most one other user and each user is included in exactly one pairing. Constraint C7 ensures that the transmit power of the AP does not exceed the maximum power budget.

[0094] Step 4: Based on the optimization problem model, a hierarchical deep deterministic policy gradient framework is proposed. The H-DDPG algorithm model designs two DDPG agents. In the first layer of DDPG, the user pairing problem is considered. In the second layer of DDPG, the power allocation problem is considered.

[0095] Joint optimization involves the optimization of multiple variables in the cell-free massive MIMO-NOMA system, which can make the optimization problem very complex. In addition, user pairing is a global decision for the entire system, while power allocation involves local decisions for each specific link after the connection is established. Therefore, these two sub-problems need to be solved at different time scales and levels because they have different scopes of influence and optimization objectives. In order to address the above challenges, an H-DDPG framework is proposed to jointly optimize user pairing and power allocation in the cell-free massive MIMO-NOMA system using the idea of ​​“divide and conquer”. The framework adopts a two-layer control network running on different time scales to improve the system rate through hierarchical coordinated optimization. The HDRL algorithm framework is as follows Figure 3 shown.

[0096] When using the H-DDPG algorithm to solve the joint optimization problem of user pairing and power allocation, two DDPG agents are also designed, namely, agent and Agent The intelligent UE aims to meet the corresponding constraints and obtain a better user pairing solution. The intelligent PA allocates transmission power to users to minimize inter-user interference and thus improve signal quality. Usually, the intelligent UE relies on the intelligent PA to group users, and the reward of the intelligent UE action depends on the action of the intelligent PA. On the other hand, the input of the intelligent PA It depends on the action of the agent UE and the state of the environment. The state, action and reward function related to the two agents are defined as follows.

[0097] Specifically, the two DDPG agents correspond to user pairing and power allocation, respectively. In the first layer of DDPG, a state space is established. Based on these state information, a policy network is trained to pair users to maximize the data transmission rate. In the second layer of DDPG, the power allocation problem is considered. Through the current network state and the user clusters obtained by introducing the first layer of DDPG, the second policy network is trained to optimize the power allocation of each AP to maximize the system rate. The states, actions, and reward functions related to the two agents are defined as follows.

[0098] 1. User pairing

[0099] User pairing is a global decision for the entire system, while power allocation is a local decision after a specific link is established. These two issues have different impact scopes and optimization goals and need to be solved at different time scales and levels. Therefore, user pairing is used as a high-level DDPG, and its algorithm design is as follows:

[0100] a. Status s t

[0101] The state space S is composed of the user cluster α in the previous time slot t-1 , channel coefficient vector g l,m,k and the user's rate R m,k The state of the tth time slot is defined as follows:

[0102]

[0103] b. Action a t

[0104] All alpha ' The joint selection formed by 2 NN The action space consists of discrete points.

[0105] Defined as

[0106] a t ={α 1,1 ,α 1,2 ,...,α N,N} (twenty four)

[0107] c. Reward r(s t ,a t )

[0108] According to the goal in the problem, the reward function for the tth time slot is defined as

[0109]

[0110] Among them, R penalty It is a penalty term. When the user pairing constraint is not met, a penalty is given.

[0111] The high-level DDPG algorithm includes: First, initialize the high-level network parameters. In each round of training, for each time step, exploratory actions are generated by adding noise to the policy output and constrained to the range of [-1,1]. Then, the reward and next state are calculated based on the action taken, and the transition is stored in the replay buffer. Next, a mini-batch is randomly drawn from the buffer, the target value is calculated based on the batch, and the gradients of the policy network and the value network are determined. Finally, the network weights are updated based on the gradients, and the target network weights are updated using a soft update method.

[0112] 2. Power distribution

[0113] Taking power allocation as the lower layer of H-DDPG algorithm, the DDPG algorithm is designed as follows.

[0114] a. Status s t

[0115] The state space consists of user clusters α obtained from high-level user pairings t , channel coefficient vector g l,m,k , the transmission power of the previous time slot η t-1 and the user's rate R m,k The state of the tth time slot is defined as follows:

[0116]

[0117] b. Action a t

[0118] The action of the agent is the transmit power allocated by the AP to the user. The action of the tth time slot is defined as

[0119] a t ={η 1,1,1 ,η 1,1,2 ,η 1,2,2 ,...,η L,M,K} (27)

[0120] c. Reward r(st ,a t )

[0121] According to the goal in the problem, the reward function for the tth time slot is defined as

[0122]

[0123] Where λ = 10 4 , u(x) is a step function, which is 0 when x≤0 and 1 otherwise. When the maximum transmit power constraint of each AP is not met, a penalty is imposed.

[0124] The low-level DDPG algorithm includes: First, initialize the low-level network parameters. In each round of training, determine whether the current time step meets certain conditions (such as being divisible by the interval time or within a certain time range). If so, perform an update. Then initialize the exploration noise, add the noise to the action to generate an exploratory action, and then apply the sigmoid function to limit the action to the range [0,1]. Obtain the reward and next state based on the action, and store the transfer in the playback buffer. Then randomly extract a small batch from the buffer, calculate the target value, determine the gradient of the network, and use the gradient to update the weights of the policy network and the value network. Finally, synchronize the weights of the target network through soft updates.

[0125] Step 5: The two networks iteratively update each other to obtain two optimal strategies;

[0126] The idea of ​​the H-DDPG algorithm is to update the high-level and low-level networks in sequence, where the agent continuously adjusts the high-level and low-level decisions to improve the transmission performance of the entire system. In the present invention, the main focus is on the convergence and cumulative rewards of the H-DDPG algorithm in different training periods. During each training process, the two agents use the strategy with the maximum reward to pair users and allocate power to each user in the network, and then update their strategies at the end of each training. The complete H-DDPG algorithm includes: first, initialize the parameters of the high-level and low-level networks. Then, in each round of training, for each time step t, execute the main steps of the high-level DDPG algorithm. If the current time step meets certain conditions (such as being divisible by the time interval or within a certain time range), the main steps of the low-level DDPG algorithm are executed. By alternating the execution of high-level and low-level strategy updates, hierarchical control and optimization of strategies are achieved.

[0127] Step 6: Allocate resources for the non-cellular massive MIMO-NOMA system based on the determined optimal solution.

[0128] In order to evaluate the performance of the H-DDPG algorithm in dealing with the resource allocation problem in non-cellular massive MIMO-NOMA systems, it is compared with three other algorithms. The first one is to use a single-layer DDPG algorithm to solve the joint user pairing and power allocation problem. The single-layer DDPG framework generates an action corresponding to user pairing and power allocation in each time slot, which is denoted as "S-DDPG". The second method applies the DDPG algorithm to user pairing, while using the traditional method to solve the power allocation problem, which is denoted as "UE-DDPG". The third one is a traditional joint optimization scheme.

[0129] from Figure 4 From the results, it can be observed that in the non-cellular massive MIMO-NOMA system, the highest sum rate is obtained by using the H-DDPG algorithm. In contrast, the performance of the S-DDPG algorithm is slightly worse than that of the H-DDPG, while the AP-DDPG algorithm is only slightly better than the traditional optimization scheme. Therefore, although the computational complexity of the S-DDPG and AP-DDPG algorithms is lower, the optimization performance of the H-DDPG algorithm is significantly better than that of these two algorithms. This is because the H-DDPG algorithm divides the user pairing and power allocation problems into different levels for optimization, and each level has an independent strategy network for different problems. Such a hierarchical structure makes the optimization decisions at different levels more flexible and efficient. With this design, the algorithm can better handle decisions at different levels and make more effective use of the state and action space. Therefore, as the number of APs and users increases, the performance advantage of the H-DDPG algorithm will become more significant.

[0130] Figure 5 As can be seen from the figure, the H-DDPG algorithm performs best, outperforming the other two algorithms. The second is the S-DDPG algorithm, which is slightly inferior to the H-DDPG algorithm, but still performs better than the AP-DDPG algorithm. Since the H-DDPG algorithm effectively utilizes the advantages brought by the hierarchical structure, it can make decisions more accurately. At the same time, it can more comprehensively understand the state of the entire system, thereby achieving more precise optimization. Therefore, H-DDPG can learn strategies more efficiently, especially when the hierarchical structure is reasonably divided, and usually has a faster convergence speed.

[0131] like Figure 6As shown in the figure, we can observe the impact of the change of the relative position between the AP and the user over time on the system rate. The H-DDPG algorithm has always performed well in terms of sum and rate performance, outperforming other methods. The S-DDPG algorithm is only slightly inferior to the H-DDPG and performs well. In addition, the AP-DDPG algorithm is only slightly better than the traditional algorithm. This comparison highlights the effectiveness of the H-DDPG algorithm. The reason is that the hierarchical control strategy of the H-DDPG algorithm enables the agent to handle multi-objective problems more effectively, improving the manageability and execution efficiency of the task. At the same time, the H-DDPG algorithm can better balance exploration and execution, and each level of strategy can play a role in different exploration and execution stages. It enables the agent to better explore the environment during the learning process and make more refined strategy adjustments.

[0132] The present invention proposes an HDRL algorithm to solve the joint optimization problem of user pairing and power allocation in a non-cellular massive MIMO-NOMA system. It aims to improve the system rate under the corresponding user pairing constraints and the maximum transmit power constraints of each AP. Specifically, for this non-convex optimization problem, a collaborative optimization framework combining user pairing and power allocation is created using the H-DDPG algorithm. Through a hierarchical design, user pairing and power allocation are divided into two sub-problems, and the corresponding DDPG agents are trained separately. During the training phase, these agents learn from the environment and gradually improve their strategies to maximize the system rate.

[0133] Embodiment 2:

[0134] The embodiment of the present invention further provides a resource allocation system for a non-cellular large-scale system, including:

[0135] A first processing device is used to construct a non-cellular massive MIMO system model based on non-orthogonal multiple access;

[0136] a second processing device for determining a downlink data transmission rate model for a non-cellular massive MIMO-NOMA system;

[0137] A third processing device is used to determine an optimization problem model based on a downlink data transmission rate model, wherein a goal of the optimization problem model is to maximize a non-cellular massive MIMO-NOMA system rate;

[0138] A fourth processing device is used to obtain a hierarchical deep deterministic policy gradient framework based on the optimization problem model;

[0139] a fifth processing means for iteratively updating the networks with each other to obtain an optimal policy according to a hierarchical deep deterministic policy gradient framework;

[0140] The sixth processing device is used to allocate resources for the non-cellular massive MIMO-NOMA system based on the optimal strategy.

[0141] As an implementation method of an embodiment of the present invention, a hierarchical deep deterministic policy gradient framework includes: a first DDPG agent and a second DDPG agent, wherein the first DDPG agent is used to optimize user pairing and the second DDPG agent is used to optimize power allocation.

[0142] As an implementation of an embodiment of the present invention, the fifth processing device uses the strategy with maximum reward for the first DDPG agent and the second DDPG agent to pair users and allocate power to each user in the network, and updates their strategies at the end of each training.

[0143] Embodiment 3:

[0144] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and when the computer program is run, the resource allocation method for a non-cellular large-scale system is executed.

[0145] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A resource allocation method for a non-cellular large-scale system, characterized in that: include: Construct a non-cellular massive MIMO system model based on non-orthogonal multiple access; Determine the downlink data transmission rate model for the cell-free massive MIMO-NOMA system; Determine the optimization problem model based on the downlink data transmission rate model, the goal of the optimization problem model is to maximize the rate of the non-cellular massive MIMO-NOMA system; Based on the optimization problem model, a hierarchical deep deterministic policy gradient framework is obtained; Based on the hierarchical deep deterministic policy gradient framework, the networks are iteratively updated to obtain the optimal strategy. Resource allocation for cell-free massive MIMO-NOMA systems based on optimal strategies.

2. The resource allocation method for a non-cellular large-scale system as claimed in claim 1, characterized in that: The hierarchical deep deterministic policy gradient framework includes: a first DDPG agent and a second DDPG agent, wherein the first DDPG agent is used to optimize user pairing and the second DDPG agent is used to optimize power allocation.

3. The resource allocation method for a non-cellular large-scale system as claimed in claim 2, characterized in that: The first DDPG agent and the second DDPG agent utilize the strategy with the maximum reward to pair users and allocate power to each user in the network, and update their strategies at the end of each training.

4. A resource allocation system for a non-cellular large-scale system, characterized in that: include: A first processing device is used to construct a non-cellular massive MIMO system model based on non-orthogonal multiple access; a second processing device for determining a downlink data transmission rate model for a non-cellular massive MIMO-NOMA system; A third processing device is used to determine an optimization problem model based on a downlink data transmission rate model, wherein a goal of the optimization problem model is to maximize a non-cellular massive MIMO-NOMA system rate; A fourth processing device is used to obtain a hierarchical deep deterministic policy gradient framework based on the optimization problem model; a fifth processing means for iteratively updating the networks with each other to obtain an optimal policy according to a hierarchical deep deterministic policy gradient framework; The sixth processing device is used to allocate resources for the non-cellular massive MIMO-NOMA system based on the optimal strategy.

5. The resource allocation system for a non-cellular large-scale system as claimed in claim 4, characterized in that: The hierarchical deep deterministic policy gradient framework includes: a first DDPG agent and a second DDPG agent, wherein the first DDPG agent is used to optimize user pairing and the second DDPG agent is used to optimize power allocation.

6. The resource allocation system for a non-cellular large-scale system as claimed in claim 5, characterized in that: The fifth processing device uses the strategy with the maximum reward to pair users and allocate power to each user in the network through the first DDPG agent and the second DDPG agent, and updates their strategies at the end of each training.

7. A storage medium, characterized in that: The storage medium stores a computer program, which, when running, executes the resource allocation method for a non-cellular large-scale system according to any one of claims 1 to 3.

Images