A distributed multi-agent wireless resource allocation method

Abstract

The application provides a distributed multi-agent wireless resource allocation method. The wireless resource allocation method comprises a subcarrier allocation method and a power allocation method. Firstly, a sum rate maximization model of the subcarrier allocation and the power allocation is established. Then, according to an agent load minimum standard, all agents and users are allocated. Further, the sum rate maximization model is decomposed into a subcarrier allocation optimization model and a power allocation optimization model. The subcarrier allocation optimization model and the power allocation optimization model are solved alternately by using a matching theory method and a dual subgradient iteration method respectively. Thus, the distributed multi-agent wireless resource allocation is realized. By using the method, compared with an intensive learning mode wireless resource allocation method, the complexity is low, the iteration number is small, the required calculation resource is small, and the maximum system capacity is obtained.

Description

Technical Field

[0001] This invention relates to a distributed multi-agent wireless resource allocation method, belonging to the field of communication technology. Background Technology

[0002] Currently, many high-performance resource allocation schemes have been proposed for maximizing the sum and rate of downlink OFDM systems with multiple cells and multiple users. However, multi-user resource allocation methods considering multiple OFDM frequency bands have not yet been studied. The small-scale fading of different OFDM frequency bands is different, and users can autonomously choose OFDM frequency bands for real-time communication. While existing methods include many multi-agent distributed resource allocation methods based on reinforcement learning, these all require a large number of iterations to converge. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a distributed multi-agent wireless resource allocation method. This method includes subcarrier allocation and power allocation methods. First, a sum-rate maximization model for subcarrier allocation and power allocation is established. Then, allocation is performed for all agents and users based on a minimum agent load criterion. Furthermore, the sum-rate maximization model is decomposed into a subcarrier allocation optimization model and a power allocation optimization model. These models are then solved alternately using matching theory and dual subgradient iteration methods, respectively, thereby achieving distributed multi-agent wireless resource allocation. Compared to reinforcement learning-based infinite resource allocation methods, this invention offers lower complexity, fewer iterations, and lower computational resource requirements, while achieving maximum system capacity.

[0004] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0005] A distributed multi-agent wireless resource allocation method includes the following steps:

[0006] Step A: Establish a sum-rate maximization model, which is expressed as follows:

[0007]

[0008] Constraints:

[0009]

[0010]

[0011]

[0012]

[0013]

[0014] Where x = (x1, ..., x n ,…,x N Assign state vectors to the carriers of all agents, where N is the number of agents, and p = (p1, ..., p2). n ,…,p N Assign state vectors to all agents for carrier power; Let n be the set of users accepted by agent n. This represents the number of users accepted by agent n; Assign a state vector to the carrier wave of agent n. Is this it? The user's ID, Assign a state vector to the carrier power of agent n; I is the number of OFDM bands for each agent, and K is the number of orthogonal subcarriers in each band; Assign state vectors to user m and agent n on the carrier, and define (i,k) as the k-th subcarrier of the i-th group. Let m be the indicator variable for user m using the (i,k)th subcarrier of the nth agent. Indicates use, otherwise B represents the OFDM bandwidth. Let m be the channel gain coefficient of user m on the (i,k)th subcarrier of the nth agent; This represents the transmit power obtained by user m on the (i,k)th subcarrier of the nth agent. The power allocation state vector between user m and agent n is represented as follows: This represents the sum of the squares of the absolute values ​​of the channel gain coefficients of all agents except agent n. This indicates that among all connected users of the current agent n′, the sum of all power used on the (i,k)th carrier is represented, and Then, σ represents the sum of noise power caused by the co-channel interference with the (i,k)th carrier wave of agent n; 2 Indicates the power of Gaussian white noise; Let ε(·) be the indicator variable for user m's choice of agent n, where ε(·) is the unit step function; constraint C1 indicates that each user can connect to at most one agent, but each user can select at least one subcarrier; Let m be the access indicator variable between user m and agent n, where This represents the access level of user m to agent n, and its value range is {0}∪[γ0,γ1]; when This indicates that user m can access agent n; otherwise... Constraint C1 indicates that user m is not accessible to agent n; Constraint C2 indicates that since the subcarriers on agent n are used by user m, user m must be connectable to agent n; Constraint C3 indicates that each subcarrier of each agent can be allocated to at most one user, where M is the number of users; P0 is the rated transmit power of the agent; Constraint C4 indicates that the sum of the transmit powers of all users in all agents is not greater than P0; Constraint C5 indicates that only when... Only when the power of the corresponding carrier (i,k) is not zero;

[0015] Step B: Solve the sum-rate maximization model A1. The specific steps are as follows:

[0016] Step B-1: Determine the set of users connected to agent n. The set must satisfy the following conditions:

[0017] 1)

[0018] 2) When n≠n′, let It is the set of users to be assigned that can only establish connections with agent n. It is the set of users to be assigned who have established connections with agent n and at least one other agent; the set of users to be connected to agent n is determined according to the following procedure.

[0019] Step B-1-1: Agent n connects all its accessible users The information was broadcast, and at the same time, it was also necessary to... The access level of this intelligent agent is broadcast out.

[0020] Step B-1-2: The intelligent agent n will Users are added to the user set. Inside;

[0021] Step B-1-3: The agent n checks like If not empty, then Let q be any user in the network, and let its accessible agent number be q. in The number of accessible intelligent agents, and The access levels of all accessible agents for user q are: Then select the agent with the lowest load. Its expression is: Here, α > 0 is an adjustable parameter; the function argmin(·) represents finding a variable that minimizes the function value within the parentheses; thus, the set of connected users for each agent is determined.

[0022] Step B-2: Allocate power and subcarriers;

[0023] Step B-2-1: Initialize power and subcarriers;

[0024] Step B-2-1-1: The power of each subcarrier is initialized to the average subcarrier power: P0 / IK;

[0025] Step B-2-1-2: Subcarrier allocation state vector x n The initial value is obtained as follows:

[0026] Step B-2-1-2-1: Transform model A1 into a state vector x about the carrier allocation. n Model B1, wherein model B1 is:

[0027]

[0028] Constraints:

[0029]

[0030]

[0031] Among them, parameters In p n When the carrier average power is, It is a constant;

[0032] Step B-2-1-2-2: Solve for model B1 and perform carrier allocation as follows: The function argmax(·) finds a variable that maximizes the value of the function within the parentheses;

[0033] Step B-2-1-3: Thus, the carrier allocation state vector x is obtained. n With power distribution state vector p n The initial solution;

[0034] Step B-2-2: Split model A1 to obtain the optimized carrier allocation state vector x. n Model A2 and optimized power allocation state vector p n Model A3;

[0035] Step B-2-2-1: The optimized carrier allocation state vector x n Model A2 is expressed as:

[0036]

[0037] Constraints:

[0038]

[0039]

[0040] in, exist When known, It is a constant;

[0041] Step B-2-2-2: The optimized power allocation state vector p n Model A3 is expressed as follows:

[0042]

[0043] Constraints:

[0044]

[0045]

[0046] Step B-2-2-3: Perform a convex approximation on model A3, expanding model A3 into model A4, which is expressed as:

[0047]

[0048] Constraints:

[0049]

[0050]

[0051] Step B-2-2-4: The objective function in model A4... exist Expanding the Taylor formula at this point yields:

[0052]

[0053] Step B-2-2-5: The model A5 after convex approximation is expressed as:

[0054]

[0055] Constraints:

[0056]

[0057]

[0058] Step B-2-3: Iteratively solve model A1. The solution method for model A1 is as follows: use matching theory to solve model A2 to obtain the subcarrier allocation state vector x. nThe dual subgradient iteration method is used to solve model A5, and the power allocation state vector p is obtained. n In the t-th iteration, the solution process is as follows:

[0059] Step B-2-3-1: Solve model A2 using a matching theory-based method to obtain the carrier allocation state vector x of agent n during the t-th iteration. n [t], where x n [t] represents the x obtained in the t-th iteration. n The value of x n [t] = x n ;

[0060] Step B-2-3-2: x n =x n [t], x n Substitute these values ​​into model A5; then solve model A5 using the dual subgradient iteration method. The solution process of this method is as follows:

[0061] Step B-2-3-2-1: Introduce the Lagrangian function into the objective function of model A5 as follows:

[0062]

[0063] Where β is the dual multiplier, and its value is β≥0;

[0064] Step B-2-3-2-2: Obtain the result using the bisection method. The zero point, which is denoted as

[0065] Step B-2-3-2-3: Update the dual multiplier β according to the subgradient iteration method as follows: Let β = β′, where ρ>0 indicates a variable step size, and the function...

[0066] Step B-2-3-2-4: Alternately optimize the dual variable β and the original variable Until the objective function of model A5 remains constant, the power allocation state vector p of agent n in the t-th iteration is obtained. n [t], p n [t] represents the p obtained in the t-th iteration. n The value of p n [t] = p n ;

[0067] Step B-2-4: x n =x n [t], p n =p n [t], xn and p n Substitute the values ​​into model A1 and calculate the objective function of model A1. When the objective function of model A1 no longer changes, the iterative method terminates, and the result x is output. n and p n Otherwise, proceed to the next iteration.

[0068] Beneficial Effects: This invention provides a distributed multi-agent wireless resource allocation method. The method includes subcarrier allocation and power allocation methods. First, a sum-rate maximization model for subcarrier allocation and power allocation is established. Then, allocation is performed for all agents and users based on the minimum agent load criterion. Furthermore, the sum-rate maximization model is decomposed into a subcarrier allocation optimization model and a power allocation optimization model. These models are then solved alternately using matching theory and dual subgradient iteration methods, respectively, thereby achieving distributed multi-agent wireless resource allocation. Compared to reinforcement learning-based infinite resource allocation methods, this invention has lower complexity, fewer iterations, requires less computational resources, and achieves maximum system capacity. Detailed Implementation

[0069] This invention presents a distributed multi-agent wireless resource allocation method. The method takes a multi-cell downlink OFDM system as its background, with the objective function of maximizing the system reachability rate, and constrained by the agent's maximum transmit power and the number of carriers. It studies the joint optimization problem of subcarrier allocation and power allocation. To solve the corresponding non-convex problem, firstly, without considering cell interference, an initial solution for subcarrier allocation is obtained based on the optimal channel gain. The initial solution for power allocation is the average subcarrier power. Then, based on the obtained initial solutions, the original problem considering cell interference is decomposed into two subproblems, which are solved separately. A matching theory-based method is used to solve the subcarrier allocation sub-optimization problem. For the power allocation subproblem, the atomic problem is first approximated as a convex optimization problem, and then a low-complexity dual subgradient iteration method is proposed for solving it.

[0070] To better illustrate the method of the present invention, more detailed examples are provided below:

[0071] Consider a system with N agents and M users based on OFDM downlink transmission. Each agent and user is equipped with an isotropic omnidirectional transmit antenna, and the rated transmit power of the agent is P0. The M users are distributed with equal probability in the area covered by the N agents, and each user can only obtain downlink data through one agent.

[0072] First step, use The access level of user m to agent n is represented by the pilot signal level of the agent measured by the user, and its value is taken from the set {0}∪[γ0,γ1]. This indicates that user m can access agent n; otherwise... This indicates that user m cannot access agent n. (Definition) Where ε(·) is the unit step function, obviously, when hour, otherwise Let represent the accessibility indicator variable for user m and agent n. Let n be the set of users accepted by agent n, i.e., if users but Assume that each agent has I sets of continuous OFDM frequency bands, each set of OFDM frequency bands has a bandwidth of B, and is divided into K orthogonal subcarriers, that is, the bandwidth of each subcarrier is B / K, and each subcarrier can only be allocated to one user.

[0073] The second step is to use (i,k) to represent the k-th subcarrier of the i-th group, and then... This indicates that user m uses the (i,k)th subcarrier of the nth agent. Indicates use, otherwise The indicator variable for user m's choice of agent n. x = (x1, ..., xn) n ,…,x N Assign state vectors to the carriers of all agents. Assign a state vector to the carrier wave of agent n. Is this it? The user's ID, This represents the number of users accepted by agent n; the subcarrier allocation state vector between user m and agent n is defined as follows: Let p represent the transmit power obtained by user m on the (i,k)th subcarrier of the nth agent, where p = (p1,...,p...). n ,…,p N ) Assign state vectors to all agents for carrier power. Let m be the power allocation state vector for user m and agent n. Assume that user m's channel gain coefficient on the (i,k)th subcarrier of the nth agent is...

[0074] The third step is to calculate the signal-to-interference-plus-noise ratio (SIR) of the received signal of user m on the (i,k)th subcarrier of the nth agent, denoted as . It can be represented as follows:

[0075]

[0076] in, σ represents the total inter-cell interference caused to user m by agents other than the agent itself on the (i,k)th subcarrier. 2 This represents the Gaussian white noise power. Additionally, the channel gain coefficient... Represented as Among them, PathLoss m X is the path loss for user m. α It is a log-normal shadowing fading. It is the Rayleigh fading gain of user m and agent n on the (i,k)th subcarrier.

[0077] In the fourth step, the achievable rate obtained by user m on the (i,k)th subcarrier of the nth agent is:

[0078]

[0079] Fifth step, the total downlink reachable rate obtained by user m is:

[0080]

[0081] Step 6: Thus, the total downlink speed obtained by all users is:

[0082]

[0083] Therefore, based on the above, we establish the corresponding sum-rate maximization model A1, which is expressed as:

[0084]

[0085] Constraints:

[0086]

[0087]

[0088]

[0089]

[0090]

[0091] The specific steps to solve model A1 are as follows:

[0092] Step 1: Determine the set of users that agent n connects to. The set must satisfy the following conditions:

[0093] 1)

[0094] 2) When n≠n′, let It is the set of users to be assigned that can only establish connections with agent n. It is the set of users to be assigned who have established connections with agent n and at least one other agent; the set of users to be connected to agent n is determined according to the following procedure.

[0095] Step 1-1: Agent n connects all its accessible users The information was broadcast, and at the same time, it was also necessary to... The access level of this intelligent agent is broadcast out.

[0096] Step 1-2: The intelligent agent n will Users are added to the user set. Inside;

[0097] Steps 1-3: The intelligent agent n checks like If not empty, then Let q be any user in the network, and let its accessible agent number be q. in The number of accessible intelligent agents, and The access levels of all accessible agents for user q are: Then select the agent with the lowest load. Its expression is: Here, α > 0 is an adjustable parameter; the function argmin(·) represents finding a variable that minimizes the function value within the parentheses; thus, the set of connected users for each agent is determined.

[0098] Step 2: Allocate power and subcarriers;

[0099] Step 2-1: Initialize power and subcarriers;

[0100] Step 2-1-1: Initialize the power of each subcarrier to the average subcarrier power: P0 / IK;

[0101] Step 2-1-2: Subcarrier allocation state vector x n The initial value is obtained as follows:

[0102] Step 2-1-2-1: Transform model A1 into a state vector x related to carrier allocation. n Model B1, wherein model B1 is:

[0103]

[0104] Constraints:

[0105]

[0106]

[0107] Among them, parameters In p n When the carrier average power is, It is a constant;

[0108] Step 2-1-2-2: Solve for model B1, and perform carrier allocation as follows: The function argmax(·) finds a variable that maximizes the value of the function within the parentheses;

[0109] Step 2-1-3: Thus, the carrier allocation state vector x is obtained. n With power distribution state vector p n The initial solution;

[0110] Step 2-2: Split model A1 to obtain the optimized carrier allocation state vector x. n Model A2 and optimized power allocation state vector p n Model A3;

[0111] Step 2-2-1: The optimized carrier allocation state vector x n Model A2 is expressed as:

[0112]

[0113] Constraints:

[0114]

[0115]

[0116] in, exist When known, It is a constant;

[0117] Step 2-2-2: The optimized power allocation state vector p n Model A3 is expressed as follows:

[0118]

[0119] Constraints:

[0120]

[0121]

[0122] Step 2-2-3: Perform a convex approximation on model A3, expanding model A3 into model A4, which is expressed as:

[0123]

[0124] Constraints:

[0125]

[0126]

[0127] Step 2-2-4: The objective function in model A4... exist Expanding the Taylor formula at this point yields:

[0128]

[0129] Step 2-2-5: The model A5 after convex approximation is expressed as:

[0130]

[0131] Constraints:

[0132]

[0133]

[0134] Steps 2-3: Iteratively solve model A1. The solution method for model A1 is as follows: use matching theory to solve model A2 to obtain the subcarrier allocation state vector x. n The dual subgradient iteration method is used to solve model A5, and the power allocation state vector p is obtained. n In the t-th iteration, the solution process is as follows:

[0135] Step 2-3-1: Solve model A2 using a matching theory-based method. The solution process is as follows:

[0136] Step 2-3-1-1: Users and carriers are two sets of elements that need to be matched, according to the parameters in model A2. Build each user's preference list from largest to smallest;

[0137] Step 2-3-1-2: In each round of matching, each user sends an access request to the carrier at the top of their preference list. If a carrier receives access requests from multiple users, then that carrier reverses its selection parameters. The largest user;

[0138] Step 2-3-1-3: Remove each matched carrier from the preference list;

[0139] Step 2-3-1-4: Repeat steps 2-3-1-2 to 2-3-1-3 until all carriers are matched; thus obtaining the carrier allocation state vector x of agent n during the t-th iteration. n [t],x n [t] represents the x obtained in the t-th iteration. n The value of x n [t] = x n ;

[0140] Step 2-3-2: x n =x n [t], x n Substitute these values ​​into model A5; then solve model A5 using the dual subgradient iteration method. The solution process of this method is as follows:

[0141] Step 2-3-2-1: Introduce the Lagrangian function into the objective function of model A5 as follows:

[0142]

[0143] Where β is the dual multiplier, and its value is β≥0;

[0144] Step 2-3-2-2: Obtain the result using the bisection method. The zero point, which is denoted as

[0145] Step 2-3-2-3: Update the dual multiplier β according to the subgradient iteration method as follows: Let β = β′, where ρ>0 indicates a variable step size, and the function...

[0146] Step 2-3-2-4: Alternately optimize the dual variable β and the original variable Until the objective function of model A5 remains constant, the power allocation state vector p of agent n in the t-th iteration is obtained. n [t], p n [t] represents the p obtained in the t-th iteration. n The value of p n [t] = p n ;

[0147] Steps 2-4: x n =x n [t], p n =p n [t], x n and p n Substitute the values ​​into model A1 and calculate the objective function of model A1. When the objective function of model A1 no longer changes, the iterative method terminates, and the result x is output.n and p n Otherwise, proceed to the next iteration.

Claims

1. A method for distributed multi-agent wireless resource allocation, the method comprising: comprising the steps of: Step A: the total downlink reachable rate obtained by user m is: the total downlink rate obtained by all users is: A sum rate maximization model is established, which is expressed as A1: Constraint condition: Where x = (x1, ..., x n ,…,x N Assign state vectors to the carriers of all agents, where N is the number of agents, and p = (p1, ..., p2). n ,…,p N Assign state vectors to all agents for carrier power; U n For the set of users accepted by agent n, |U n | represents the number of users accepted by agent n; Assign a state vector to the carrier wave of agent n. Is this | U n | User ID, Assign a state vector to the carrier power of agent n; I is the number of OFDM bands for each agent, and K is the number of orthogonal subcarriers in each band; Assign state vectors to user m and agent n on the carrier, and define (i,k) as the k-th subcarrier of the i-th group. Let m be the indicator variable for user m using the (i,k)th subcarrier of the nth agent. Indicates use, otherwise B represents the OFDM bandwidth. Let m be the channel gain coefficient of user m on the (i,k)th subcarrier of the nth agent; This represents the transmit power obtained by user m on the (i,k)th subcarrier of the nth agent. The power allocation state vector between user m and agent n is represented as follows: This represents the sum of the squares of the absolute values ​​of the channel gain coefficients of all agents except agent n. This indicates that among all connected users of the current agent n′, the sum of all power used on the (i,k)th carrier is represented, and Then, σ represents the sum of noise power caused by the co-channel interference with the (i,k)th carrier wave of agent n; 2 Indicates the power of Gaussian white noise; Let ε(·) be the indicator variable for user m's choice of agent n, where ε(·) is the unit step function; constraint C1 indicates that each user can connect to at most one agent, but each user can select at least one subcarrier; Let m be the access indicator variable between user m and agent n, where This represents the access level of user m to agent n, and its value range is {0}∪[γ0,γ1]; when This indicates that user m can access agent n; otherwise... C2: user m must be connectable to agent n if a subcarrier on agent n is used by user m; C3: each subcarrier of each agent can be assigned to at most one user, where M is the number of users; P0is the rated transmission power of the agents, C4: the sum of the transmission powers of all users in all agents is not greater than P0; C5: the power of the corresponding carrier (i, k) is not 0 only if C5 is true. Step B: solving the sum rate maximization model A1, the specific steps are as follows: Step B-1: Determine the set of users U connected to the agent n n which must satisfy the following conditions: 1) 2) when n≠n', F n is the set of users to be allocated that can only establish a connection with the agent n, D n is the set of users to be allocated that can establish a connection with the agent n and at least one other agent; the set of access users U for the agent n is determined according to the following procedure n : Step B-1-1: Agent n broadcasts its information of all accessible users F n ∪D n together with the access level of D n to this agent. Step B-1-2: The agent n adds the user in F n to the set of users U n ; Step B-1-3: The agent n checks D n If D n If not empty, then D n Let q be any user in the network, and let its accessible agent number be q. in The number of accessible intelligent agents, and The access levels of all accessible agents for user q are: Then select the agent with the lowest load. Its expression is: Where a>0 is an adjustable parameter; the function argmin(·) represents finding a variable that minimizes the function value within the parentheses; thus, the set of connected users U for each agent is determined. n ; Step B-2: allocating power and subcarriers; Step B-2-1: initializing power and subcarriers; Step B-2-1-1: the power of each subcarrier is initialized as the average power of subcarriers: P0 / IK; Step B-2-1-2: Allocation state vector x of subcarriers n The initial value of the vector x is obtained as follows: Step B-2-1-2-1: Transforming model A1 into model B1 regarding the carrier allocation state vector x n which is the model B1: Constraint condition: wherein the parameters In p n is the average power of the carrier, is a constant; Step B-2-1-2-2: Solving model B1, carrier allocation is performed as follows: The function argmax(•) means finding a variable that makes the function value in the parentheses maximum; Step B-2-1-3: The carrier allocation state vector x is thus obtained n with the initial solution of the power allocation state vector p n ​ Step B-2-2: Splitting of model A1 to get model A2 for optimized carrier allocation state vector x n and model A3 for optimized power allocation state vector p n ​ Step B-2-2-1 : The optimized carrier allocation state vector x n The model A2 is formulated as: Constraint condition: wherein In Known, is constant; Step B-2-2-2: The optimized power allocation state vector p n The model A3 is expressed as: Constraint condition: Step B-2-2-3: convex approximation is performed on the model A3, and the model A3 is expanded into a model A4, which is expressed as: Constraint condition: Step B-2-2-4: The Taylor expansion of the objective function in model A4 at At the Taylor expansion is Step B-2-2-5: the model A5 after convex approximation is expressed as: Constraint condition: Step B-2-3: Solve model A1 iteratively, by solving model A2 using the method of matching theory to obtain the subcarrier allocation state vector x n ; solving model A5 using the dual subgradient iterative method to obtain the power allocation state vector p n ; in the t-th iteration, the solving process is as follows: Step B-2-3-1 : Solve model A2 using the matching theory based approach to obtain the carrier allocation state vector x of agent n in the tth iteration n [t], where x n [t] denotes the value of x n obtained in the tth iteration, i.e. x n [t] = x n ; Step B-2-3-2: x n = x n [t], x n is substituted into model A5; and then model A5 is solved using a dual subgradient iteration method, which is solved as follows: Step B-2-3-2-1: a Lagrange function is introduced into the objective function in the model A5 as: Wherein β is a dual multiplier, and the value β≥0; Step B-2-3-2-2: The zero of is found using bisection, and the zero is noted as Step B-2-3-2-3: Update the dual multiplier β according to the dual subgradient iteration method as follows: Let β = β', where ρ > 0 is a variable step size, function Step B-2-3-2-4: Alternating optimization of dual variables β and primal variables until the value of the objective function of model A5 remains unchanged, thereby obtaining the power allocation state vector p of the agent n in the tth iteration process n [t], p n [t] represents the value of p obtained in the tth iteration n [t], i.e. p n [t] = p n ; Step B-2-4: x n = x n [t], p n = p n [t], x n and p n are substituted into model A1, and the value of the model A1 objective function is calculated, when the value of the model A1 objective function no longer changes, the iterative method terminates, and the results x n and p n are output; otherwise, the next round of iteration is performed.

Images