Resource allocation method and system based on intelligent reflective surface assisted edge network

Abstract

The present application provides a resource allocation method and system based on an intelligent reflective surface-assisted edge network, which optimizes the user-base station association, determines the computational offloading strategy, resource allocation strategy and IRS phase shift design strategy to minimize the long-term total energy consumption of all users, while ensuring the user's quality of service. Since the time scale of the user and base station association decision is larger than the computational offloading, resource allocation and phase shift design strategy, the present application proposes a dual time scale algorithm to divide the large-scale time into large-scale time slots and small-scale time slots. For the user-base station association problem in large-scale time slots, a matching algorithm with low complexity is used for one-to-many matching. In small-scale time slots, a deep Q network algorithm is used to learn computational offloading, resource allocation and IRS phase shift design strategies, which can adapt to the dynamics of the wireless environment. The resource allocation method of the present application can effectively reduce the energy consumption of the edge network.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a resource allocation method and system based on an intelligent reflective surface-assisted edge network. Background Art

[0002] With the upcoming 6G technology and the Internet of Things technology, smart devices are growing explosively. The emergence of a series of new applications and services such as autonomous vehicles, augmented reality (AR) and ultra-high-definition (UHD) video streaming has put forward higher requirements for transmission bandwidth, latency, energy consumption, application performance and reliability. Multi-access edge computing (MEC) brings computing, storage and control functions to the edge of mobile networks (such as base stations and access points), thereby enabling computationally intensive and latency-critical applications on mobile devices with limited resources. Therefore, in edge networks, emerging mobile applications can offload computationally intensive tasks to MEC servers for computational execution, thereby reducing transmission latency and user device energy consumption, and alleviating the burden on the backhaul network.

[0003] However, when the wireless channel is in deep fading, the link for data and computation offloading is hindered, limiting the performance of the edge network. For example, user devices at the edge of the cell are usually unable to successfully offload because their computation offloading latency may be higher than the latency of local computation tasks. Therefore, these mobile devices must use their own computing resources, which often cannot support resource-intensive applications. Therefore, it is very important to improve the performance of edge networks from a communication perspective. Summary of the invention

[0004] In view of this, the purpose of the present application is to propose a resource allocation method and system based on an intelligent reflective surface assisted edge network.

[0005] Based on the above purpose, the present application provides a resource allocation method based on an intelligent reflective surface assisted edge network, wherein the edge network includes multiple base stations and multiple users, the multiple base stations include a macro base station and multiple small base stations, the user communicates directly with the small base station or uses an intelligent reflective surface to assist in communicating with the macro base station, and the resource allocation method includes:

[0006] For each large-scale time slot in the large-scale time, in response to receiving a computing task request sent by the user, the macro base station calculates a user base station association strategy using a matching algorithm based on the computing task request and all channels in the edge network; and

[0007] For each small-scale time slot, the macro base station uses a deep Q network algorithm to determine a computing offloading strategy, a resource allocation strategy, and a phase shift design strategy based on the computing task request, the channel state in the edge network, and the currently available resources in the edge network, with the total energy consumption of the edge network being minimized and the delay limit of the computing task request being met as the optimization goal;

[0008] Based on all of the user-base station association strategies, the computation offloading strategies, the resource allocation strategies, and the phase shift design strategies in large-scale time, the macro base station performs resource allocation on the user,

[0009] The large-scale time is divided into a group of large-scale time slots, and each large-scale time slot is divided into a group of small-scale time slots.

[0010] Based on the same inventive concept, the present application also provides a resource allocation system based on an intelligent reflective surface assisted edge network, wherein the edge network includes multiple base stations and multiple users, the multiple base stations include a macro base station and multiple small base stations, the user communicates directly with the small base station or uses an intelligent reflective surface to assist in communicating with the macro base station, and the macro base station is configured as:

[0011] For each large-scale time slot in the large-scale time, in response to receiving a computing task request sent by the user, the macro base station calculates a user base station association strategy using a matching algorithm based on the computing task request and all channels in the edge network; and

[0012] For each small-scale time slot, the macro base station uses a deep Q network algorithm to determine a computing offloading strategy, a resource allocation strategy, and a phase shift design strategy based on the computing task request, the channel state in the edge network, and the currently available resources in the edge network, with the total energy consumption of the edge network being minimized and the delay limit of the computing task request being met as the optimization goal;

[0013] Based on all of the user base station association strategies, the computation offloading strategies, the resource allocation strategies and the phase shift design strategies, the macro base station performs resource allocation on the user,

[0014] The large-scale time is divided into a group of large-scale time slots, and each large-scale time slot is divided into a group of small-scale time slots.

[0015] As can be seen from the above, the present application provides a resource allocation method and system based on an intelligent reflective surface-assisted edge network. The system includes a macro base station and multiple small base stations. The user's computing tasks can be offloaded to the macro base station for execution through the intelligent reflective surface. By optimizing the user-base station association, the calculation offloading strategy, resource allocation strategy and IRS phase shift design strategy are determined to minimize the long-term total energy consumption of all users while ensuring the user's service quality. Since the time scale of the user and base station association decision is larger than the calculation offloading, resource allocation and phase shift design strategy, the present application proposes a dual time scale algorithm to divide the large-scale time into large-scale time slots and small-scale time slots. For the user-base station association problem in large-scale time slots, a matching algorithm with low complexity is used for one-to-many matching. In small-scale time slots, a deep Q network algorithm is used to learn calculation offloading, resource allocation and IRS phase shift design strategies, which can adapt to the dynamics of the wireless environment. The resource allocation method of the present application can effectively reduce the energy consumption of the edge network. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A schematic diagram of a process of resource allocation method based on intelligent reflective surface assisted edge network according to an embodiment of the present application;

[0018] Figure 2 A schematic diagram of an edge network model according to an embodiment of the present application;

[0019] Figure 3 This is a schematic diagram of the method flow of step S11 of an embodiment of the present application;

[0020] Figure 4 Schematic diagram of the method flow of step S12 of an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0022] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0023] In the prior art, Intelligent Reflecting Surface (IRS) is a new paradigm for realizing the intelligent and reconfigurable wireless propagation environment in B5G / 6G wireless communication systems in recent years. IRS is a two-dimensional artificial electromagnetic material surface, namely a metasurface, which is composed of a large number of passive reflective elements with special physical structures, and each reflective element can be controlled in a software-defined manner. Through the joint phase control of all reflective elements, the reflection phase and angle of the incident RF signal can be arbitrarily adjusted to produce an ideal multipath effect. The reflected RF signals can be coherently superimposed to increase the received signal power, or destructively combined to reduce interference. IRS is a promising technology to improve the performance of edge networks.

[0024] For example, in a single-user scenario, IRS passive beamforming can improve the wireless link capacity between it and the edge server; thereby reducing computing latency and not causing high transmission energy consumption; for large-scale edge networks, IRS has an important impact on offloading decisions and strategies, and can adjust offloading channels to different edge servers to achieve improved channel status and reasonable allocation of resources. For example, if offloading all computing tasks to the same neighboring server will cause congestion and high computing latency, IRS can assist users in offloading computing tasks to farther edge servers with lighter computing loads, and the channel status will be greatly improved by IRS passive beamforming. Therefore, this application considers an edge network with IRS assistance.

[0025] This application proposes an IRS-assisted edge network, which includes a macro base station and multiple small base stations. IRS can assist users in offloading computing tasks to a distant macro base station edge server with strong computing power. This application is based on reality and considers that the user's application contains multiple subtasks with sequential dependencies, so fine-grained partial computing offloading for each subtask is required. By optimizing user-base station association, computing offloading, resource allocation and IRS phase shift design decisions to minimize the long-term total energy consumption of all users, while ensuring the user's quality of service (quality of service, QoS), that is, the execution delay of computing tasks. Since the time scale of user and base station association decisions is larger than that of computing offloading, resource allocation and phase shift design decisions, this application solves the optimization problem on a dual time scale.

[0026] Time scale refers to the average measure of the time it takes to complete a physical process. Generally speaking, the slower the evolution of a physical process, the longer its time scale; the larger the spatial scope of the physical process design, the longer its time scale.

[0027] For the user-base station association problem on a large time scale, this application uses a low-complexity matching algorithm to perform one-to-many matching based on the preferences of both parties. Since the interference between users matched with small base stations will affect the rate, the interdependent preferences between user preferences are called externalities. This application considers the concept of exchange matching to deal with externalities. Since the number of users and base stations is limited, it will converge to a stable match after a limited number of iterations.

[0028] On a small time scale, since the dynamics of the environment caused by wireless channels, computing task requests and resource changes will affect the decision-making, it is modeled as a Markov decision process (MDP) and a reinforcement learning algorithm is used to solve the problem. At the same time, considering that this optimization problem has a high-dimensional state space, the calculation time and complexity of the traditional reinforcement learning algorithm are greatly increased, and the algorithm is difficult to converge. In order to solve the above problems, this application uses a deep neural network (DNN) to approximate the estimated value function, as well as experience replay and independent target network methods to speed up the convergence of the deep reinforcement learning algorithm. Therefore, this application uses a deep Q-network (DeepQ-Network, DQN) algorithm to learn computational offloading, resource allocation and IRS phase shift design strategies.

[0029] In this application, scalars are represented by italic letters. Lowercase and uppercase letters in boldface represent vectors and matrices, respectively. Superscript (·) T and(·) HRepresent the transpose and conjugate transpose (Hermitian transpose) operations respectively. Represents a complex matrix with a space of M×N. The representation space is an M×N real matrix.

[0030] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0031] The present application provides a resource allocation method based on an intelligent reflective surface assisted edge network, wherein the edge network includes multiple base stations and multiple users, the multiple base stations include a macro base station and multiple small base stations, the users communicate directly with the small base stations or use intelligent reflective surfaces to assist in communicating with the macro base stations, and reference Figure 1 , the resource allocation method comprises the following steps:

[0032] Step S1 includes the following two steps:

[0033] Step S11, for each large-scale time slot in the large-scale time, in response to receiving a computing task request sent by the user, the macro base station calculates a user base station association strategy using a matching algorithm based on the computing task request and all channels in the edge network; and

[0034] Step S12: For each small-scale time slot, the macro base station uses a deep Q network algorithm to determine the computing offloading strategy, resource allocation strategy and phase shift design strategy based on the computing task request, the channel status in the edge network and the currently available resources in the edge network, with the optimization goal of minimizing the total energy consumption of the edge network and meeting the delay limit of the computing task request.

[0035] Specifically, Figure 2 The figure shows a schematic diagram of the edge network model. The edge network contains multiple base stations with single antennas and multiple users. The base stations include a macro base station and N small base stations. The symbol n represents the nth base station, where n=0 represents a macro base station. Each base station is equipped with a multi-access edge computing MEC server with a multi-core CPU. There are K single-antenna users in the edge network model, and the symbol k represents the kth user. Each user can only be served by one base station. Assume that base station n is connected to K n User equipment (UE), the set of connected users is represented by where k nIndicates that the kth user is connected to base station n. This embodiment takes into account that mobile applications are divided into many components, such as face recognition and AR applications, so it is necessary to partially unload the user's task model. The dependencies between the components of many applications cannot be ignored. This model is called a task graph model, which is a typical directed acyclic graph. The typical dependency model between the three subtasks (i.e., task components) includes sequential, parallel, and general dependencies. Each user has a delay-sensitive application that includes multiple subtasks with dependencies. It means that user k has U k Each component of each computing task can be performed on the user's local device or on the MEC server ( Figure 2 The MEC server in the small base station is used for calculation. Since the computing resources of the small base station are limited (while avoiding frequent switching), the user can connect to the macro base station for calculation offloading. Since some users (such as cell edge users) are far away from the macro base station and the channel quality is not high, this embodiment assumes that if the user is served by the macro base station, the IRS is used to assist the user and the macro base station to communicate. There are M IRS reflection elements, and the symbol m represents the mth reflection element. The IRS is connected to an intelligent controller ( Figure 2 The IRS controller in the system dynamically adjusts the reflection of the required IRS elements and exchanges (control) information with the base station through a separate wireless link. The base station is connected to the base station controller responsible for user association and resource allocation. In this embodiment, the macro base station is used as a centralized base station controller.

[0036] Since the time scale of user-base station association is larger than computation offloading and resource allocation, this embodiment models the IRS-assisted MEC model as a dual-time scale edge computing model. The IRS-assisted MEC process occurs on two different time scales: user-base station association occurs on a large time scale, and computation offloading, resource allocation, and IRS phase shift design occur on a small time scale. The basic time unit of the large time scale is defined as a large-scale time slot, with Represents the number of large-scale time slots, each large-scale time slot Start user association decision. Each large-scale time slot can be divided into a group of small-scale time slots. represents the number of small-scale time slots, and calculates offloading and resource allocation in each small-scale time slot Start execution, where is the maximum computation offloading latency of all users.

[0037] In this embodiment, if the user is connected to a macro base station, there are two unloading links: a user-base station direct link and a user-IRS-base station reflection link. If the user is connected to a small base station, there is only one unloading link, namely the user-base station direct link. In this embodiment, the amplitude in the IRS phase shift matrix is ​​set to a maximum value of 1, and a discrete phase shift design with p phases is set. Orthogonal frequency division multiple access (OFDMA) is adopted, and there are B equally divided sub-bands (sub-band), and the symbol b represents the bth sub-band. The spectrum used by users of the same base station is orthogonal, and the spectrum used by the macro base station and the small base station is orthogonal, so only the interference between small base stations is considered in this embodiment. One subband per small-scale time slot can only be allocated to at most one user in any base station. Each user must be assigned at least one subband, assuming that the complete channel information can be obtained at the base station.

[0038] The optimization goal of this embodiment is to minimize the system energy consumption within the entire time range while meeting the user delay constraint. The optimization problem is a mixed integer nonlinear programming problem with variables at different time scales, which is a typical NP-hard problem. In order to solve the problem efficiently, this embodiment solves the optimization problem on a dual time scale, that is, optimizes large time scale variables and small time scale variables respectively. On a large time scale, a user base station association decision is obtained based on a matching algorithm. On a small time scale, a deep reinforcement learning algorithm is used to learn calculation offloading, resource allocation and IRS phase shift design strategies. The reason for applying a matching algorithm to solve the user base station association problem is that matching theory has the advantage of low complexity over traditional methods such as exhaustive search. Since the dynamics of the environment caused by wireless channels, calculation requests and resource changes will affect calculation offloading and resource allocation decisions, the optimization problem on a small time scale can be regarded as a sequential decision problem, which is modeled as a Markov decision process (MDP) and solved by reinforcement learning. At the same time, considering that the optimization problem contains a high-dimensional state space, a deep reinforcement learning algorithm is considered to be used for solution.

[0039] Step S2: Based on all the user base station association strategies, the calculation offloading strategies, the resource allocation strategies and the phase shift design strategies in the large-scale time, the macro base station performs resource allocation on the user.

[0040] The large-scale time is divided into a group of large-scale time slots, and each large-scale time slot is divided into a group of small-scale time slots.

[0041] After the above step S1, the corresponding strategy is obtained. The macro base station performs a one-to-many match between base stations and users based on the user base station association strategy. The calculation offloading strategy is used to determine whether part of the user's application is offloaded to the edge computing server. The resource allocation strategy is used to determine the number of subbands allocated to the user and the transmission power of the subbands. The phase shift of the reflective element of the smart reflective surface is determined by the phase shift design strategy, thereby completing the resource allocation based on the user's computing task request.

[0042] In some embodiments, reference Figure 3 The macro base station calculates the user base station association strategy based on the computing task request and all channels in the edge network using a matching algorithm, that is, step S11, including:

[0043] Step S111: Initialize matching of all the users and all the base stations in the edge network to obtain a matching set, and obtain a user preference list of each user and a base station preference list of each base station based on the computing task request and all the channels;

[0044] Step S112: judging whether there is a switching blocking pair in the matching set based on the user preference list and the base station preference list;

[0045] Step S113: in response to determining that the exchange blocking pair exists, exchanging the two base stations in the exchange blocking pair respectively associated with the two users.

[0046] In response to determining that the exchange blocking pair does not exist, the current matching set is used as the user base station association strategy.

[0047] Specifically, the user-to-base station association problem can be modeled as a bilateral matching game, in which there are two sets of unrelated players: the user set and base station collection Each user k can be matched with one base station, and each base station n can be matched with multiple users. The matching is defined as x. This embodiment is a one-to-many matching.

[0048] The one-to-many matching game considered in this embodiment consists of two groups of players, namely and Define the match x as arrive A function of the set of all subsets, with the following characteristics

[0049]

[0050]

[0051]

[0052] Therefore, the user-base station association index can be determined by matching x,

[0053]

[0054] x k,n (t i ) represents the association variable between user k and base station n, x k,n ∈{0,1},x k,n =1 indicates that user k is connected to base station n, otherwise it is 0.

[0055] The preference list of each user and each base station is calculated based on the computing task request and all the channels. Since the interference between users matched with the small base station will affect the rate, the user preference also depends on the matching status of other users. The interdependent preferences between players are called externalities, that is, the player's preferences change dynamically with the matching status of other players in the same group. Therefore, this embodiment considers the concept of exchange matching to deal with externalities. By searching for exchange blocking pairs in the matching set x, the matching relationship between the base station and the user in the matching set x is adjusted to achieve a stable one-to-many matching result. When there is no exchange blocking pair in the matching set x, the current matching set is used as the user base station association strategy, that is, a stable matching relationship in the current matching set is maintained.

[0056] In some embodiments, the user preference list is composed of the utility vector of each user after the initial matching, and the utility vector of the user is specifically:

[0057] ψ k (x) = r k (x k ,Y,P,φ)

[0058] Among them, r k (x k ,Y,P,φ) is the uplink rate obtained by the kth user based on the initialization matching, x k represents the association matrix between the kth user and the base station, Y represents the subband allocation matrix, P represents the power allocation matrix, φ represents the phase shift design matrix, and the utility vectors of the users in the user preference list are arranged in descending order.

[0059] In some embodiments, the base station preference list is composed of the utility vector of each base station after the initialization matching, and the utility vector of the base station is specifically:

[0060]

[0061] in, represents the total energy consumption of all users connected to the nth base station due to computing tasks in a large-scale time slot, X represents the user-base station association matrix, z n represents the computational subtasks of all the users associated with the nth base station received therefrom, Y represents the subband allocation matrix, P represents the power allocation matrix, φ represents the phase shift design matrix, and the utility vectors of the base stations in the base station preference list are arranged in descending order.

[0062] Specifically, each user's goal is to associate with a base station that can achieve its maximum uplink rate. Therefore, the user's matching preference for base stations is based on the uplink rate in descending order. The base station's preference list is defined based on minimizing the energy consumption among all users. Therefore, the base station's matching preference for users is based on the negative energy consumption in descending order.

[0063] The optimization goal in this application is to minimize the system energy consumption over the entire time range while meeting the user delay constraints, so the optimization goal can be expressed as follows:

[0064]

[0065]

[0066]

[0067]

[0068]

[0069]

[0070]

[0071] In formula (1-1), is the user-base station association matrix, where the kth column of the matrix The nth element x in the kth column of the matrix k,n It is represented as the association variable between user k and base station n, x k,n ∈{0,1},x k,n =1 indicates that user k is connected to base station n, otherwise it is 0. Assign a matrix to the subband, where the kth column of the matrix The bth element y in the kth column of the matrix k,b represents the subband allocation variable, y k,b ∈{0,1},y k,b =1 indicates that subband b is allocated to user k, otherwise it is 0. is the power allocation matrix, where the kth column of the matrix The bth element p in the kth column of the matrix k,b represents the transmit power on subband b, Where power is discrete, p d is the number of discrete levels, is the maximum transmit power of user k. is the phase shift design matrix. Considering a discrete phase shift design with ρ phases, the phase shift φ of the mth reflection element is m Defined as To calculate the unloading matrix task queue, the set of unloading decision actions in the nth column of the matrix is ​​represented as z n ={z 1,n ,…,z l,n}, the i′th element z in the nth column of the matrix i′,n ∈{0,1} is the offloading action variable of the i′th user subtask. When the subtask is offloaded to the edge computing server, z i′,n = 1, when the subtask is executed locally, z i′,n =0.

[0072] Represents a small-scale time slot The total energy consumption of all users connected to base station n due to computing tasks, including transmission energy consumption (only uplink transmission is considered) and computing energy consumption.

[0073]

[0074] Where l is the maximum length of the user subtask queue received by base station n. The energy consumption of subtask i′ of base station n is Defined as:

[0075]

[0076] ζ mob and e They represent the effective capacitance coefficients determined by the chip structures of the user equipment and the MEC server, respectively. In this embodiment, only the uplink computing delay and energy consumption are concerned, and the downlink delay and energy consumption are not considered. represents the input data size of subtask i′, Indicates the number of clock cycles executed by the microprocessor for each byte of data processed. is the computing power (CPU frequency) of user device k, is the computing power (CPU frequency) of a CPU core of the edge computing server connected to base station n. The transmit power on subband b is represented by p k,b The achievable rate of user k connected to base station n Defined as

[0077]

[0078] W is the subband bandwidth. It is represented as the channel frequency response (CFR) between user k and the base station on subband b, where is represented by the discrete Fourier transform (DFT) matrix of the bth row. k The zero-padded concatenated channel between user k and IRS and macro base station path is uniformly represented as follows: Γ is the gap between the channel capacity and the specific modulation and coding scheme. The receiver noise of each subband is modeled as having zero mean and variance σ 2 Independent circularly symmetric complex Gaussian (CSCG) random variables.

[0079] Formula (1-1, a) means that for all user sets Any user k in can only be associated with one base station. Formula (1-2, b) indicates that for all base stations connected The user set of any base station m For all subband sets To ensure that one subband of each small-scale time slot can only be allocated to at most one user in any base station, K m is the number of users connected to base station m. Formula (1-1,c) means that for all user sets Each user must be assigned at least one subband. Formula (1-1, d) means that for all user sets The power of all subbands connected by the user does not exceed the maximum transmit power of the user. Formula (1-1, e) reflects the The computing power of edge computing servers is limited and can only serve C n users, the number of CPU cores of the server of base station n is C n Formula (1-1, f) ensures that in large-scale time slots, all user sets The computing task delay of user k connected to base station n does not exceed the maximum tolerable delay of the user application, where Indicates that in small time slots The execution delay of the subtask of user k connected to base station n (including uplink transmission delay and calculation delay) is defined as

[0080]

[0081] In some embodiments, the exchange blocking pair is specifically defined as:

[0082] Two matching subsets are randomly selected from the matching set, each of the matching subsets includes one of the users and one of the base stations associated therewith. After matching and exchanging the two matching subsets, if the value of each element in the user preference lists of all the users and the base station preference lists of all the base stations in the two matching subsets does not change or the value of at least one element increases, then the two users in the two matching subsets are defined as the exchange blocking pair.

[0083] For example, for the matching x, (k, k′) is a pair of exchange blocking pairs if and only if the two users exchange their respective matching base stations, the utility value of each utility vector in the user preference list and the base station preference list will not decrease, and at the same time, at least one of the utility values ​​of all utility vectors in the user preference list and the base station preference list will increase. Since the preference lists are arranged in descending order, the larger the utility value, the more obvious the preference of the user and the base station. If the utility value can be increased after adjusting the matching of the user and the base station, the matching relationship is more reasonable, and the matching relationship between the base station and the user is swapped until there is no exchange blocking pair, at which time the matching is bilaterally exchange stable.

[0084] In some embodiments, the matching exchange is specifically:

[0085] The two users in the two matching subsets will exchange the base stations associated therewith and the other matching subsets in the matching set will not change.

[0086] For example, given a one-to-many matching x with user k connected to base station n and user k′ connected to base station n′, a swap matching allows a pair of users (k, k′) to swap their matched base stations (n, n′) while keeping the other user-base station matchings unchanged.

[0087] In summary, the matching algorithm is shown in Table 1

[0088] Table 1 Matching algorithm

[0089]

[0090] The matching algorithm first initializes a match x and calculates the utility of the user and the base station to obtain the preference list of both the user and the base station. Then the cyclic iteration process begins, looking for exchange blocking pairs and updating the exchange matching and calculating the utility until bilateral exchange stability is achieved and the user-base station association strategy is determined. Each matching exchange operation strictly reduces the system energy consumption and generates a new match. Since the number of users and base stations is limited, it will converge to a stable match after a limited number of iterations, ensuring the convergence of the matching algorithm. In this state, the exchange of any two users will not reduce the system energy consumption, so a local optimal solution is reached.

[0091] In some embodiments, reference Figure 4 The macro base station determines the computing offloading strategy, resource allocation strategy and phase shift design strategy using a deep Q network algorithm based on the computing task request, the channel state in the edge network and the currently available resources in the edge network, with the optimization goal of minimizing the total energy consumption of the edge network and satisfying the delay limit of the computing task request, that is, step S12, including:

[0092] Step S121: Modeling the determination process of each strategy as a Markov decision process, and constructing a deep neural network model with the environment of the edge network as the state and the macro base station as the intelligent agent;

[0093] Step S122, pre-training the deep neural network model using a training set;

[0094] Step S123: input the current state of the edge network into a pre-trained deep neural network model, and output a Q function through the deep neural network model using a deep Q network algorithm;

[0095] Step S124: selecting an optimal action based on the Q function through an ε-greedy strategy to serve as the computation offloading strategy, the resource allocation strategy, and the phase shift design strategy.

[0096] In some embodiments, the state includes a channel state, a computing task queue of the user, and current computing resources of the edge network.

[0097] In some embodiments, the actions include computational offloading actions, subband allocation actions, power allocation actions, and IRS phase shift design actions,

[0098] The selecting the optimal action based on the Q function through the ε-greedy strategy as the calculation offloading strategy, the resource allocation strategy and the phase shift design strategy includes:

[0099] The computation offloading action in the optimal action is used as the computation offloading strategy, the subband allocation action and the power allocation action in the optimal action are used as the resource allocation strategy, and the IRS phase shift design action in the optimal action is used as the phase shift design strategy.

[0100] Specifically, this embodiment uses a deep reinforcement learning algorithm to solve the small time scale optimization problem. At this time, the intelligent agent (macro base station) continuously interacts with the environment and evaluates the quality of the actions taken (computational offloading, subband allocation, power allocation, and IRS phase shift design actions) based on the feedback from the environment, i.e., rewards, so as to continuously improve the strategy until the optimal solution for the actions in each state is found. First, the state, action, and reward definitions of reinforcement learning in this optimization problem are given.

[0101] State: The state of the agent in a small-scale time slot includes the channel state, the task queue of the user's computing task request, and the computing resources in the current edge network;

[0102] Action: In small time slots The action of the agent is defined as

[0103]

[0104] They represent subband allocation, power allocation, IRS phase shift design, and computational offloading actions, respectively.

[0105] Reward: The weighted sum of the negative energy consumption and delay penalties for each small-scale time slot.

[0106] In order to minimize the long-term energy consumption of the IRS-assisted MEC system, the DQN algorithm is used to learn computation offloading, resource allocation, and IRS phase shift design strategies. The agent can π (s, a)(Q function) optimization strategy π can use a deep neural network (DNN) to approximate the value function, that is, DNN parameterizes a value function approximation function Q(s, a; θ i ), called a Q network, where s is the state, a is the action, and θ i is the weight of the Q network at the i-th iteration. The input of the neural network is the state s, and then after two fully connected layers, the output is the value function of all actions corresponding to the input state s. The neural network is trained through some training sets (the training set is the data obtained by the interaction between the agent and the environment), simulating a function (value function approximation) to improve the estimation accuracy, thereby accelerating the convergence speed of the reinforcement learning algorithm.

[0107] In DQN, using experience replay requires a data set to store the agent's experience, called the experience cache unit D. t={e1,…,e t}, the experience e at each step t =(s t ,a t ,r t ,s t+1 ), including the current state, action, reward, and next state. In DQN, the Q learning update method is applied to randomly and uniformly sample a batch of data (s, a, r, s′) in the experience cache unit, called Minibatch, and use the following loss function L i (θ i ) Update the neural network parameters:

[0108]

[0109] in, represents the expectation, U(D) represents the uniform sampling of data (s,a,r,s′) in the experience buffer unit D. γ is the discount factor. a′ is the action corresponding to the maximum value function in the next state s′. There are two networks in the update, Q network Q(s,a;θ i ) and the target network in is the parameter used to calculate the target network at the i-th iteration, θ i and The update frequency of the target network is different. Follow the parameters θ of the Q network every C steps i Update, the other times are fixed, and θ i Each step is updated. Training the Q network is the process of updating the parameters θ with the goal of minimizing the loss function. j is called a temporal difference (TD) target. Therefore, the target Y is generated using an older set of parameters j The time to update the Q network is the same as the time to update the TD target Y j The time delay is increased between θ and θ, thus solving the problem of non-convergence or oscillation of reinforcement learning. The loss function is optimized using the gradient descent algorithm. The loss function is about the parameter θ. i Gradient for:

[0110]

[0111] The computational offloading, resource allocation and IRS phase shift design algorithm based on DQN are shown in Table 2. First, the experience cache unit D, Q network and target network are initialized. The process from the initial state of the interaction between the agent and the environment to the terminal state is represented as an episode, and the total number of episodes is M. For each episode, the following steps are performed: Initialize the initial state s1 of each episode. For each step t of the episode (the total number of steps is T), use the ε-greedy strategy to select an action, and then the agent performs action a t , observe the reward r t And transfer to the next state s t+1 . This experience (s t ,a t ,r t ,s t+1 ) is stored in the experience cache unit D, a batch of experience is uniformly randomly sampled from D, and gradient descent is performed to update the Q network parameters θ. Every C steps, The network updates its parameters θ using the parameters θ - .

[0112] Table 2 Computation offloading, resource allocation and IRS phase shift design algorithm based on DQN

[0113]

[0114] Based on Algorithm 2, the deep neural network model is pre-trained through the training set to obtain a pre-trained Q network, the current state of the edge network is input into the pre-trained Q network, and the Q function is output through the Q network. Based on the Q function, the optimal action is selected through the ε-greedy strategy as the computation offloading strategy, the resource allocation strategy and the phase shift design strategy, specifically, the computation offloading action in the optimal action is selected as As the computation offloading strategy, the subband allocation action in the optimal action is and the power distribution action As the resource allocation strategy, the IRS phase shift design action in the optimal action As the phase shift design strategy.

[0115] The resource allocation method (dual time scale algorithm) proposed in this embodiment is shown in Table 3.

[0116] Table 3 Dual time scale algorithm

[0117]

[0118]

[0119] As shown in Table 3, the large-scale time is divided into large-scale time slots, and the large-scale time slots are divided into small-scale time slots. In each large-scale time slot, the user base station association strategy is obtained by Algorithm 1, and the user-base station association strategy is executed. Since the user equipment can only be associated with one base station, and the base station can serve multiple users, this embodiment considers a one-to-many matching algorithm to associate each base station with multiple user equipment. After that, the calculation offloading, resource allocation and IRS phase shift design strategy on each small-scale time slot are optimized by using Algorithm 2 with a deep reinforcement learning algorithm. After completing the iteration on the large-scale time, all user base station association strategies, calculation offloading strategies, resource allocation strategies and IRS phase shift design strategies are obtained.

[0120] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0121] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application further provides a resource allocation system based on an intelligent reflective surface-assisted edge network, wherein the edge network includes multiple base stations and multiple users, the multiple base stations include a macro base station and multiple small base stations, the user communicates directly with the small base station or communicates with the macro base station using an intelligent reflective surface to assist, and the macro base station is configured as follows:

[0122] For each large-scale time slot in the large-scale time, in response to receiving a computing task request sent by the user, the macro base station calculates a user base station association strategy using a matching algorithm based on the computing task request and all channels in the edge network; and

[0123] For each small-scale time slot, the macro base station uses a deep Q network algorithm to determine a computing offloading strategy, a resource allocation strategy, and a phase shift design strategy based on the computing task request, the channel state in the edge network, and the currently available resources in the edge network, with the total energy consumption of the edge network being minimized and the delay limit of the computing task request being met as the optimization goal;

[0124] Based on all of the user base station association strategies, the computation offloading strategies, the resource allocation strategies and the phase shift design strategies, the macro base station performs resource allocation on the user,

[0125] The large-scale time is divided into a group of large-scale time slots, and each large-scale time slot is divided into a group of small-scale time slots.

[0126] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0127] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0128] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A resource allocation method based on intelligent reflective surface assisted edge network, characterized in that: The edge network includes multiple base stations and multiple users, the multiple base stations include a macro base station and multiple small base stations, the user communicates directly with the small base station or uses a smart reflective surface to assist in communicating with the macro base station, and the resource allocation method includes: For each large-scale time slot in the large-scale time, in response to receiving a computing task request sent by the user, the macro base station uses a matching algorithm to calculate a user base station association strategy based on the computing task request and all channels in the edge network, including: initializing matching of all the users and all the base stations in the edge network to obtain a matching set, and calculating a user preference list of each user and a base station preference list of each base station based on the computing task request and all channels; Determining whether there is a switch blocking pair in the matching set based on the user preference list and the base station preference list; In response to determining that the exchange blocking pair exists, two base stations respectively associated with two users in the exchange blocking pair are exchanged, In response to determining that the exchange blocking pair does not exist, using the current matching set as the user base station association strategy; The user preference list is composed of the utility vector of each user after the initial matching, and the utility vector of the user is specifically ψ k (x) = r k (x k , Y, P, φ) Among them, r k (x k , Y, P, φ) is the uplink rate obtained by the kth user based on the initialization matching, x k represents the association matrix between the kth user and the base station, Y represents the subband allocation matrix, P represents the power allocation matrix, φ represents the phase shift design matrix, and the utility vectors of the users in the user preference list are arranged in descending order; The base station preference list is composed of the utility vector of each base station after the initialization matching, and the utility vector of the base station is specifically: in, represents the total energy consumption of all users connected to the nth base station due to computing tasks in a large-scale time slot, X represents the user-base station association matrix, z n represents the computational subtasks of all the users associated with the nth base station received therefrom, Y represents a subband allocation matrix, P represents a power allocation matrix, φ represents a phase shift design matrix, and the utility vectors of the base stations in the base station preference list are arranged in descending order; The exchange blocking pair is specifically defined as: Randomly select two matching subsets from the matching set, each matching subset includes one of the users and one of the base stations associated therewith, and after matching and exchanging the two matching subsets, if the value of each element in the user preference lists of all the users and the base station preference lists of all the base stations in the two matching subsets does not change or the value of at least one element increases, then the two users in the two matching subsets are defined as the exchange blocking pair; The matching exchange is specifically: The two users in the two matching subsets will exchange the base stations associated with them and the other matching subsets in the matching set will not change; as well as, For each small-scale time slot, the macro base station uses a deep Q network algorithm to determine a computing offloading strategy, a resource allocation strategy, and a phase shift design strategy based on the computing task request, the channel state in the edge network, and the currently available resources in the edge network, with the total energy consumption of the edge network being minimized and the delay limit of the computing task request being met as the optimization goal; Based on all of the user-base station association strategies, the computation offloading strategies, the resource allocation strategies, and the phase shift design strategies in large-scale time, the macro base station performs resource allocation on the user, The large-scale time is divided into a group of large-scale time slots, and each large-scale time slot is divided into a group of small-scale time slots.

2. The method according to claim 1, characterized in that The macro base station determines a computing offloading strategy, a resource allocation strategy, and a phase shift design strategy using a deep Q network algorithm based on the computing task request, the channel state in the edge network, and the currently available resources in the edge network, with the optimization goal of minimizing the total energy consumption of the edge network and satisfying the delay limit of the computing task request, including: The determination process of each strategy is modeled as a Markov decision process, and a deep neural network model is constructed with the environment of the edge network as a state and the macro base station as an intelligent agent; Pre-training the deep neural network model using a training set; Inputting the current state of the edge network into a pre-trained deep neural network model, and outputting a Q function via the deep neural network model using a deep Q network algorithm; Based on the Q function, an optimal action is selected through an ε-greedy strategy to serve as the computation offloading strategy, the resource allocation strategy and the phase shift design strategy.

3. The method according to claim 2, characterized in that The state includes the channel state, the computing task queue of the user and the current computing resources of the edge network.

4. The method according to claim 2, characterized in that: The actions include calculation unloading action, subband allocation action, power allocation action and IRS phase shift design action, The selecting the optimal action based on the Q function through the ε-greedy strategy as the calculation offloading strategy, the resource allocation strategy and the phase shift design strategy includes: The computation offloading action in the optimal action is used as the computation offloading strategy, the subband allocation action and the power allocation action in the optimal action are used as the resource allocation strategy, and the IRS phase shift design action in the optimal action is used as the phase shift design strategy.

5. A resource allocation system based on intelligent reflective surface assisted edge network, characterized in that: The edge network includes multiple base stations and multiple users, the multiple base stations include a macro base station and multiple small base stations, the users communicate directly with the small base stations or use smart reflective surfaces to assist in communicating with the macro base station, and the macro base station is configured as follows: For each large-scale time slot in the large-scale time, in response to receiving a computing task request sent by the user, the macro base station uses a matching algorithm to calculate a user base station association strategy based on the computing task request and all channels in the edge network, including: initializing matching of all the users and all the base stations in the edge network to obtain a matching set, and calculating a user preference list of each user and a base station preference list of each base station based on the computing task request and all channels; Determining whether there is a switch blocking pair in the matching set based on the user preference list and the base station preference list; In response to determining that the exchange blocking pair exists, two base stations respectively associated with two users in the exchange blocking pair are exchanged, In response to determining that the exchange blocking pair does not exist, using the current matching set as the user base station association strategy; The user preference list is composed of the utility vector of each user after the initial matching, and the utility vector of the user is specifically ψ k (x) = r k (x k , Y, P, φ) Among them, r k (xk, Y , P, φ) is the uplink rate obtained by the kth user based on the initialization matching, x k represents the association matrix between the kth user and the base station, Y represents the subband allocation matrix, P represents the power allocation matrix, φ represents the phase shift design matrix, and the utility vectors of the users in the user preference list are arranged in descending order; The base station preference list is composed of the utility vector of each base station after the initialization matching, and the utility vector of the base station is specifically: in, represents the total energy consumption of all users connected to the nth base station due to computing tasks in a large-scale time slot, X represents the user-base station association matrix, z n represents the computational subtasks of all the users associated with the nth base station received therefrom, Y represents a subband allocation matrix, P represents a power allocation matrix, φ represents a phase shift design matrix, and the utility vectors of the base stations in the base station preference list are arranged in descending order; The exchange blocking pair is specifically defined as: Randomly select two matching subsets from the matching set, each matching subset includes one of the users and one of the base stations associated therewith, and after matching and exchanging the two matching subsets, if the value of each element in the user preference lists of all the users and the base station preference lists of all the base stations in the two matching subsets does not change or the value of at least one element increases, then the two users in the two matching subsets are defined as the exchange blocking pair; The matching exchange is specifically: The two users in the two matching subsets will exchange the base stations associated with them and the other matching subsets in the matching set will not change; and, For each small-scale time slot, the macro base station uses a deep Q network algorithm to determine a computing offloading strategy, a resource allocation strategy, and a phase shift design strategy based on the computing task request, the channel state in the edge network, and the currently available resources in the edge network, with the total energy consumption of the edge network being minimized and the delay limit of the computing task request being met as the optimization goal; Based on all of the user-base station association strategies, the computation offloading strategies, the resource allocation strategies, and the phase shift design strategies in large-scale time, the macro base station performs resource allocation on the user, The large-scale time is divided into a group of large-scale time slots, and each large-scale time slot is divided into a group of small-scale time slots.

Images