A device energy saving method in a cellular edge computing environment

Abstract

The present invention provides a device energy-saving method in a cellular edge computing environment, constructing an edge computing network including a macro base station, wherein the coverage area of ​​the macro base station includes multiple user terminals and a small base station deployed with an edge server and multiple radio frequency units, and each user terminal is connected to any radio frequency unit in the small base station through a radio frequency unit; constructing a power consumption minimization problem model and constraint conditions; describing the power consumption minimization problem model as a distributed partially observable Markov decision process; inputting all local states in the state space of the distributed partially observable Markov decision process into a trained decision power consumption allocation model for decision allocation, and obtaining the switching state and power consumption of each device in the small base station; while meeting the user experience quality requirements, dynamically adjusting the switching state and power consumption of each radio frequency unit and edge server in the small base station, thereby minimizing the total energy consumption of the device.

Description

Technical Field

[0001] The present invention relates to the technical field of network resource management, and in particular to a device energy saving method in a cellular edge computing environment. Background Art

[0002] With the rapid development of mobile Internet and Internet of Things, the demand for mobile communication services has shown an unprecedented growth trend. In this context, Ultra-Dense Network (UDN), as a key technology in the future wireless communication field, has attracted much attention because of its ability to significantly improve throughput. UDN not only reduces path loss but also improves transmission quality by densely deploying a large number of small cells within the coverage of macro cells. In addition, UDN greatly improves the reuse rate of spectrum resources per unit area, thereby significantly improving the throughput of wireless networks. Due to the uneven distribution of traffic generated by network communication services in time and space, this densely deployed network architecture will also lead to a huge waste of resources under low traffic load. In addition to the initial investment cost of network construction, the energy consumption cost of base station operation has become an issue that cannot be ignored. Due to the limited coverage and dense deployment of small base stations, mobile users may experience frequent switching, which not only affects the throughput of the network, but also generates additional energy consumption. At the same time, the high power consumption characteristics of millimeter wave / terahertz circuits (such as power amplifiers, analog RF devices, and baseband processors) and the excessive data links between small base stations and user terminals have led to a surge in network energy consumption. Currently, there is a significant mismatch between the total amount of network energy consumption and the actual amount of energy consumption required to process and transmit user traffic loads, which has led to a sharp decline in network energy efficiency and a significant increase in operating expenses, reducing the profitability of network operators. Therefore, improving energy efficiency and reducing network energy consumption have become the key to ensuring the sustainability of wireless networks. When the demand for data transmission is low, switching the base station to sleep mode or turning off the transceiver is an effective strategy to reduce energy consumption. Therefore, it is crucial to reasonably adjust the working mode of small base stations for the energy efficiency of the entire network, and related work has also focused on this issue.

[0003] With the deployment of edge computing platforms in cellular scenarios, the computationally intensive and data-intensive tasks of user terminals can be offloaded to edge computing platforms. Therefore, in addition to the demand for accessing remote Internet services, there is also a large demand for accessing edge computing platforms to request computing services. The demand for computing services also has an uneven distribution in time and space, which is rarely addressed in existing work when considering equipment energy saving solutions. In addition, in the study of base station energy consumption optimization, base station switch models are more commonly used, but the binary mode of this type of method is not accurate enough to optimize the energy consumption of base stations, and cannot fully utilize the uniqueness of directional transmission in millimeter wave networks. Finally, with the increasing number of base stations and edge servers in cellular environments, traditional mathematical modeling methods take too long to solve these problems and are difficult to meet the requirements of practical applications. Although machine learning has shown great potential in solving such problems, supervised learning and unsupervised learning methods usually rely on a large number of offline training samples, and it is often difficult to obtain complete data sets in reality, which limits the effectiveness of these methods in practical applications. The deep reinforcement learning (DRL) method, with its unique advantages, provides a new idea for energy consumption optimization in cellular edge computing environments. It can dynamically acquire data and perform model training through real-time interaction with the environment without any training samples, which makes the DRL method more suitable for solving problems such as energy consumption optimization in cellular edge computing environments. Summary of the invention

[0004] The present invention provides a device energy-saving method in a cellular edge computing environment, the purpose of which is to reduce the total energy consumption of the network while meeting the user experience quality requirements.

[0005] In order to achieve the above object, the present invention provides a device energy saving method in a cellular edge computing environment, comprising:

[0006] Step 1: Build an edge computing network including a macro base station. The coverage area of ​​the macro base station includes multiple user terminals and a small base station with an edge server and multiple radio frequency units deployed. Each user terminal is connected to any radio frequency unit in the small base station through its own radio frequency unit to access network services or offload tasks to the edge server.

[0007] Step 2: Based on the edge computing network, a power consumption minimization problem model of total power consumption is constructed, and constraints are constructed for the power consumption minimization problem model. The total power consumption includes the power consumption of small base stations, user terminals, and edge servers.

[0008] Step 3: Take the small base station as the agent, all action sets of the agent as the action space, all local state sets of the agent as the state space, and construct a reward function to describe the power consumption minimization problem model as a distributed partially observable Markov decision process;

[0009] Step 4: Input all local states in the state space into the trained decision power consumption allocation model for decision allocation, and obtain the switch state and power consumption allocation results of each device in the coverage area of ​​the macro base station. The devices in the coverage area of ​​the macro base station include each radio frequency unit and edge server of the small base station and the user terminal;

[0010] Step 5: According to the switch status and power consumption allocation results of each device in the coverage area of ​​the macro base station, the power consumption of the radio frequency unit, edge server and user terminal in the small base station is regulated to achieve equipment energy saving.

[0011] The above scheme of the present invention has the following beneficial effects:

[0012] Compared with the prior art, the present invention constructs an edge computing network including a macro base station, wherein the coverage area of ​​the macro base station includes multiple user terminals and a small base station with an edge server and multiple radio frequency units deployed, wherein each user terminal is connected to any radio frequency unit in the small base station through its own radio frequency unit for accessing network services or offloading tasks to the edge server; a power consumption minimization problem model of total power consumption is constructed, and constraints are constructed for the power consumption minimization problem model; the small base station is taken as an intelligent agent, all action sets of the intelligent agent are taken as an action space, and all local state sets of the intelligent agent are taken as a state space, and a reward function is constructed to describe the power consumption minimization problem model as a distributed partially observable Markov decision process; all local states in the state space are input into a trained decision power consumption allocation model for decision allocation, and the switch state and power consumption allocation results of each device in the coverage area of ​​the macro base station are obtained; according to the switch state and power consumption allocation results of each device in the coverage area of ​​the macro base station, the power consumption of the radio frequency unit, the edge server and the user terminal in the small base station is regulated to achieve equipment energy saving; while meeting the user experience quality requirements, the switch state and power consumption of each radio frequency unit and edge server in the small base station are dynamically regulated through the trained decision power consumption allocation model, thereby minimizing the total energy consumption of the device.

[0013] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of a flow chart of an embodiment of the present invention;

[0015] Figure 2 A schematic diagram of an application scenario of an embodiment of the present invention;

[0016] Figure 3 It is a structural diagram of a state completion module based on GRU in an embodiment of the present invention;

[0017] Figure 4 A schematic diagram comparing the average reward functions under various schemes during model training;

[0018] Figure 5 This is a comparison diagram of the average constraint satisfaction rate of the system under various schemes during model training;

[0019] Figure 6 This is a schematic diagram comparing the average power consumption under various schemes during model training. DETAILED DESCRIPTION

[0020] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a locking connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0022] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0023] In view of the existing problems, the present invention provides a device energy saving method in a cellular edge computing environment.

[0024] like Figure 1 As shown, an embodiment of the present invention provides a device energy saving method in a cellular edge computing environment, including:

[0025] Step 1: Build an edge computing network including a macro base station. The coverage area of ​​the macro base station includes multiple user terminals and a small base station with an edge server and multiple radio frequency units deployed. Each user terminal is connected to any radio frequency unit in the small base station through its own radio frequency unit to access network services or offload tasks to the edge server.

[0026] Step 2: Based on the edge computing network, a power consumption minimization problem model of total power consumption is constructed, and constraints are constructed for the power consumption minimization problem model. The total power consumption includes the power consumption of small base stations, user terminals, and edge servers.

[0027] Step 3: Take the small base station as the agent, all action sets of the agent as the action space, all local state sets of the agent as the state space, and construct a reward function to describe the power consumption minimization problem model as a distributed partially observable Markov decision process;

[0028] Step 4: Input all local states in the state space into the trained decision power consumption allocation model for decision allocation, and obtain the switch state and power consumption allocation results of each device in the coverage area of ​​the macro base station. The devices in the coverage area of ​​the macro base station include each radio frequency unit and edge server of the small base station and the user terminal;

[0029] Step 5: According to the switch status and power consumption allocation results of each device in the coverage area of ​​the macro base station, the power consumption of the radio frequency unit, edge server and user terminal in the small base station is regulated to achieve equipment energy saving;

[0030] The power consumption control result is as follows: according to the number of user terminals that successfully request resources, an equal number of RF units in the small base station are controlled to be in the on state, the edge server is in the on state, and the remaining RF units in the small base station are controlled to be in the off state; when no user terminal requests resources, all RF units in the edge server and the small base station are controlled to be in the off state.

[0031] In the embodiment of the present invention, the network architecture considered is different from the typical millimeter wave cellular network in the following ways:

[0032] The millimeter wave (mmWave) cellular network that cooperates with small base stations (SBS) in the coverage area of ​​a macro base station (MBS) adds edge servers. The macro base station is responsible for controlling information transmission and backhaul data transmission, while the small base station is responsible for access data transmission.

[0033] Each small cell has a small base station with an edge server deployed;

[0034] The embodiment of the present invention considers a bidirectional (uplink / downlink) transmission mode;

[0035] In the network, each small base station is deployed with N radio frequency units, each user terminal is equipped with a radio frequency unit, and each radio frequency unit is equipped with a millimeter wave antenna;

[0036] There is a millimeter wave backhaul link between each small base station and the macro base station, and each user terminal can also interact with the macro base station through the frequency band below 6GHz.

[0037] In order to focus on the switching problem of the small base station radio frequency unit and the edge server, the embodiment of the present invention assumes that the beam direction of the millimeter wave antenna of the small base station is fixed and the beam width does not overlap; at the same time, it is assumed that the beam width of each user terminal is fixed, and its beam direction is aligned with the small base station associated with it. Therefore, the small base station only needs one channel resource to serve N user terminals at the same time. In addition, in order to avoid introducing unnecessary complexity when solving the switching problem, it is assumed that the channels used by the user terminals associated with the small base station are mutually orthogonal to avoid mutual interference.

[0038] Specifically, user terminals can be divided into two types: Remote Service Request User Equipment (RS-UE), which is a user that remotely accesses the Internet by accessing a small base station, and Local Resource Request User Equipment (LR-UE), which requests edge computing resources by accessing a small base station and offloads computing tasks to a nearby edge server. As long as there is an active radio frequency unit in the small base station, the user terminal can easily apply for computing resources from the edge server.

[0039] If an edge server receives a resource request from a local resource requesting user terminal, it creates an execution environment based on the resource requirements of the local resource requesting user terminal by adopting virtualization technology. The resource requirements can be the central processing unit (CPU), random access memory (RAM), storage, and input / output interface (I / O).

[0040] Without loss of generality, consider a common situation where processor capability is the main resource limitation of the edge server. Usually, the uplink request data packet of the remote service requesting user terminal is small. Therefore, in order to simplify the switching problem, the embodiment of the present invention ignores the power consumption generated by its uplink transmission. Similarly, given that the downlink response data packet of the local resource requesting user terminal is also small, the power consumption generated by its downlink transmission is ignored during performance analysis. These request and response processes occur after the user terminal successfully requests and obtains the resources scheduled and allocated by the macro base station through the macro base station. In each time slot, the macro base station needs to synchronously update the status information of the user terminal associated with the small base station.

[0041] In an embodiment of the present invention, the normalized beam gain of the millimeter wave communication system is given by the following formula:

[0042]

[0043] in, represents the mainlobe beamwidth in radians, Indicates the beam deviation angle in radians, that is, the angle between the transceiver connection and the antenna centerline or the main lobe pointing direction. Represents the sidelobe gain, which is a small positive number;

[0044] Take the millimeter wave antenna directional uplink as an example, that is, the user terminal To small base station RF unit ,Right now , if the user terminal Beams and small base stations The RF unit in If the beam is aligned, the directional transmit and receive gains are given by the following formulas:

[0045]

[0046] in, represents the directional transmission gain of the uplink, represents the uplink receiving gain, represents the transmit beamwidth of the user terminal, Indicates the receive beam width of the radio frequency unit in the small base station;

[0047] The uplink channel gain is given by:

[0048]

[0049] in, Represents the total number of paths, Indicates the path index, which is used to indicate the path between uplinks. , Respectively represent user terminals From Paths to small base stations The RF unit in Directional transmission gain and receiving gain, represents the Dirac function, represents the propagation delay of the path, which is expressed as:

[0050]

[0051] in, Indicates The distance of the path, represents the speed of light;

[0052] To simplify the analysis, assume that When the value of is 1, it represents a line-of-sight (LOS) path, that is, a direct, obstacle-free path between the source and the receiving point. Based on this assumption, Values ​​from 2 to All other paths are considered non-line-of-sight (NLOS) paths, that is, indirect paths that may experience reflection, diffraction, or direct radiation. For line-of-sight paths, the user terminal From Paths to small base stations The RF unit in Amplitude It can be given by the following formula:

[0053]

[0054] in, represents the wavelength, Indicated in On the path through The reflection coefficient at the second reflection is is the number of reflections on the path. Since the reflection loss of millimeter waves is very high, only one reflection on a given path can be considered, that is, RPH=1. Therefore, the formula can be simplified to:

[0055]

[0056] The channel gain can be given by:

[0057]

[0058] In the embodiment of the present invention, the transmission power consumption of the directional light beam in the uplink is: , then determine the receiving power consumption , receiving power consumption The expression is:

[0059]

[0060] In the embodiment of the present invention, each user terminal can only be connected to one small base station at the same time. According to the above formula, the uplink signal-to-noise ratio (SNR) can be derived as follows:

[0061]

[0062] in, and denote the bandwidth of the millimeter wave antenna and the power consumption spectrum density of the environmental noise respectively. The achievable data rate of the uplink can be expressed as follows:

[0063]

[0064] For mmWave directional downlinks (e.g. ), the corresponding communication parameters , , , , , Estimated by a formula similar to that for the uplink. During this period, Taking the channel gain, transmission power, signal-to-noise ratio and data rate as an example, , , ,as well as They are expressed in detail as , , as well as .

[0065] Specifically, the total power consumption of the edge computing network includes the power consumption of the edge server, small base station, and all devices associated with the small base station. The power consumption expression is:

[0066] ;

[0067] in, Indicates that in the time slot The total power consumption of the internal edge computing network, , Both represent binary indicator variables, Indicates that in the time slot Internal small base station The The transmission power consumption of each radio unit, Indicates that in the time slot User terminal within For small base stations The transmission power consumption is Indicates that in the time slot Deployed in small base stations The edge server receives the The maintenance power consumption of each RF unit when unloading tasks, Indicates that in the time slot Internal small base station Maintenance power consumption, Indicates that in the time slot Internal small base station The The mode switching power consumption of each RF unit, Indicates that in the time slot Internal small base station The mode switching power consumption of the edge server in Indicates user terminal In the time slot Offload tasks to small base stations The edge server in the computing power consumption of the edge server, Indicates small base station The RF units, represents the user terminal, Indicates the total number of user terminals, Indicates the number of radio frequency units.

[0068] In the embodiment of the present invention, the total power consumption of the edge computing network includes the power consumption of the small base station, that is, the energy consumption per unit time. The embodiment of the present invention reduces the overall consumption required for its long-term operation by reducing the power consumption of the small base station. The small base station includes three parts of power consumption. The first part is the transmission power consumption of the radio frequency unit. The second part is the maintenance power consumption consumed by the power supply, signal processing and air conditioning equipment of the small base station. The third part is the mode switching power consumption, that is, the power consumption when the radio frequency unit in the small base station switches from on to off or from off to on. Specifically, the radio frequency unit in the small base station In the time slot The transmission power consumption during this period is:

[0069]

[0070] in, Represents a binary vector, defined as if the small base station The RF unit in In the open state ,otherwise , Indicates the amplifier efficiency of the small base station, with a value between 0 and 1;

[0071] Due to the limitation of amplifier efficiency, only about 40% to 50% of the total transmission power consumption can actually be used for signal transmission, and the remaining energy is dissipated in the form of heat. For example, in a macro base station, 50% of the total power consumption is used for transmission power consumption, and in a small base station, 50% of the total power consumption is used for transmission power consumption. The RF unit in The maintenance power consumption is:

[0072]

[0073] in, and Small base stations The RF unit in In the power consumption in the on and off states, the maintenance power consumption of the small base station accounts for nearly 35% of the total power consumption; if the small base station switches from the "on" state to the "off" state or vice versa, mode conversion power consumption will occur. , small base station RF unit The mode switching power consumption can be expressed as:

[0074]

[0075] in, In small base stations The RF unit in Power consumption during mode switching. Typically, mode switching consumes about 15% of the total power consumption;

[0076] In addition to the power consumption of the above-mentioned small base station, the embodiment of the present invention also takes into account the transmission power consumption of the user terminal, and the computing power consumption, maintenance and mode switching power consumption of the edge server; the first two types of power consumption are mainly caused by requests from local resource requesting user terminals.

[0077] set up , , Request user terminals for local resources respectively In the time slot The data size, offload delay requirement, and offload data rate requirement of the offload task during the period. and Known in advance, then It can be estimated by the following formula:

[0078]

[0079] To meet the offload data rate requirements , local resource request user terminal In the time slot The transmission power consumption of the offload task during is given by:

[0080]

[0081] Specifically, the time slot Local resources within the request user terminal The transmission power consumption is expressed as:

[0082]

[0083] in, represents the amplifier efficiency of the user terminal, and .

[0084] Consider a user terminal in a wireless resource area , in the time slot During this period, the computing tasks offloaded to the edge computing nodes have three key parameters: computing load , calculate the delay requirement and processor requests , the latter represents the CPU frequency requirement required to complete the computing task. It is measured by the number of CPU cycles required to complete the computational task, while computational latency requires is set in advance. and If the processor requirements are known in advance, It can be estimated by the following formula:

[0085]

[0086] Specifically, local resources request user equipment In the time slot The computational power consumption of the offload task during the period can be expressed as:

[0087]

[0088] in, is a constant related to chip technology and design, and is the CPU operating voltage value (such as 2~3V). The computing power of each edge server has an upper limit. Indicates small base station The maximum computing power of the edge processor.

[0089] Without loss of generality, assuming that the maintenance power consumption of edge server equipment such as power supply, signal processing, and air conditioning is similar to that of small base stations, these power consumptions can be expressed as:

[0090]

[0091]

[0092] It is worth noting that Indicates small base station The edge server is in the time slot is closed.

[0093] In order to avoid the complexity of the problem description, no binary indicator variable is specifically specified to represent the on / off status of the edge server. Whether it is 0 to indicate a small base station The edge server is in the time slot Is it closed? , respectively, with small base stations The power consumption of the corresponding edge server in the on and off states.

[0094] At the same time, assume that the mode switching power consumption generated by the edge server switching between the "on" and "off" modes Similar to small base stations, this power consumption can be expressed as:

[0095]

[0096] in, Indicates small base station The power consumption of the edge server during the mode conversion process.

[0097] in, and is a binary indicator variable, expressed as follows:

[0098]

[0099] The embodiment of the present invention focuses on the problem of long-term cumulative power consumption saving, and therefore minimizes the power consumption of small base stations, edge servers and associated user terminals. The total power consumption in a time slot, the specific expression of the power consumption minimization problem model is:

[0100]

[0101] in, Represents the set of radio frequency units in the small base station. Indicates that in the time slot Binary indicator variable for small cells within the represents the transmission power consumption of the downlink, Indicates the uplink transmission power consumption.

[0102] Specifically, the constraints include:

[0103] Downlink rate of user terminal Not less than the preset threshold ,Right now ;

[0104] Uplink rate of user terminal Not less than the preset threshold ,Right now ;

[0105] As long as the RF unit of the small base station is not completely turned off, the downlink transmission power consumption of all RF units corresponding to the small base station is The sum does not exceed the maximum transmission power consumption of the small base station ,Right now ;

[0106] The transmission power consumption of each user terminal does not exceed its own maximum transmission power consumption , and minimum transmission power consumption Not less than 0, that is , otherwise it would be meaningless;

[0107] In the time slot The sum of the computing resource requirements of all tasks offloaded to the same edge server does not exceed the computing capacity limit of the edge server, that is, ;

[0108] Constrained Binary Indicator Variable The value of ,Right now .

[0109] Since the above constraints are both non-convex and combinatorial, it is difficult to solve them using pure mathematical methods. To solve combinatorial problems, it is necessary to explore huge decision combinations. If there are 5 small base stations working together, and each small base station has 8 RF units, then the size of the entire decision space will be , it is not feasible to find the optimal switching pattern of small base stations and edge servers due to the huge computational overhead. In addition, it can only be found when the channel information and task arrival conditions of all L time slots are available. However, due to the causal relationship, this is not feasible in reality.

[0110] In the embodiment of the present invention, the power consumption minimization problem is described as a Dec-POMDP distributed partially observable Markov decision process (Decentralized Partially Observable Markov Decision Process, Dec-POMDP), which is an extension of the Markov decision process (Markov Decision Process, MDP) and can be solved by tuples. Describe the multi-agent reinforcement learning environment.

[0111] Specifically, each small base station is regarded as an intelligent agent. represents the set of all agents, Represents the global state information of the environment; at each time slot , each agent simultaneously chooses an action , the actions selected by all agents constitute the joint action ; Perform joint actions After that, the next time slot state From the state transition matrix Finally, each agent will receive a reward .

[0112] In Dec-POMDP, each agent can only use individual local observations , not global state , all local observations of the agents together constitute the joint observation The agent's local observations and historical actions are used , and the history of all agents' local observations and actions is represented by Representation; Each agent is based only on its individual history Through a decentralized strategy to select actions; each agent tries to learn a joint strategy , which maximizes their expected discounted reward .

[0113] In the time slot In the time slot, the agent first determines the maximum number of connected uplink users of the current small base station according to the number of assignable uplink users obtained in the previous time slot, and then performs user association; then, each agent needs to adjust its switching mode and reasonably allocate the transmission power consumption of uplink and downlink access to improve the energy efficiency of the network. Therefore, the action space of the agent during the time slot can be described as, and its expression is as follows:

[0114]

[0115] in, Indicates that the RF unit in the agent is in the time slot On / off mode at Then record the agent in the time slot The maximum number of uplink local resource requesting user terminals that should be served. This value can constrain the sum of server resources consumed by uplink users accessed by the edge server connected to the small base station to not exceed its maximum capacity limit, and can be used when the base station selects user association in the next time slot. This optimization strategy aims to enable the base station with more uplink users currently connected to it to access as many uplink users as possible in the next time slot, thereby ensuring that those base stations that are currently relatively idle can also remain relatively idle in the next time slot; in this way, the system can reduce the power consumption of base station mode conversion and improve the overall energy efficiency; Records the agent's time slot The transmission power consumption value of each intelligent agent must be based on the time slot The type of user terminal served by the radio unit in the time slot (local resource requesting user terminal or remote service requesting user terminal) determines the allocation of uplink or downlink transmission power consumption. The first small base station in the A radio unit serves local resource requesting user terminals ,but Indicates that in the time slot The first The local resource served by the radio unit requests the uplink transmission power consumption allocated by the user terminal; otherwise Indicates that in the time slot The first The remote service request user terminal served by each RF unit allocates downlink transmission power consumption.

[0116] The action space is the set of all actions of the agent. , action space The expression is:

[0117] ;

[0118] Due to the limitation of local state space, and considering that agents cannot exchange information, each agent only observes relevant information within its scope. The local state It is expressed as follows:

[0119]

[0120] in, Indicates that in the time slot Internal small base station The weighted average channel gain value vector of the downlink, Indicates that in the time slot Internal small base station The weighted average channel gain value vector of the uplink, Indicates that in the time slot Internal small base station The weighted average downlink data rate requirement vector of the downlink, Indicates that in the time slot Internal small base station The weighted average offload data rate requirement vector for the uplink, Indicates that in the time slot User terminals deployed in small base stations The weighted average processor capacity demand vector of the edge server offload tasks, Deployed in small base stations The edge server is in the time slot The remaining processor capacity at the start.

[0121] For the edge server in the agent, the current time slot The remaining edge processor capacity at the beginning is the remaining capacity updated at the end of the previous time slot minus the edge processor demand received at the beginning of the current time slot. This value is always less than ; In the time slot After making an action selection, set the RF unit in the agent (i.e. ) will be fixed until the next time slot action selection; at the same time, the action space will only be (Except when the time slot starts In addition, If it was on at the end of the previous time period, (or The value of (or ) to estimate ( ) is the user terminal to The total transmission gain, corresponding to for The total transmission gain to the user terminal is was closed at the end of the previous time period. (or The completion will be performed through the GRU-based state completion module. (or Given by:

[0122]

[0123] in, is a weighting factor, and its value is generally set to 0.125. Similarly, (or ) can be observed by (or ) can also be completed by a GRU-based state completion module. This choice depends on the time slot Inside, Whether a service is being provided to a user terminal and whether the type of this user terminal is LR-UE or RS-UE. or( ) in the time slot When expressed as follows:

[0124]

[0125] It is worth noting that only when the user terminal is a LR-UE and In the time slot When in the on state, is greater than 0; otherwise, it is unknown and therefore cannot be used for updating .if In the time slot If it is turned on in It is expressed as follows; otherwise, it will be completed by the GRU-based state completion module.

[0126]

[0127] also, and are initialized to an empirical value respectively;

[0128] The state space is the set of all local states of the agent , state space The expression is:

[0129] ;

[0130] When all agents take joint action After that, they will all receive a reward immediately. The core objective of the embodiment of the present invention is to ensure that the user's minimum required rate is met, that is, to ensure that the uplink / downlink rate of each user terminal is not lower than its preset minimum required threshold (i.e., the minimum required rate), and to minimize the overall network consumption. In order to achieve this dual goal, the embodiment of the present invention cannot rely solely on a single reward function, but needs to construct a comprehensive, multi-dimensional optimization strategy. This strategy aims to optimize network performance by balancing user satisfaction and network resource consumption. Based on this, in the proposed multi-agent deep reinforcement learning system, Specifically, in the time slot The reward function for the state and action during the period measures the effect of the action taken by the agent in a given state. The expression is:

[0131]

[0132] in, , , Both represent weights, Indicates that in the time slot Constraints on the quality of service requirements of internal user terminals, Indicates that in the time slot Constraints on the internal edge computing processor capacity requirements, Indicates that in the time slot Constraints on the total energy consumption of the system;

[0133] Reward Function The first term is related to the throughput and data rate requirements of the user terminal; It is used to constrain the time slot If the user service quality requirement is violated, then It will be set as a strong penalty;

[0134] Reward Function The second term is related to the edge processor capabilities and the processor requirements of the offloaded tasks of the local resource requesting user terminal; Used to constrain time slots Edge computing requirements do not exceed the maximum resource limits it has;

[0135] Reward Function The third term of is related to the total system power consumption; Used to constrain time slots The total system energy consumption is kept low.

[0136] When constructing the power consumption-constrained reward function, it is observed that the power consumption of the edge server offload task in the uplink Mainly determined by user demand, and this power consumption is not affected by the choice of action space. Therefore, when designing the reward function, it was decided not to take it into consideration and focus on other power consumption factors that can be optimized. A fixed value is set , as the maximum allowed power consumption of the entire network excluding the uplink power consumption. The core goal is to force the actual agent to The total power consumption is as low as possible. To achieve this goal, a Total power consumption (remove ), when the difference is larger, it means the total power consumption is lower, and a higher reward value will be given to encourage the agent to maintain a low power consumption state. On the contrary, when the actual total power consumption gradually approaches , the reward value will decrease accordingly, prompting the agent to try other activities to reduce power consumption. In this way, it is ensured that the reward function can accurately and efficiently guide the optimization agent to select actions, thereby effectively reducing the total power consumption of the network; They are expressed as:

[0137]

[0138] in, Represents the time slot is the set of user terminals associated with the agent. is a piecewise function defined as follows:

[0139] ;

[0140] in, is a negative number, indicating the penalty value when the constraint is not satisfied. In addition, It is also a piecewise function, which is defined as follows:

[0141]

[0142] in, is a non-negative number representing the reward value when the constraint is satisfied.

[0143] Based on the action space, state space and reward function, the power consumption minimization problem model is described as a distributed partially observable Markov decision process.

[0144] Specifically, before step 4, it also includes:

[0145] Construct a decision-making power consumption allocation model, which consists of a GRU-based state completion module and an actor-critic neural network architecture connected in sequence;

[0146] The decision power consumption allocation model is trained to obtain a trained decision power consumption allocation model.

[0147] Specifically, the GRU-based state completion module is set up in the edge server;

[0148] The input of the GRU-based state completion module is all local states of the small base stations in the state space;

[0149] The output of the GRU-based state completion module is the completed state space.

[0150] Specifically, all local states in the state space are input into the trained decision power consumption allocation model for decision allocation, and the switch state and power consumption allocation results of each device in the coverage area of ​​the macro base station are obtained. The devices in the coverage area of ​​the macro base station include the radio frequency units and edge servers of the small base station and the user terminals, including:

[0151] Determine whether there is a missing state value in the state space through the switch states of all radio frequency units;

[0152] When there are missing state values, the GRU-based state completion module completes the missing state values ​​to obtain the completed state space;

[0153] The completed state space is input into the actor-critic neural network architecture for decision allocation, and the switch state and power consumption allocation results of each device in the coverage area of ​​the macro base station are obtained. The devices in the coverage area of ​​the macro base station include the radio frequency units and edge servers of the small base station and user terminals.

[0154] When there are no missing values ​​in the state, all local states in the state space are input into the actor-critic neural network architecture for decision allocation, and the switch state and power consumption allocation results of each device in the coverage area of ​​the macro base station are obtained. The devices in the coverage area of ​​the macro base station include the radio frequency units and edge servers of the small base station and the user terminals.

[0155] For example, when there is no user terminal that needs to request resources, there is no need to turn on the radio frequency unit and edge server in the small base station, and the power of the radio frequency unit and edge server is 0;

[0156] When there is a user terminal that needs to request resources, the edge server is turned on, and an appropriate number of RF units are turned on according to the number of user terminals that need to request resources. The power consumption of the RF unit is the sum of the power consumption of all turned-on RF units, and the power consumption of the edge server is the actual power consumption.

[0157] Therefore, the embodiment of the present invention constructs a decision-making power consumption allocation model (MAPPO-MP-GRU) by adopting an actor-critic neural network structure based on multi-agent proximal policy optimization (MAPPO). In the scenario provided by the embodiment of the present invention, the macro base station can serve as a place for information collection and centralized training.

[0158] In the embodiment of the present invention, the decision-making power consumption allocation model combines a centralized critic network and a distributed actor network. The centralized critic network will be based on global information. To perform value assessment, the distributed actor network can only rely on local state information To make a decision; the state value function of the centralized critic is marked as ,in The parameters of the actor network representing the agent. For each agent, its policy loss function can be expressed as:

[0159]

[0160] in and Representing an Agent The current strategy and the old strategy, Representing an Agent The advantage function value of .

[0161] Specifically, the advantage function value Dependent on global information, the expression of the advantage function is:

[0162]

[0163] Agent Actor network parameters Update the function through the policy gradient:

[0164]

[0165] in, is the sampling expectation function, represents the centralized critic state value function, are the parameters of the critic network of the agent.

[0166] Update by minimizing the loss function , the expression is as follows:

[0167]

[0168] in, .

[0169] The training process of the decision power allocation model is as follows:

[0170] Input: Discount Factor , number of training iterations 3500, small batch size , GAE weight , learning rate , maximum step length .

[0171] Output: Model parameters of the actor network in the power allocation model of the small base station

[0172] For small base stations :

[0173] Step 1: Initialize actor network parameters , critic network parameters , experience cache ;

[0174] Step 2: Initialization ;

[0175] Step 3: Initialize the current step time slot ;

[0176] Step 4: Get the local observation state , if a radio frequency unit in a small base station In the time slot During the period, the data is missing due to the closed state, so the pre-trained GRU-based state completion module is used for prediction to obtain the state To complete the state space In the above example, we get the completed state space.

[0177] Step 5: Generate actions based on the actor network and local observations : ;

[0178] Step 6: Get local observations from other agents and compose the global state And input it into the critic network to get ;

[0179] Step 7: After all agents perform actions, they receive rewards , the system goes to the next state ;

[0180] Step 9: If , then jump to step 10, otherwise let , jump to step 4;

[0181] Step 10: Get the action trajectory of the agent And the output value of the critic network ;

[0182] Step 11: Calculate the advantage function based on the advantage function expression , and store data To Experience Cache middle;

[0183] Step 12: Experience Cache Randomly select a small batch Data ;

[0184] Step 13: Calculate based on the policy gradient update function , according to the minimization loss function ;

[0185] Step 14: Calculate the gradient , update the actor network using the Adam optimizer;

[0186] Step 15: Calculate the gradient , update the critic network using the Adam optimizer;

[0187] Step 16: If , the training is completed, otherwise , skip to step 3.

[0188] The network dynamics caused by the arrival of time-varying tasks and changes in channel quality will lead to unknown network state transitions, which may increase the uncertainty of learning and reduce the accuracy of learning. The macro base station that executes the proposed decision power allocation model needs to obtain the instantaneous and complete status of all small base stations. Each small base station should notify its status information in a timely manner. However, the notified status information may be incomplete because the small base station cannot observe the status of its inactive RF units. This will affect the learning efficiency and accuracy of the decision power allocation model. To this end, the GRU-based state completion module aims to predict the unobservable state of the RF unit of the small base station, which will be input into the decision power allocation model as state information.

[0189] like Figure 2 The GRU-based state completion module shown runs on the edge server. During the operation, the historical observation data of the small base station is input into the GRU-based state completion module in sequence as input. In this way, the GRU-based state completion module can predict the missing state of the small base station and obtain the completed (but outdated) state. The completed state space is input into the actor-critic neural network architecture for decision allocation. The training process of the GRU-based state completion module is as follows:

[0190] Input: Local state cache pool and the number of empirical samples , number of training iterations , the maximum time step length ;

[0191] Output: trained GRU network parameters;

[0192] Step 1: Initialize GRU network parameters , historical local cache pool ;

[0193] Step 2: Use Experience sample filling Construct initial training data;

[0194] Step 3: For each , initialize the training sequence index and the current time step If the training sequence index exceeds the training data set size, the training will end. , go to step 9;

[0195] Step 4: Select the next training sequence from the training dataset;

[0196] Step 5: For each time step in the current training sequence (from 0 to ): If t reaches , then the processing of the current sequence ends and returns to step 5 to select the next sequence; obtain the input data of the current time step ,according to and the hidden state at the previous time step Calculate the hidden state at the current time step and predicted status , and update the current time step t to t+1;

[0197] Step 6: Calculate the loss function, which is defined as the mean square error between the training data and the predicted value;

[0198] Step 7: Calculate the gradient and update the network parameters using the Adam optimizer;

[0199] Step 8: Add the training epoch indicator variable, which means that an epoch iteration has been completed;

[0200] Step 9: If the preset number of epochs is not reached, return to step 4 and continue training for the next epoch. Otherwise, end the training process and save the relevant network parameters for use in completion.

[0201] Each small base station actively reports its current complete status to the macro base station at the end of each time period. It is assumed that each small base station can always be connected to the macro base station through a backhaul or a dedicated channel. The reported status is used to update the content in the experience replay storage pool, from which samples are extracted to train the decision power allocation model. In addition, the reported status will be reported to the trained decision power allocation model to make a decision, expressed as:

[0202] ;

[0203] Based on the above formula, .

[0204] Without loss of generality, in order to reduce the complexity of the model, and , respectively for each type of status information maintained by the small base station (e.g. ) trains a separate GRU to complete this type of state information, and finally sends the completed value to the edge server. Figure 2 The GRU unit structure in the GRU unit processes the input sequence ; Among them, the reset gate is responsible for adjusting the degree of fusion of the information of the previous time step and the current time step, while the update gate controls the degree of updating the hidden state. These gate mechanisms allow the GRU unit to flexibly decide which information to retain or forget when processing sequence data, thereby more efficiently capturing long-term dependencies in the sequence. The use of reset gates and update gates enables the GRU-based state completion module to avoid the problem of gradient vanishing or exploding when processing long sequences, and improves the stability and performance of the model.

[0205] Specifically, the function expression of the GRU-based state completion module is:

[0206]

[0207] in, Indicates that in the time slot The completion status value obtained in represents the update gate, Indicates that in the time slot The completion status value obtained in represents the output candidate, represents the sigmoid function, Represents the input sequence (i.e. the historical value of the state that needs to be completed), represents the bias of the update gate, Represents the reset gate, represents the deviation of the reset gate, represents the hyperbolic tangent function, represents the deviation of the output candidate set, , , They represent the weight of the update gate, the weight of the reset gate, and the weight of the output candidate set respectively.

[0208] The embodiment of the present invention uses the python+pytorch tool to simulate the provided method, and the simulation parameters are shown in Table 1 and Table 2 below:

[0209] Table 1

[0210]

[0211] Table 2

[0212]

[0213] By comparing the convergence of the four integrated solutions and obtaining Figures 3 to 5 The four integration schemes are as follows:

[0214] (1) MAPPO-MP-GRU: Decision-making power consumption allocation model. For the situation where the RF unit of the small base station is in the off state and the status information cannot be directly obtained in a specific time slot, the algorithm uses the GRU model for training to predict the missing RFU status information. The status value predicted by the GRU model will be integrated into the overall status information to form a complete and accurate status information set. This approach not only improves the integrity of the model, but also enhances its accuracy. With this complete and accurate status information, the system can make more efficient and accurate decisions, thereby further optimizing network performance and resource allocation.

[0215] (2) MAPPO-MP: Switching mode decision and power allocation algorithm based on MAPPO. The main difference between this solution and solution (1) is that when the RFU enters the off state, the algorithm chooses to directly use the state information value of the previous on time slot instead of relying on real-time prediction to obtain a more accurate state value. Although this strategy ensures data continuity, it may still affect system performance and efficiency.

[0216] (3) VDN-MP: Switching mode decision and power allocation algorithm based on VDN (Value Decomposition Networks). The difference between this solution and solution (2) is that VDN, a deep reinforcement learning, is used to obtain the base station / server switching mode selection and resource allocation strategy.

[0217] (4) QMIX-MP: Switching mode decision and power allocation algorithm based on QMIX (Q-value Mixing Networks). The difference between this solution and solution (2) is that it uses deep reinforcement learning such as QMIX to obtain base station / server switching mode selection and resource allocation strategy.

[0218] Figures 3 to 5 The comparison between the decision-making power allocation model integration scheme designed by the present invention and the other three schemes in terms of model convergence is intuitively demonstrated. Figure 3 From the total reward change trend, we can see that after introducing the GRU prediction missing value strategy, the system performance and convergence speed of the model have been significantly improved. Figure 4 The comparison of user demand satisfaction rate shows that the user demand satisfaction rate of this model is significantly higher than that of other strategies, which fully proves its superiority in meeting the minimum needs of users. Figure 5 It can be found that the decision-making power consumption allocation model can reduce the total power consumption of individuals faster and more stably in the early stage of training, which lays a solid foundation for its subsequent model training. At the same time, it can be observed from the figure that the curve of the decision-making power consumption allocation model is smoother overall and has fewer spikes, which fully reflects the stability of its model training. Figures 3 to 5From the analysis, we can conclude that the decision-making power consumption allocation model strategy effectively reduces the total network energy consumption while improving the user demand satisfaction rate, and the entire model training process is stable and fast.

[0219] In general, the decision-making power consumption allocation model designed by the present invention is significantly better than the other three construction schemes in terms of performance, which is mainly attributed to two major factors. First, when the RFU is frequently in the off state, if the state value of the previous time slot is simply used to fill the unobserved state, this will inevitably affect the agent's correct action selection, thereby affecting the overall performance. In contrast, the decision-making power consumption allocation model scheme can obtain more accurate and complete environmental state values, and then make more reasonable decisions and improve system performance. Secondly, the MAPPO algorithm shows significant advantages over QMIX and VDN in this scenario. Although VDN learns a global action value function and simply decomposes it into the sum of the individual action value functions of each agent, this simple "sum" operation method shows the limitations of its fitting ability when facing the complex tasks described in this article. QMIX attempts to replace the "sum" operation with a hybrid network by introducing a monotone value function decomposition method to capture a more complex relationship between the global action value function and the individual action value function. However, these methods may perform well in a purely cooperative environment, but their performance is insufficient in the mixed cooperation and competition environment described in this article. In contrast, the decision-making power allocation model strategy surpasses the other three methods in terms of performance by virtue of its ability to accurately perceive the environmental status and more flexible decision-making mechanism. It can not only effectively cope with complex and changing environments, but also make optimal decisions under the coexistence of cooperation and competition, thereby improving the overall performance of the system.

[0220] Compared with the prior art, the embodiment of the present invention constructs an edge computing network including a macro base station, wherein the coverage area of ​​the macro base station includes multiple user terminals and a small base station with an edge server and multiple radio frequency units deployed, and each user terminal is connected to any radio frequency unit in the small base station through its own radio frequency unit for accessing network services or offloading tasks to the edge server; a power consumption minimization problem model of total power consumption is constructed, and constraints are constructed for the power consumption minimization problem model; the small base station is taken as an intelligent agent, all action sets of the intelligent agent are taken as an action space, and all local state sets of the intelligent agent are taken as a state space, and a reward function is constructed to describe the power consumption minimization problem model as a distributed partially observable Markov decision process; all local states in the state space are input into the trained decision power consumption allocation model for decision allocation, and the switch state and power consumption allocation results of each device in the coverage area of ​​the macro base station are obtained; according to the switch state and power consumption allocation results of each device in the coverage area of ​​the macro base station, the radio frequency unit, edge server and user terminal in the small base station are regulated to achieve equipment energy saving; while meeting the user experience quality requirements, the switch state and power consumption of each radio frequency unit and edge server in the small base station are dynamically regulated through the trained decision power consumption allocation model, thereby minimizing the total energy consumption of the device.

[0221] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A device energy saving method in a cellular edge computing environment, characterized in that: include: Step 1: construct an edge computing network including a macro base station, wherein the coverage area of ​​the macro base station includes multiple user terminals and a small base station with an edge server and multiple radio frequency units deployed, and each of the user terminals is connected to any radio frequency unit in the small base station through its own radio frequency unit to access network services or offload tasks to the edge server; Step 2: construct a power consumption minimization problem model of total power consumption based on the edge computing network, and construct constraint conditions for the power consumption minimization problem model, wherein the total power consumption includes the power consumption of the small base station, the user terminal, and the edge server; Step 3, taking the small base station as an intelligent agent, taking all action sets of the intelligent agent as an action space, taking all local state sets of the intelligent agent as a state space, and constructing a reward function, describing the power consumption minimization problem model as a distributed partially observable Markov decision process; Step 4: input all local states in the state space into the trained decision power consumption allocation model for decision allocation, and obtain the switch state and power consumption allocation results of each device in the coverage area of ​​the macro base station. The devices in the coverage area of ​​the macro base station include each radio frequency unit and edge server of the small base station and the user terminal; Step 5: According to the switch status and power consumption allocation results of each device in the coverage area of ​​the macro base station, the power consumption of the radio frequency unit and the edge server in the small base station and the user terminal is regulated to achieve equipment energy saving.

2. The device energy saving method in a cellular edge computing environment according to claim 1, characterized in that: The total power consumption of the edge computing network includes the power consumption of the edge server, the small base station, and all devices associated with the small base station. The power consumption expression is: ； in, Indicates that in the time slot The total power consumption of the internal edge computing network, , Both represent binary indicator variables, Indicates that in the time slot Internal small base station The The transmission power consumption of each radio frequency unit is Indicates that in the time slot User terminal within For small base stations The transmission power consumption is Indicates that in the time slot Deployed in small base stations The edge server receives the The maintenance power consumption of each RF unit when unloading tasks, Indicates that in the time slot Internal small base station Maintenance power consumption, Indicates that in the time slot Internal small base station The The mode switching power consumption of each RF unit, Indicates that in the time slot Internal small base station The mode switching power consumption of the edge server in Indicates user terminal In the time slot Offload tasks to small base stations The edge server in the computing power consumption of the edge server, Indicates a small base station The RF units, represents the user terminal, Indicates the total number of user terminals, Indicates the number of radio frequency units.

3. The device energy saving method in a cellular edge computing environment according to claim 2, characterized in that: The expression of the power consumption minimization problem model is: in, Represents the set of radio frequency units in the small base station. Indicates that in the time slot Binary indicator variable for small cells within the represents the transmission power consumption of the downlink, Indicates the uplink transmission power consumption.

4. The device energy saving method in a cellular edge computing environment according to claim 3, characterized in that: The constraints include: Downlink rate of user terminal Not less than the preset threshold ; Uplink rate of user terminal Not less than the preset threshold ; The sum of the downlink transmission power consumption of all radio frequency units corresponding to the small base station shall not exceed the maximum transmission power consumption of the small base station. ; The transmission power consumption of each user terminal does not exceed its own maximum transmission power consumption ; In the time slot The sum of the computing resource requirements of all tasks offloaded to the same edge server does not exceed the computing capacity limit of the edge server; Constrained Binary Indicator Variable The value of .

5. The device energy saving method in a cellular edge computing environment according to claim 4, characterized in that: The step 3 comprises: The small base station is taken as an intelligent agent, and all action sets of the intelligent agent are taken as an action space , the action space The expression is: in, Indicates small base station The RF unit in the time slot The switch mode within Indicates small base station In the time slot The maximum number of user terminals that should be served to request offload tasks, Indicates small base station In the time slot Transmission power consumption; The state space is the set of all local states of the agent , the state space The expression is: in, Indicates that in the time slot Internal small base station The weighted average channel gain value vector of the downlink, Indicates that in the time slot Internal small base station The weighted average channel gain value vector of the uplink, Indicates that in the time slot Internal small base station The weighted average downlink data rate requirement vector of the downlink, Indicates that in the time slot Internal small base station The weighted average offload data rate requirement vector for the uplink, Indicates that in the time slot User terminals deployed in small base stations The weighted average processor capacity demand vector of the edge server offload tasks, Deployed in small base stations The edge server is in the time slot Remaining processor capacity at the start; Building the Reward Function , the reward function The expression is: in, , , Both represent weights, Indicates that in the time slot Constraints on the quality of service requirements of internal user terminals, represents the constraint on the edge computing processor capacity requirement within the time slot, Indicates that in the time slot Constraints on the total energy consumption of the system; Based on the action space, the state space and the reward function, the power consumption minimization problem model is described as a distributed partially observable Markov decision process.

6. The device energy saving method in a cellular edge computing environment according to claim 5, characterized in that: Before step 4, the method further includes: Constructing a decision-making power consumption allocation model, wherein the decision-making power consumption allocation model is composed of a GRU-based state completion module and an actor-critic neural network architecture connected in sequence; The decision power consumption allocation model is trained to obtain a trained decision power consumption allocation model.

7. The device energy saving method in a cellular edge computing environment according to claim 6, characterized in that: The GRU-based state completion module is arranged in the edge server; The input of the GRU-based state completion module is all historical local states of the small base stations in the state space; The output of the GRU-based state completion module is the completed state space.

8. The device energy saving method in a cellular edge computing environment according to claim 7, characterized in that: All local states in the state space are input into the trained decision power consumption allocation model for decision allocation, and the switch state and power consumption allocation results of each device in the coverage area of ​​the macro base station are obtained, including: Determining whether there is a state missing value in the state space according to the switch states of all radio frequency units; When there are missing values ​​of the state, the GRU-based state completion module completes the missing values ​​of the state to obtain a completed state space; The completed state space is input into the actor-critic neural network architecture for decision allocation, and the switch state and power consumption allocation result of each device in the coverage area of ​​the macro base station are obtained. The devices in the coverage area of ​​the macro base station include each radio frequency unit and edge server of the small base station and the user terminal; When there is no missing value of the state, all local states in the state space are input into the actor-critic neural network architecture for decision allocation to obtain the switch state and power consumption allocation results of each device in the coverage area of ​​the macro base station. The devices in the coverage area of ​​the macro base station include the radio frequency units and edge servers of the small base station and the user terminal.

9. The device energy saving method in a cellular edge computing environment according to claim 8, characterized in that: The function expression of the GRU-based state completion module is: in, Indicates that in the time slot The completion status value obtained in represents the update gate, Indicates that in the time slot The completion status value obtained in represents the output candidate, represents the sigmoid function, Indicates the historical value that needs to be completed. represents the bias of the update gate, Represents the reset gate, represents the deviation of the reset gate, represents the hyperbolic tangent function, represents the deviation of the output candidate set, , , They represent the weight of the update gate, the weight of the reset gate, and the weight of the output candidate set respectively.

10. The device energy saving method in a cellular edge computing environment according to claim 1, characterized in that: The user terminals include remote service requesting user terminals and local resource requesting user terminals.

Images