A 5G base station energy saving method based on decision model

Abstract

The present invention discloses a 5G base station energy-saving method based on a decision model, which belongs to the field of 5G base station energy-saving. The flow data is divided into different network flow scenarios; base stations are deployed, and a MIMO network environment is established; data is preprocessed to obtain the distribution law of network flow and establish a simulation environment, and a trajectory is generated by recording the interaction between a reinforcement learning agent and the simulation environment; the decision model is pre-trained, and the trajectory is used as the input of the decision model to learn the best trajectory action; the decision model is interacted with the MIMO network environment, and the time step is used as input to generate action probabilities and obtain output actions through sampling; the output action is input into the MIMO network environment to obtain feedback, and a base station energy-saving strategy is obtained through feedback. The present invention adopts the above-mentioned 5G base station energy-saving method based on a decision model, which can solve the problem of poor generalization ability of the reinforcement learning model, can handle the problem of changes in the number of base stations in a large-scale MIMO network, and achieve the purpose of reducing energy consumption.

Description

Technical Field

[0001] The present invention relates to the field of 5G base station energy saving technology, and in particular to a 5G base station energy saving method based on a decision model. Background Art

[0002] In recent years, global mobile network traffic has grown exponentially, and is expected to double every two years, reaching 115EB (115×1018 bytes) per month by the end of 2022. The deployment of modern mobile communication networks has also grown tremendously, driven by an increasing number of mobile devices: smartphones, tablets, and IoT devices. The rapid growth of network traffic and mobile devices is inseparable from the hardware support brought by 5G base stations, but while providing excellent performance, the explosive growth in the number of base stations has led to a rapid increase in global cellular network energy consumption, which has now reached about 543.6 terawatt hours per year, accounting for about 2% of global electricity demand.

[0003] The advent of the fifth generation (5G) cellular network will bring a range of enhancements compared to previous generations, including lower latency, higher data rates, and wider connectivity. To match these improvements, 5G networks will need to deploy denser base stations. However, this will lead to a significant increase in energy consumption, as base stations are the most energy-consuming component in wireless networks, accounting for about 80% of network energy consumption.

[0004] In addition, 5G's higher bit rate requirements require the use of more power-hungry processors and RF hardware on base stations, resulting in energy consumption that is approximately two to three times higher than fourth-generation (4G) base stations. While 5G presents challenges in terms of energy consumption, it also provides opportunities to implement new energy-saving methods. By shutting down underused base stations, the power consumed can be minimized. However, mobile network traffic is highly dynamic, the delays that users can tolerate are usually very short, and the control of 5G mobile network systems is highly complex, which requires base station control strategies to be dynamic, adaptive, and cooperative. At present, deep learning and reinforcement learning are seen as a viable approach to solving this complex problem.

[0005] However, reinforcement learning still has many shortcomings, such as the difficulty in handling the balance between exploration and utilization and poor generalization. A new decision model technology (Decision Transformer, DT) that transforms reinforcement learning into sequence modeling to solve problems and combines it with deep learning has emerged. Summary of the invention

[0006] The purpose of the present invention is to provide a 5G base station energy saving method based on a decision model to solve the problems mentioned in the above background technology.

[0007] To achieve the above object, the present invention provides a 5G base station energy saving method based on a decision model, comprising the following steps:

[0008] S1. Due to the differences in traffic data, machine learning is used to cluster traffic data in time and space and divide it into different network traffic scenarios;

[0009] S2. Deploy base stations in open space areas and establish a multiple-input multiple-output (MIMO) network environment including channel, base station and user parameters for subsequent energy consumption analysis and decision model control;

[0010] S3, data preprocessing, clustering the traffic data, obtaining the distribution law of network traffic and establishing a simulation environment based on the distribution law, generating trajectories through the interaction between the reinforcement learning model and the simulation environment;

[0011] S4. Pre-train the decision model, take the trajectory as the input of the decision model, and the decision model learns from the trajectory and learns the best trajectory action based on the target report;

[0012] S5. Test the interaction between the decision model and the multi-input multi-output network environment. Take the time step as input, embed the time step into the decision model through the linear layer after state completion, pass the embedded decision model into the Transformer model, generate action probability, perform action masking, and obtain the output action through sampling;

[0013] S6. Input the output action in S3 into a multi-input multi-output network environment to obtain feedback, adjust the base station state through the feedback, and obtain a base station energy-saving strategy.

[0014] Preferably, a time step in S3 includes a reward, a state, and an action.

[0015] Preferably, the state completion in S3 refers to filling the state space with zeros at the end so that the state space can match the number of neural network layers of the decision model; action masking refers to setting the actions after the retained action probability to zero after determining the action probability retained in the output of the decision model when the number of base stations in the simulation environment decreases.

[0016] Preferably, the output action in S3 is an agent.

[0017] Preferably, in a multi-input multi-output network environment, the decision made by the agent is defined as follows:

[0018] A=M×S×U

[0019] in

[0020] M = {-4, 0, 4}

[0021] Among them, -4: turn off 4 antennas; 0: do not switch; 4: turn on 4 antennas again;

[0022] S = {0, 1, 2, 3}

[0023] Indicates the operation of switching the sleep level of the base station to the corresponding level;

[0024] U = {-1, 0, 1}

[0025] When U=1, it means accepting the service request without uninstalling;

[0026] When U=0, it means that the new user is offloaded to other base stations;

[0027] When U=-1, it means that all associated users are separated and the new user is uninstalled.

[0028] Preferably, the reward function of the agent when making decisions is defined as follows:

[0029] Assume the average data rate and required data rate of the user to be (r avg,k ) and (r reg,k ), the reward function is:

[0030] R=ω qos ξ-ω pc ρ

[0031]

[0032] Among them, R represents return; ω qos and ω pc represents the parameter used to balance service quality and energy consumption; ρ represents the average power consumption of the network within an action interval; ξ represents the reward for high-quality service; ρ k Represents the ratio of the average user data rate to the required data rate, when ρ k <1 is a negative value, ρ k Positive value when ≥1; is a balance parameter.

[0033] Therefore, the present invention adopts the above-mentioned 5G base station energy saving method based on the decision model, which has the following beneficial effects:

[0034] (1) Realize the energy-saving control requirements of 5G base stations in large-scale networks, such as dynamic data traffic, low data latency requirements, and complex network control;

[0035] (2) It solves the problem that traditional reinforcement learning is difficult to handle changing action space and state space when the number of base stations in each network is different.

[0036] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a method flow chart of an embodiment of the present invention;

[0038] Figure 2 is a data processing flow chart of the S5 test process of an embodiment of the present invention;

[0039] Figure 3 A physical model framework diagram of an embodiment of the present invention;

[0040] Figure 4 The decision model training loss function curve of the embodiment of the present invention;

[0041] Figure 5 It is a comparison of some indicators of the embodiment of the present invention in the rural network traffic scenario with reinforcement learning;

[0042] Figure 6 Comparison of some indicators of the embodiment of the present invention with those of reinforcement learning in the urban network traffic scenario;

[0043] Figure 7 This is a diagram of the total energy consumption results of simulation under different numbers of base stations in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] Example

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Reference Figure 1-Figure 7 The present invention discloses a 5G base station energy saving method based on a decision model, comprising the following steps:

[0047] S1. Due to the diversity of traffic data in reality, machine learning is used to cluster traffic data in time and space, and divide it into different network traffic scenarios, representing different regions (such as rural areas, urban areas, office areas, etc.) and the latency constraints of different network services.

[0048] S2. Deploy base stations in open space areas and establish a multi-input multi-output network environment including channel, base station and user parameters for subsequent energy consumption analysis and decision model control.

[0049] An environment including parameters such as channels, energy consumption, and user information is recommended for subsequent energy consumption analysis and decision model control. Seven base stations are deployed in the open space area, and a massive MIMO channel with zero interference precoding is considered for network services. The purpose of this is to make each user receive only its expected signal while offsetting the interference caused by other users. In this way, each user can obtain high-quality services even in a multi-user environment. The network traffic is simulated as a non-homogeneous Poisson process of user arrival, and each user has a certain traffic demand scale and delay budget and arrival rate to simulate the network traffic scenarios divided in data clustering.

[0050] Centralized training is performed, and a decision model is used as an intelligent agent to control each sub-base station to make decisions to reduce the energy consumption of the entire system. In a multi-input multi-output network environment, the decisions that the intelligent agent can make are defined as follows:

[0051] A=M×S×U,

[0052] in

[0053] M = {-4, 0, 4}

[0054] Among them, -4: turn off 4 antennas, 0: do not switch, 4: turn on 4 antennas again.

[0055] S = {0, 1, 2, 3}

[0056] Indicates the operation of switching the sleep level of the base station to the corresponding level.

[0057] U = {-1, 0, 1}

[0058] (U=1) Accept the service request without offloading (when the base station is active, this means immediate connection; when the base station is dormant, the user will be placed in the service queue of the base station).

[0059] (U=0) offload the new user to other base stations.

[0060] (U=-1) Detach all associated users and uninstall the new user.

[0061] When a user equipment is associated with a base station, it is either receiving service or waiting in the queue of the base station. For the operation in U, -1 is particularly useful when the base station is dormant, so that the separated user equipment can request a new connection with other base stations. However, the base station can still choose to keep the users in its current queue until it is awakened (taking action 0), or put the newly arrived user equipment into the queue by taking action 1. The operation interval of the base station agent is set to 20 milliseconds, which is also the default period for the transmission of the synchronization signal (SS) block. However, this period is too long for SM1 and SM2 dormancy. Therefore, this embodiment adds an automatic wake-up mechanism for both SM1 and SM2, and the base station will wake up by itself as long as there is traffic demand during these two levels of dormancy. Since the activation delay of the SM1 state is very short, this embodiment hopes that the base station will return to SM1 after completing the service, so that the base station is in active mode when there is traffic load and switches to sleep mode when there is no load. However, in the SM2 state, it takes 10 milliseconds for the base station to wake up, so the base station will remain active until the next operation.

[0062] In this example, the agent's decision should be based on the goal of base station energy saving, but at the same time to meet user service. Therefore, it is necessary to design a reward function to ensure that both energy saving and service quality are met.

[0063] For a user, let its average data rate and required data rate be (r avg,k 0 and (r reg,k ),

[0064] The reward function is defined as:

[0065] R=ω qos ξ-ω pc ρ

[0066]

[0067] Where R is the return, ω qos and ω pc is a parameter used to balance service quality and energy consumption, ρ is the average power consumption of the network within an action interval. ξ is the reward for high-quality service. k is the ratio of the average user data rate to the required data rate. k <1 is a negative value, ρ k It is positive when ≥1. is the balance parameter. The specific parameter values ​​are shown in Table 1.

[0068] Table 1 Parameter values ​​in the reward function

[0069]

[0070]

[0071] S3, data preprocessing, clustering the traffic data, obtaining the distribution law of network traffic and establishing a simulation environment based on the distribution law, and generating trajectories through the interaction between the reinforcement learning model and the simulation environment.

[0072] S4. Pre-train the decision model and use the trajectory as the input of the decision model. The decision model learns from the trajectory and learns the best trajectory action based on the target report, thereby achieving high-level action output in the environment.

[0073] The decision model DT is a technology that transforms reinforcement learning problems into sequence modeling. By using the technology that transforms reinforcement learning into sequence modeling (Decision Transformer, DT), the DT model is used for base station energy-saving decision-making. Compared with reinforcement learning, it has better generalization ability and has adaptability that exceeds reinforcement learning in different scenarios with large differences in network traffic distribution. It can meet the decision-making tasks of different scenarios in large-scale networks.

[0074] It uses Transformer technology. Unlike previous reinforcement learning methods that fit value functions or calculate policy gradients, DT simply outputs the best action by using a causal masked Transformer. By adjusting the autoregressive model of expected return (reward), past state, and action, the DT model can generate future actions that achieve the expected return.

[0075] In large-scale MIMO networks, DT agents are used as decision-making models to output actions to optimize base station power allocation, antenna configuration and other parameters to reduce energy consumption. For example, power can be reduced when the load is low, or the antenna direction can be adjusted when the load is high. It can handle base station energy saving issues in a variety of scenarios.

[0076] S5. Test the interaction between the decision model and the multi-input multi-output network environment. Take the time step as input, embed the time step into the decision model through the linear layer after state completion, pass the embedded decision model into the Transformer model, generate action probability and perform action masking, and obtain the output action through sampling.

[0077] S6. Input the output action in S3 into a multi-input multi-output network environment to obtain feedback, adjust the base station state through the feedback, and obtain a base station energy-saving strategy.

[0078] State completion and action masking: State completion refers to filling the state space in a multi-input multi-output network environment with zeros at the end so that it can match the number of neural network layers of the pre-trained decision model, allowing the model to process the input state. Action masking means that when the number of base stations in the environment decreases, the output of the decision model will contain unnecessary action probabilities, so after determining the action probabilities to be retained, the subsequent actions are set to zero to ensure that these actions will not be selected during the adoption process.

[0079] In the test, DT will interact with the MIMO network environment, taking K time steps as input, including rewards, states, and actions. The states and actions will undergo state completion and action masking at this time. State completion refers to filling the state space with zeros at the end so that it can match the number of neural network layers of the pre-trained model, so that the model can process the input state. Action masking means that when the number of base stations in the MIMO network environment decreases, the output of the decision model will contain unnecessary action probabilities, so after determining the action probabilities to be retained, the subsequent actions are set to zero to ensure that these actions will not be selected during the adoption process. The processed input is embedded through a linear layer, and the embedded decision model is passed to an autoregressive model, which generates action probabilities given the required rewards, past states, and actions and obtains actions through sampling. These actions are output to the MIMO network environment and feedback is obtained for the next action, ultimately achieving energy reduction.

[0080] Therefore, the present invention adopts the above-mentioned 5G base station energy-saving method based on the decision model, which can solve the problem of poor generalization ability of the reinforcement learning model, and can handle the problem of changes in the number of base stations in large-scale MIMO networks, thereby achieving the purpose of reducing energy consumption.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A 5G base station energy saving method based on a decision model, characterized in that: The following steps are involved: S1. Due to the differences in traffic data, machine learning is used to cluster traffic data in time and space and divide it into different network traffic scenarios; S2. Deploy base stations in open space areas and establish a multi-input multi-output network environment including channel, base station and user parameters for subsequent energy consumption analysis and decision model control; S3, data preprocessing, clustering the traffic data, obtaining the distribution law of network traffic and establishing a simulation environment based on the distribution law, generating trajectories through the interaction between the reinforcement learning model and the simulation environment; S4. Pre-train the decision model, take the trajectory as the input of the decision model, and the decision model learns from the trajectory and learns the best trajectory action based on the target report; S5. Test the interaction between the decision model and the multi-input multi-output network environment. Take the time step as input. The time step includes reward, state and action. The state and action will be completed and masked at this time. After the state is completed, the time step is embedded in the decision model through the linear layer. The embedded decision model is passed to the Transformer model. After generating the action probability, action masking is performed, and the output action is obtained by sampling. State completion refers to filling the state space with zeros at the end so that the state space can match the number of neural network layers of the decision model; action masking refers to setting the actions after the retained action probability to zero after determining the action probability retained in the output of the decision model when the number of base stations in the simulation environment decreases; S6. Input the output action in S5 into a multi-input multi-output network environment to obtain feedback, adjust the base station state through the feedback, and obtain a base station energy-saving strategy.

2. According to a 5G base station energy saving method based on a decision model according to claim 1, it is characterized in that: The output action in S5 is the agent.

3. A 5G base station energy saving method based on a decision model according to claim 2, characterized in that: In a MIMO network environment, the decision made by the agent is defined as follows: A=M×S×U in M＝{-4，0，4} in, -4: turn off 4 antennas; 0: do not switch; 4: Open 4 more antennas; S＝{0，1，2，3} Indicates the operation of switching the sleep level of the base station to the corresponding level; U＝{-1，0，1} When U=1, it means accepting the service request without uninstalling; When U=0, it means that the new user is offloaded to other base stations; When U=-1, it means that all associated users are separated and the new user is uninstalled.

4. A 5G base station energy saving method based on a decision model according to claim 3, characterized in that: The reward function of the agent when making decisions is defined as follows: Assume the average data rate and required data rate of the user to be r avg,k and r reg,k , the reward function is: R=ω qos x-o pc r Among them, R represents return; ω qos and ω pc represents the parameter used to balance service quality and energy consumption; ρ represents the average power consumption of the network within an action interval; ξ represents the reward for high-quality service; ρ k Represents the ratio of the average user data rate to the required data rate, when ρ k <1 is a negative value, ρ k Positive value when ≥1; is a balance parameter.

Images