A base station sleep method, device and medium based on reinforcement learning

Abstract

The present invention proposes a base station dormancy method, device and medium based on reinforcement learning. The method comprises: establishing a deep reinforcement learning model, predicting the future traffic load of each base station in an ultra-dense network, and determining the shutdown status of each base station in the ultra-dense network. The method introduces an LSTM model and predicts the future traffic load of each base station in the network through historical data; then, based on the predicted traffic load, the method of reinforcement learning is used to select a suitable base station for dormancy while ensuring the quality of user service, thereby reducing network energy consumption.

Description

Technical Field

[0001] The present invention belongs to the field of mobile communication technology, and in particular relates to a base station sleep method, device and medium based on reinforcement learning. Background Art

[0002] As the amount of mobile communication data continues to increase significantly, mobile user devices are showing an explosive growth trend, and people's demand for data traffic is growing exponentially. The International Telecommunication Union Radio Standardization Organization (ITU-R) has proposed the concept of ultra-dense networking (UDN). Ultra-dense networks increase the throughput of the system by densely deploying micro base stations to meet people's demand for high speeds. While ultra-dense networks bring high speeds and strong coverage, they also bring unbearable energy consumption problems. Moreover, ultra-dense networks (UDN) and millimeter wave (mmWave) communications are key technologies for future communications. The increase in the number of micro base stations caused by the dense deployment of networks has further increased the energy consumption of mobile communication networks, making energy consumption a key indicator of mobile communications. According to relevant data, base stations consume 60%-80% of the total power consumption of the network, so reducing the energy consumption of base stations is of great significance to improving the energy efficiency of the entire network. Since base stations are deployed based on the peak value of network load, and there is still a large capacity redundancy. The traffic load in the network fluctuates greatly in the space-time dimension. Due to the tidal effect of the business, the time when a single base station is at peak load only occupies a small part of the entire operation cycle. Most of the time, the actual load of the base station is far less than the maximum capacity of the network. When the base station is zero-loaded, the energy consumption of the base station is still as high as 50%-60%, which will inevitably lead to low network energy efficiency (EE) and energy waste. Therefore, during periods of low traffic demand (such as early morning), some access users of low-load base stations can be transferred to adjacent powered-on base stations, which are then switched to sleep mode, thereby significantly reducing the energy consumption of the entire network.

[0003] How to reduce the energy consumption of base stations? The academic and industrial circles have proposed a series of solutions, among which base station dormancy is considered to be one of the best methods, because this solution does not require changing the network architecture and is easy to implement. Base station dormancy needs to reduce the energy consumption level of the network on the one hand, and ensure the service quality of users on the other hand. The power consumption of base stations can be divided into three parts: the first part is the transmission power consumed by the power amplifier and the radio frequency circuit, which accounts for 40%-50% of the energy consumption of the overall base station power consumption; the second part is the power consumed by equipment such as signal processing and air conditioning and the power consumed when the base station state is switched, which accounts for 35% of the energy consumption of the base station; the third part is the base station state conversion power (conversion between active and closed states), which accounts for 15% of the total power consumption of the base station. Ensuring the quality of user service mainly means that after the user accesses a certain base station, the user's reachable network rate is greater than or equal to the required rate. Therefore, base station dormancy not only needs to determine which base stations are dormant, but also needs to consider the number of base station switches and the quality of user service.

[0004] At present, most of the research on base station dormancy is a model-based analysis method. The model-based method can obtain good results in theory, but the model in reality is far from the theoretical model, which makes it difficult to reasonably apply the theoretical model and the corresponding algorithm in real life. Compared with traditional algorithms, machine learning algorithms have shown better performance in preliminary research results. Deep learning, as a model-free method, reduces the complexity of calculation through input and output training in a data-driven way. However, in complex wireless systems, it is difficult to obtain perfect training data, and the training itself is very time-consuming. In this context, reinforcement learning, which belongs to the category of machine learning, is better at radio resource management. Reinforcement learning is an autonomous learning algorithm with autonomous adaptability, thereby empowering future networks. In reinforcement learning, the requirements for system models and prior data are very loose because it has strong autonomous learning capabilities. Summary of the invention

[0005] The purpose of the present invention is to solve the problems in the prior art and propose a base station dormancy method, device and medium based on reinforcement learning. The method predicts the future traffic of the base station through LSTM, determines the candidate base station to be shut down according to the prediction result, and then determines the shutdown state of the base station through reinforcement learning while ensuring the service quality of the user and reducing the working state switching.

[0006] The present invention is implemented by the following technical solution. The present invention proposes a base station sleep method based on reinforcement learning, which specifically includes the following steps:

[0007] Step 1: Assume that the ultra-dense network is a heterogeneous network consisting of one macro base station and N micro base stations, where the macro base station is responsible for controlling the entire network and the micro base station is responsible for business data transmission; the macro base station is always in an active state; the state of the micro base station is divided into two states: active and dormant. The decision problem is modeled as a binary integer programming, and these base stations are denoted as a set BS = {SBS 1 ,...,SBS N}, the traffic load of each base station is counted and recorded every period, and each micro base station uses the historical traffic data to train its own traffic prediction model in the LSTM network;

[0008] Step 2: According to the traffic prediction model obtained in step 1, the traffic value of each micro base station at each moment is counted and input into the LSTM network to obtain the traffic prediction value of the micro base station at the next statistical moment, and according to the traffic load at the current moment and the predicted traffic load, the corresponding reserved resources are set to avoid the communication link congestion caused by burst services;

[0009] Step 3: When the sum of the micro base station traffic prediction value and resource reservation in step 2 is greater than the threshold, the micro base station is added to the candidate shutdown micro base station set;

[0010] Step 4: Count the number of users within the coverage area of ​​the candidate micro base station and the traffic demand of each user; obtain the signal to interference plus noise ratio (SINR) of each user through CSI-RS feedback;

[0011] Step 5: According to the candidate micro base station set to be shut down obtained in step 3 and the traffic demand of the base station service user obtained in step 4, the candidate micro base stations are sorted by the predicted traffic size, and the micro base stations with small traffic have a higher shutdown priority; according to the value function, the users covered by the micro base station with small traffic load are transferred to the adjacent opened micro base station, and the receiving micro base station adjusts the size of its reserved resources according to the current rate demand of the user;

[0012] Step 6. Obtain the current shutdown status set of each base station and the rate at which the micro base stations in the candidate shutdown micro base station set serve users; use the ε-greedy strategy to select the base station shutdown set to obtain the micro base station set to be shut down. If the minimum rate requirement is met, it will be used as an optional shutdown micro base station set, and the reward return value will be calculated and stored in the storage unit memory. Then, data is randomly selected from the memory and input into the network to correct the network model; if the conditions are not met, the set will be discarded.

[0013] Furthermore, the signal-to-interference-to-noise ratio of user j served by micro base station i is denoted as SINR i,j , the current user's transmission rate is recorded as R i,jAccording to Shannon's formula, we can get R i,j =Blog 2 (1+SINR i,j ), where B is the transmission bandwidth.

[0014] Furthermore, in step 1, the historical traffic data of the micro base station is in It represents the traffic data of micro base station i at statistical time t; the LSTM network is used to train the respective LSTM models according to the traffic data of each micro base station.

[0015] Furthermore, the training of each LSTM model is specifically as follows: inputting the collected historical traffic data of the base station, dividing it into a training data set and a test data set, training the LSTM network with the training data, obtaining the LSTM model, inputting the test data set, calculating the error function, and correcting the network model according to the error. When the error is less than the threshold value, the traffic prediction model of the current micro base station is output, and the traffic data at the next statistical moment is predicted.

[0016] Furthermore, for micro base station i, the resource reservation value is determined based on the current traffic data, historical traffic data and the rate of user transfer of micro base station i. When the sum of the resource reservation value and the traffic prediction value is greater than the traffic threshold value, micro base station i is added to the candidate closed micro base station set, otherwise the base station will be in the on state.

[0017] Furthermore, in step 4, assuming that the serving micro base station of user j is i, in the actual network, the base station transmits CSI-RS to the user, and the user performs channel estimation based on the pilot signal to obtain the channel matrix to calculate the downlink signal-to-interference-noise ratio, and then feeds back to the serving base station through the uplink CSI-RS feedback signaling. According to the Shannon formula, R i,j =Blog 2 (1+SINR i,j ), where B is the transmission bandwidth.

[0018] Furthermore, in step 5, based on the traffic demand of the candidate micro base stations and users, the candidate micro base stations are sorted by the predicted traffic size. Micro base stations with small traffic have higher shutdown priorities. For micro base stations with high priorities, the handover of users within the coverage of the micro base station is considered first and handed over to the adjacent micro base stations. When the handover of the service users in the micro base station is completed, the micro base station is shut down.

[0019] Furthermore, in step 6, two identical network eval_model and target_model are initialized first, and only the candidate closed micro base station set is considered, and the state space is the closed state set and user rate of each base station at the current statistical moment; the ε-greedy strategy is used to explore the base station closed set, and a number R is randomly generated. When R is less than epsilon, the state is input into the eval_model to obtain the value function, and the closed base station set is obtained according to the value function; when R is less than epsilon, the candidate micro base station closed state is randomly determined; according to the current base station closed state, if the users covered by each micro base station meet the minimum rate requirement conditions, it is regarded as an optional closed micro base station set, and the reward return value is calculated and stored in the storage unit memory, and then data is randomly selected from the memory and input into the two networks, so as to correct the network model; if the conditions are not met, and this set is discarded, the micro base station that needs to be closed is determined according to the final micro base station decision action.

[0020] The present invention proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a base station sleep method based on reinforcement learning when executing the computer program.

[0021] The present invention proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of a base station sleep method based on reinforcement learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a general flow chart of base station dormancy in the present invention;

[0023] Figure 2 It is a structural diagram of the LSTM framework in the present invention;

[0024] Figure 3 This is a flow chart of using LSTM to predict base station traffic data in the present invention;

[0025] Figure 4 A flowchart for resource reservation in the present invention;

[0026] Figure 5 The flowchart of the base station sleep decision using reinforcement learning in the present invention. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Combination Figure 1-5 The present invention proposes a base station sleep method based on reinforcement learning, which specifically includes the following steps:

[0029] Step 1: Assume that the ultra-dense network is a heterogeneous network consisting of one macro base station and N micro base stations, where the macro base station is responsible for controlling the entire network and the micro base station is responsible for business data transmission; the macro base station is always in an active state; the state of the micro base station is divided into two states: active and dormant. The decision problem is modeled as a binary integer programming, and these base stations are denoted as a set BS = {SBS 1 ,...,SBS N}, the traffic load of each base station is counted and recorded every period, and each micro base station uses the historical traffic data to train its own traffic prediction model in the LSTM network;

[0030] Step 2: According to the traffic prediction model obtained in step 1, the traffic value of each micro base station at each moment (the time granularity is recommended to be 1 hour but not limited to 1 hour) is counted and input into the LSTM network to obtain the traffic prediction value of the micro base station at the next statistical moment, and according to the traffic load at the current moment and the predicted traffic load, the corresponding reserved resources are set to avoid the communication link congestion caused by burst services;

[0031] Step 3: When the sum of the micro base station traffic prediction value and resource reservation in step 2 is greater than the threshold, the micro base station is added to the candidate shutdown micro base station set;

[0032] Step 4: Count the number of users within the coverage area of ​​the candidate micro base station and the traffic demand of each user; obtain the signal to interference plus noise ratio (SINR) of each user through CSI-RS (Channel State Information Reference Signal) feedback;

[0033] Step 5: According to the candidate micro base station set to be shut down obtained in step 3 and the traffic demand of the base station service user obtained in step 4, the candidate micro base stations are sorted by the predicted traffic size, and the micro base stations with small traffic have a higher shutdown priority; according to the value function, the users covered by the micro base station with small traffic load are transferred to the adjacent opened micro base station, and the receiving micro base station adjusts the size of its reserved resources according to the current rate demand of the user;

[0034] Step 6. Obtain the current shutdown status set of each base station and the rate at which the micro base stations in the candidate shutdown micro base station set serve users; use the ε-greedy strategy to select the base station shutdown set to obtain the micro base station set to be shut down. If the minimum rate requirement is met, it will be used as an optional shutdown micro base station set, and the reward return value will be calculated and stored in the storage unit memory. Then, data is randomly selected from the memory and input into the network to correct the network model; if the conditions are not met, the set will be discarded.

[0035] The signal-to-interference-to-noise ratio of user j served by micro base station i is denoted as SINR i,j , the current user's transmission rate is recorded as R i,j According to Shannon's formula, we can get R i,j =Blog 2 (1+SINR i,j ), where B is the transmission bandwidth.

[0036] In step 1, the historical traffic data of the micro base station is in It represents the traffic data of micro base station i at statistical time t; the LSTM network is used to train the respective LSTM models according to the traffic data of each micro base station.

[0037] The training of each LSTM model is specifically as follows: input the collected historical traffic data of the base station, divide it into a training data set and a test data set, train the LSTM network with the training data, obtain the LSTM model, input the test data set, calculate the error function, and correct the network model according to the error. When the error is less than the threshold value, output the traffic prediction model of the current micro base station, and predict the traffic data at the next statistical moment (either a single-step prediction or a multi-step prediction).

[0038] According to step 1, the traffic prediction value of each micro base station is obtained. In order to avoid insufficient communication resources due to business bursts, resource reservation is introduced. For micro base station i, the resource reservation value is determined according to the current traffic data, historical traffic data and the rate of user transfer of micro base station i. When the sum of the resource reservation value and the traffic prediction value is greater than the traffic threshold value, micro base station i is added to the candidate closed micro base station set, otherwise the base station will be in the open state.

[0039] In step 4, it is assumed that the serving micro base station of user j is i. In the actual network, the base station transmits CSI-RS to the user. The user performs channel estimation based on the pilot signal to obtain the channel matrix and calculate the downlink signal-to-interference-noise ratio, and then feeds it back to the serving base station through the uplink CSI-RS feedback signaling. According to the Shannon formula, R i,j =Blog 2 (1+SINR i,j ), where B is the transmission bandwidth.

[0040] In step 5, based on the traffic requirements of the candidate micro base stations and users, the candidate micro base stations are sorted by predicted traffic size. Micro base stations with small traffic have higher shutdown priorities. For micro base stations with high priorities, the handover of users within the coverage of the micro base station is considered first and handed over to adjacent micro base stations. When the handover of service users within the micro base station is completed, the micro base station is shut down.

[0041] In step 6, two identical networks eval_model and target_model are initialized first, and only the candidate closed micro base station set is considered. The state space is the closed state set and user rate of each base station at the current statistical moment; the ε-greedy strategy is used to explore the base station closed set, and a number R is randomly generated. When R is less than epsilon, the state is input into the eval_model to obtain the value function, and the closed base station set is obtained according to the value function; when R is less than epsilon, the candidate micro base station closed state is randomly determined; according to the current base station closed state, if the users covered by each micro base station meet the minimum rate requirement conditions, it is regarded as an optional closed micro base station set, and the reward return value is calculated and stored in the storage unit memory, and then data is randomly selected from the memory and input into the two networks to correct the network model; if the conditions are not met, the set is discarded, and the micro base station that needs to be closed is determined according to the final micro base station decision action.

[0042] The present invention proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a base station sleep method based on reinforcement learning when executing the computer program.

[0043] The present invention proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of a base station sleep method based on reinforcement learning.

[0044] Example

[0045] The present invention provides a base station sleep method based on reinforcement learning for base station sleep in ultra-dense networks. The specific method is: Figure 1 As shown in the figure, a deep reinforcement learning model is established to predict the future traffic load of each base station in the ultra-dense network and determine the shutdown status of each base station in the ultra-dense network. This method introduces the LSTM model and uses historical data to predict the future traffic load of each base station in the network; then, based on the predicted traffic load, the reinforcement learning method is used to select appropriate base stations for hibernation while ensuring the quality of user service, thereby reducing network energy consumption.

[0046] The base station dormancy method based on reinforcement learning includes the following steps:

[0047] (I) Training the traffic prediction model for micro base stations

[0048] Before introducing the traffic prediction model for training micro base stations, the following LSTM framework is introduced. Figure 2 , recurrent neural networks are often used for time series prediction, which allows information to persist, thereby using knowledge from previous data to predict future information. LSTM network is a special recurrent neural network that avoids the gradient vanishing problem in recurrent neural networks by introducing cell states and different gates. The standard LSTM unit has three gates that interact with each other for learning, namely the input gate, forget gate, and output gate. These gates are defined as:

[0049] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0050] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0051]

[0052]

[0053] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0054] h t =o t ⊙tanh(C t )

[0055] Among them, σ(·) is the sigmoid function, ⊙ is the Hadamard product, W and b are the weight matrix and bias matrix. Next, we introduce the traffic prediction model for training micro base stations.

[0056] refer to Figure 3 ,Training the base station traffic prediction model mainly includes the following steps:

[0057] (1) First build the LSTM network structure model and initialize the LSTM network;

[0058] (2) Input the collected base station historical traffic data and divide it into a training data set and a test data set;

[0059] (3) Train the LSTM network with training data to obtain the LSTM model;

[0060] (4) Input the test data set, calculate the error function, and correct the network model according to the error. When the error is less than the threshold value, output the traffic prediction model of the current micro base station and predict the traffic data at the next statistical moment (single-step prediction or multi-step prediction)

[0061] When it comes to base station dormancy, since the granularity of traffic data is much smaller than the time interval for switching base station states, multi-step prediction is required. Assuming that each statistical time is δ, when predicting Z steps (Z statistical times), the prediction interval can be expressed as T pred =δZ, so at T pred The total business data volume during the time period is expressed as an average value, which is:

[0062]

[0063] in, is the traffic data value of base station n at time j.

[0064] (II) Setting the Base Station Reserved Resource Ratio

[0065] refer to Figure 4 By training on the historical traffic data of the base station, a fuzzy prediction model of the base station traffic can be obtained. The base station reserved resource size is determined according to the predicted base station traffic data value and the maximum error value obtained from historical training. The size of the resource reservation value can also be manually set according to the network conditions. When the sum of the resource reservation value and the traffic prediction value is greater than the traffic threshold value, micro base station i is added to the candidate closed micro base station set, otherwise the base station will be in the open state.

[0066] (III) Selecting a dormant base station

[0067] Assume that the serving micro base station of user j is i. In the actual network, the base station transmits CSI-RS to the user. The user performs channel estimation based on the pilot signal to obtain the channel matrix and calculate the downlink signal-to-interference-noise ratio. Then, the uplink CSI-RS feedback signaling can be fed back to the serving base station. According to the Shannon formula, R i,j =Blog 2 (1+SINR i,j ), where B is the transmission bandwidth.

[0068] Based on the above description of the power consumption of various aspects of the base station, the power consumption of the base station can be divided into three parts: the transmission power consumed by the power amplifier and the RF circuit Power consumed by equipment such as signal processing and air conditioning Power consumed when the base station switches from active to sleep Therefore, the total power consumption of time slot t can be expressed as:

[0069]

[0070] The power consumption caused by the power amplifier and RF circuit of the base station can be expressed as:

[0071]

[0072] The energy consumption of signal processing and air conditioning equipment can be expressed as:

[0073]

[0074] In summary, the power consumption of the t-time slot network is:

[0075]

[0076] In order to jointly minimize the switching power, device consumption, and transmission power, this paper considers the energy consumption of T time slots:

[0077]

[0078] Therefore, the energy efficiency can be obtained as:

[0079]

[0080] Finally, the problem becomes to minimize E total The base station transmission power and air conditioning equipment power are determined before leaving the factory. The only thing that can be changed is p m,k and The restrictions are:

[0081]

[0082] st

[0083]

[0084]

[0085] The following discusses how to combine reinforcement learning and base station sleep strategy. First, each micro base station can be regarded as an agent, and its state space, behavior space, and reward function are explained below.

[0086] (a) State space: The minimum rate of service required by each user in the current time slot is:

[0087]

[0088] The base station mode state matrix of the previous time slot:

[0089]

[0090] (b) Action space: The action space is the sleep strategy of the base station in the current time slot.

[0091] Reward: When the sleep strategy is adopted in the current time slot, excessive power consumption of the network will cause a penalty, and the reward can be set as: At this point, the problem of maximizing rewards becomes a problem of minimizing energy consumption.

[0092] refer to Figure 5 First, initialize two identical networks eval_model and target_model, only consider the candidate closed micro base station set, and the state space is the closed state set and user rate of each base station at the current statistical moment. Use the ε-greedy strategy to explore the closed base station set, randomly generate a number R, when R is less than ε, input the state into eval_model, get the value function, and get the closed base station set according to the value function; when R is less than ε, randomly determine the closed state of the candidate micro base station. According to the current closed state of the base station, if the users covered by each micro base station meet the minimum rate requirement, it will be regarded as an optional closed micro base station set, and the reward return value will be calculated and stored in the storage unit memory, and then randomly select data from the memory and input it into the two networks to correct the network model; if it does not meet the conditions, the set will be discarded. According to the final micro base station decision action, determine the micro base station that needs to be closed.

[0093] The above is a detailed introduction to the base station sleep method, device and medium based on reinforcement learning proposed in the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A base station sleep method based on reinforcement learning, characterized in that: The specific steps include: Step 1: Assume that the ultra-dense network is a heterogeneous network consisting of one macro base station and N micro base stations, where the macro base station is responsible for controlling the entire network and the micro base station is responsible for business data transmission; the macro base station is always in an active state; the state of the micro base station is divided into two states: active and dormant. The decision problem is modeled as a binary integer programming, and the N micro base stations are denoted as the set BS = {SBS1,...,SBS N }, the traffic load of each base station is counted and recorded every period, and each micro base station uses the historical traffic data to train its own traffic prediction model in the LSTM network; Step 2: According to the traffic prediction model obtained in step 1, the traffic value of each micro base station at each moment is counted and input into the LSTM network to obtain the traffic prediction value of the micro base station at the next statistical moment, and according to the traffic load at the current moment and the predicted traffic load, the corresponding reserved resources are set to avoid the communication link congestion caused by burst services; Step 3: When the sum of the micro base station traffic prediction value and resource reservation in step 2 is less than the threshold, the micro base station is added to the candidate shutdown micro base station set; Step 4: Count the number of users within the coverage area of ​​the candidate micro base station and the traffic demand of each user; obtain the signal to interference plus noise ratio (SINR) of each user through CSI-RS feedback; Step 5: According to the candidate micro base station set to be shut down obtained in step 3 and the traffic demand of the base station service user obtained in step 4, the candidate micro base stations are sorted by the predicted traffic size, and the micro base stations with small traffic have a higher shutdown priority; according to the value function, the users covered by the micro base station with small traffic load are transferred to the adjacent opened micro base station, and the receiving micro base station adjusts the size of its reserved resources according to the current rate demand of the user; Step 6: Obtain the current shutdown state set of each base station and the rate of micro base stations serving users in the candidate micro base station set; use the ε-greedy strategy to select the base station shutdown set to obtain the micro base station set to be closed. If the minimum rate requirement is met, it will be used as an optional micro base station set to be closed, and the reward return value will be calculated and stored in the storage unit memory. Then, data is randomly selected from the memory and input into the network to correct the network model; if it does not meet the conditions, the set is discarded; In step 1, the historical traffic data of the micro base station is in r i t represents the traffic data of micro base station i at statistical time t; the LSTM network is used to train the respective LSTM models according to the traffic data of each micro base station; In step 6, two identical networks eval_model and target_model are initialized first, and only the candidate closed micro base station set is considered. The state space is the closed state set and user rate of each base station at the current statistical moment; the ε-greedy strategy is used to explore the base station closed set, and a number R is randomly generated. When R is less than epsilon, the state is input into the eval_model to obtain the value function, and the closed base station set is obtained according to the value function; when R is less than epsilon, the candidate micro base station closed state is randomly determined; according to the current base station closed state, if the users covered by each micro base station meet the minimum rate requirement conditions, it is regarded as an optional closed micro base station set, and the reward return value is calculated and stored in the storage unit memory, and then data is randomly selected from the memory and input into the two networks to correct the network model; if the conditions are not met, the set is discarded, and the micro base station that needs to be closed is determined according to the final micro base station decision action.

2. The method according to claim 1, characterized in that The signal-to-interference-to-noise ratio of user j served by micro base station i is denoted as SINR i,j , the current user's transmission rate is recorded as R i,j According to Shannon's formula, we can get R i,j =Blog2(1+SINR i,j ), where B is the transmission bandwidth.

3. The method according to claim 2, characterized in that The specific training of each LSTM model is as follows: input the collected historical traffic data of the base station, divide it into a training data set and a test data set, train the LSTM network with the training data, obtain the LSTM model, input the test data set, calculate the error function, and correct the network model according to the error. When the error is less than the threshold value, output the traffic prediction model of the current micro base station and predict the traffic data at the next statistical moment.

4. The method according to claim 3, characterized in that For micro base station i, the resource reservation value is determined based on the current traffic data, historical traffic data and the rate of user handover of micro base station i. When the sum of the resource reservation value and the traffic prediction value is less than the traffic threshold value, micro base station i is added to the candidate shutdown micro base station set, otherwise the base station will be in the on state.

5. The method according to claim 4, characterized in that In step 4, it is assumed that the serving micro base station of user j is i. In the actual network, the base station transmits CSI-RS to the user. The user performs channel estimation based on the pilot signal to obtain the channel matrix and calculate the downlink signal-to-interference-noise ratio, and then feeds it back to the serving base station through the uplink CSI-RS feedback signaling. According to the Shannon formula, R i,j =Blog2(1+SINR i,j ), where B is the transmission bandwidth.

6. The method according to claim 5, characterized in that In step 5, based on the traffic requirements of the candidate micro base stations and users, the candidate micro base stations are sorted by predicted traffic size. Micro base stations with small traffic have higher shutdown priorities. For micro base stations with high priorities, the handover of users within the coverage of the micro base station is considered first and handed over to adjacent micro base stations. When the handover of service users within the micro base station is completed, the micro base station is shut down.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images