5G base station energy-saving decision-making method based on causal Decision ConvFormer model

Abstract

The invention discloses a 5G base station energy-saving decision-making method based on a causal Decision ConvFormer model, and the method comprises the steps: carrying out the cumulative return calculation of trajectory data obtained through interaction with a 5G base station-photovoltaic-battery environment, sampling a part of trajectory fragment sequence, and carrying out the position coding of the state, action and cumulative return in the sequence; carrying out dimension segmentation processing on the coded vector, and calculating multi-scale feature information of a base station track fragment which can be used for decision making by using a double-window convolution filter; based on multi-scale features, adopting a supervised learning training model, and outputting a prediction action 1, a prediction state and a prediction reward in the first round; splicing the prediction state and the reward, and inputting the spliced prediction state and reward into an additional prediction layer to generate a prediction action 2; and finally, integrating the predicted action 1 and the predicted action 2 to obtain an optimal decision. And model parameters are optimized by combining cross entropy loss and mean square error loss, so that the decision precision and stability are ensured. The method has good decision-making precision and good time overhead, and can have good generalization ability in multiple scenes.

Description

Technical Field

[0001] This invention relates to the field of decision-making, and in particular to a 5G base station energy-saving decision-making method based on the causal Decision ConvFormer model. Background Technology

[0002] The energy consumption of 5G base stations is approximately four times that of 4G base stations, and the high operating costs have become one of the key challenges restricting the sustainable development of 5G. Furthermore, the complex energy-saving system of 5G base stations, photovoltaic systems, and batteries presents challenges in decision-making.

[0003] Existing decision-making methods include optimization methods and reinforcement learning. Optimization methods (such as greedy algorithms) can get stuck in local optima, making it difficult to approach the global optimum. Online reinforcement learning methods suffer from insufficient safety due to the risky actions that can cause irreversible damage to the environment during online exploration. Offline reinforcement learning methods, especially the sequential reinforcement learning method Decision ConvFormer (DC), are able to guarantee both safety and accuracy because they are trained offline and leverage the advantages of the Transformer architecture. However, DC still has limitations. DC excels at handling scenarios with strong Markov properties, but its weaknesses become apparent when dealing with scenarios with weak Markov properties. Summary of the Invention

[0004] The purpose of this invention is to provide a 5G base station energy-saving decision-making method based on causal decision ConvFormer, which solves the limitations of traditional DC.

[0005] To achieve the above objectives, this invention provides a 5G base station energy-saving decision-making method based on causal Decision ConvFormer, the specific steps of which are as follows:

[0006] S1, Sample Collection:

[0007] Sample data including rewards, actions, and states are collected from the 5G base station-photovoltaic-cell environment, and the cumulative reward is calculated; the rewards in the sample data are replaced with the cumulative reward to form a trajectory sequence including cumulative reward, actions, and states for training.

[0008] S2, Model Training:

[0009] Based on the trajectory sequence in step S1, the causal Decision ConvFormer model is trained to obtain the base station energy-saving decision prediction model.

[0010] S3: Decision Reasoning

[0011] By utilizing the base station energy-saving decision prediction model, predictive actions for energy-saving decisions are obtained.

[0012] As a further optimization of the present invention, the states in step S1 include base station traffic, photovoltaic power generation, battery status, effective battery capacity, battery discharge depth, and electricity price; the actions include the battery charging status and power value; and the reward is: , It is the reward at time step t. , , These represent the base station power consumption, photovoltaic power generation, and battery power supply at time step t, respectively. The electricity price is at time step t. This refers to the battery's device wear at time step t.

[0013] As a further optimization of the present invention, the formula for calculating the cumulative reward in step S1 is as follows:

[0014] ,

[0015] in, This represents the cumulative reward at time step t, where T is the total number of time steps in the trajectory sequence. Indicates in Rewards for time steps.

[0016] As a further optimization of the present invention, step S2 includes:

[0017] S21, convert the trajectory sequence into vector form and embed the position encoding information into the vector;

[0018] S22, the vector containing the embedded position encoding information is divided into two dimensional parts, and convolution kernels with different preset window sizes are used to move on the two parts to calculate multi-scale feature information, and the calculated multi-scale feature information is added along the third dimension. The length of the resulting sequence remains unchanged from the length of the input sequence.

[0019] S23, The sequence length obtained in step S22 is input into the causal Decision ConvFormer model, which outputs the first predicted action, predicted state, and predicted reward, and calculates the loss of each of the three with the corresponding true value.

[0020] S24. The predicted state and predicted reward are connected together to form new feature information. The new feature information is input into the multilayer perceptron layer, outputting the second predicted action. The loss between the second predicted action and the corresponding real action is calculated and added to the loss calculated in S23 as a joint loss to update the parameters of the causal Decision ConvFormer model.

[0021] S25, after adding the probability distributions of the first and second predicted actions, the final predicted action is obtained through the activation layer.

[0022] As a further optimization of the present invention, in step S22, the vector containing the embedded position encoding information is dimensionally split at a 1:1 ratio.

[0023] As a further optimization of the present invention, in step S22, a local window and two windows are set. In each window, a filter is set for cumulative reports, actions and rewards respectively. The window moves on the sequence and calculates the relevant information in the current window. Each window calculates a sequence with the same length as the input sequence. Then the two sequences are added together to obtain the final sequence with multi-scale feature information. The size of the local window is smaller than the size of the global window.

[0024] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and when the program is executed on the processor, it implements the steps in the 5G base station energy-saving decision method based on the causal DecisionConvFormer model as described above.

[0025] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps in the 5G base station energy-saving decision-making method based on the causal Decision ConvFormer model as described above.

[0026] This invention proposes a 5G base station energy-saving decision-making method based on the causal Decision ConvFormer model. It improves upon the DC model by first using a dual-window approach to capture features of the 5G base station energy-saving system at different scales, which enhances the model's adaptability to Markov scenarios of varying strengths. Secondly, a causal module is added to the DC framework to improve the model's decision accuracy. Finally, to address the increased training time resulting from the first two improvements, this invention uses dimensionality partitioning to reduce computational complexity. This invention enables the model to achieve higher decision accuracy, good generalization, and reasonable training time overhead. The improved model can reduce base station operating costs, achieve peak shaving and valley filling, and its good generalization and reasonable training time overhead make this method more practically valuable. Attached Figure Description

[0027] Figure 1 This is a system block diagram of a 5G base station energy-saving decision-making method based on the causal Decision ConvFormer model.

[0028] Figure 2 This is a flowchart of a 5G base station energy-saving decision-making method based on the causal Decision ConvFormer model.

[0029] Figure 3 This is an environmental diagram of a 5G base station energy-saving decision-making method based on the causal Decision ConvFormer model.

[0030] Figure 4 This is a simulation diagram of the training of a 5G base station energy-saving decision-making method based on the causal Decision ConvFormer model.

[0031] Figure 5 This is a generalization effect diagram of the 5G base station energy-saving decision-making method based on the causal Decision ConvFormer model.

[0032] Figure 6 This is a comparison chart of the time overhead of 5G base station energy-saving decision-making methods based on the Decision Causal ConvFormer model. Detailed Implementation

[0033] In complex decision-making scenarios involving 5G base stations, photovoltaic systems, and batteries, there are periods with strict Markov characteristics, such as low electricity prices, and periods with weak Markov characteristics, such as high electricity prices. The timing of battery charging and discharging significantly determines the energy efficiency of the base station. Furthermore, the irreversible damage to the battery caused by the number of charge-discharge cycles must be considered. When the decision-making entity no longer relies on a single piece of information, traditional decision-making methods struggle to achieve optimal results. Employing offline reinforcement learning methods, particularly a decision-making method based on a causal Decision ConvFormer model, can help the system adjust to its optimal state, achieving maximum peak shaving and valley filling.

[0034] Therefore, this invention proposes a 5G base station energy-saving decision-making method based on causal Decision ConvFormer, the specific steps of which are as follows:

[0035] S1: Collect sample data from the 5G base station energy-saving system environment and perform preprocessing.

[0036] S2: Input the trajectory sequence into the causal Decision ConvFormer (DCC) model for training. The steps in each training round include:

[0037] S21: Embedding and position encoding, converting the trajectory sequence into a vector form and embedding position encoding information into the vector.

[0038] S22: Perform dual-window convolution calculation in the causal convolution machine. First, normalize the vector by layer, then divide the vector into two dimensional parts. Convolution kernels of different window sizes move on the divided vector to calculate multi-scale feature information, and add the calculated multi-scale feature information together. Keep the sequence length unchanged, then normalize by layer, and finally enter the feedforward layer to increase the nonlinearity and expressiveness of the model.

[0039] S23: Generate prediction information and calculate loss. The causal Decision ConvFormer (DCC) model generates prediction action 1 (prediction action 1 refers to the complete action dimension, i.e., charge / discharge state and battery power value, but is limited to the prediction in the first stage), prediction state, and prediction reward, and calculates the loss of each of the three to the corresponding true value.

[0040] S24: Causal reinforcement connects the predicted state and the predicted reward to form new feature information. The new feature information is input into the causal reinforcement module, which outputs predicted action 2 (complete action dimension). The loss between predicted action 2 and the real action is calculated and added to the loss calculated in S23 as the joint loss to update the model parameters. The probability distributions of predicted action 1 and predicted action 2 are added together and passed through the activation layer to obtain the final predicted action.

[0041] S3: Decision Reasoning

[0042] By utilizing the base station energy-saving decision prediction model, predictive actions for energy-saving decisions are obtained.

[0043] The following is combined Figure 1 and Figure 2 The specific implementation of the 5G base station energy-saving decision-making method based on the causal Decision ConvFormer model of the present invention is described in detail below:

[0044] Step S1: Collect trajectory data from the base station-photovoltaic-cell environment and calculate the cumulative reward.

[0045] By reward ,state and actions The resulting triplet is: The status includes: base station traffic, photovoltaic power generation, battery status, battery effective capacity, battery discharge depth, electricity price, etc.; the actions include: adjusting the charging status (charging, not charging or discharging, discharging), and adjusting the battery power value (selecting from the range of minimum power to maximum power).

[0046] The specific calculation method of the reward function is as follows:

[0047]

[0048] in, It is the reward at time step t. , , These represent the base station power consumption, photovoltaic power generation, and battery power supply at time step t, respectively. The electricity price is at time step t. This refers to the battery's device wear at time step t.

[0049] In step S11, multiple reward-state-action triples form a time-ordered base station decision trajectory data: .

[0050] Step S12: Base station energy-saving decisions focus on long-term returns, so the cumulative return is calculated. After replacing the reward, we get: trajectory sequence ;

[0051] The formula for calculating cumulative returns is as follows: ,in, This represents the cumulative reward at time step t, where T is the total number of time steps in the trajectory sequence. Indicates in Rewards for time steps.

[0052] Step S2: Encode the base station energy-saving decision trajectory data into a high-dimensional representation to show the complex relationships between the data. Simultaneously, set two windows of different sizes to extract the information required for base station energy-saving decisions. The local window is set based on Markov properties, focusing only on the triples from the previous time step and the state of the current time step. , where t represents the current time step; the expanded window is based on the weak Markov property to obtain decision information over a longer time period. Two windows process their respective dimensions to reduce computational cost, and the loss rate is used to prevent overfitting and allow the method to perform better on unknown data.

[0053] In this embodiment, the embedding dimension is set to 128, the local window is set to 6, the expanded window is set to 50, the dimension splitting ratio is 1:1, the loss rate is set to 0.1, and the learning rate is set to 5.5 * 10^6. -4 The above parameters are all the optimal parameters obtained through experimental verification.

[0054] In this embodiment, the base station energy-saving decision involves two discrete actions, employing the cross-entropy loss function. The state and reward are continuous values, using the mean squared error (MSE) loss function. The joint loss function formula is as follows: ,in and These are the cross-entropy loss functions for two discrete actions. It is the mean squared error loss function for both state and reward. During training, a fixed batch of trajectory segments is sampled from the trajectory data.

[0055] Step S21, process the sampled trajectory segments Embedding and positional encoding are performed, specifically by passing the vector through a linear layer to obtain the embedding vector.

[0056] Step S22 involves segmenting the embedded vectors according to a predetermined 1:1 dimension ratio, followed by convolution calculation. Windows of different sizes are moved across the segmented vectors to calculate multi-scale features. These windows are implemented as filters to focus only on the contextual information within the window. Since actions, states, and cumulative rewards have different meanings, three independent dual-window filters are used for separate calculations, i.e., three dual-window filters are constructed to update their respective parameters. The calculated multi-scale feature information is then summed, while the sequence length remains unchanged.

[0057] The formula for calculating the window size is:

[0058]

[0059]

[0060] in, and These refer to the size of the local window and the size of the window, respectively. These are the convolutional filter parameters for the cumulative reward, state, and action of the local and expanded windows, respectively. It is the embedded vector. and These are the convolution outputs for the local window and the expanded window, respectively. `p` represents the position index of R, `s`, and `a`. Taking the modulo of the position index determines which formula branch to use. `q` represents the dimension. It's important to note that each line of the window formula takes the modulo of the position index, handling cumulative reward, state, and action respectively.

[0061] Step S23: Add the corresponding convolutional feature information calculated for different window sizes, and then input the convolutional feature information obtained after addition into the DCC model. The DCC model outputs the predicted action 1, the predicted state, and the predicted reward.

[0062] Step S24: The causal enhancement module establishes a mapping relationship between the current time step state and reward of the 5G base station energy-saving decision and the action. This can correct the redundant information decision made by the DCC model when facing the complex information of the 5G base station energy-saving system. In the joint layer of the causal module, the predicted state and predicted reward output by the DCC model are concatenated into a new feature. The new feature is input into a multilayer perceptron (MLP) layer. The MLP layer outputs predicted action 2. Combining the predicted action 1, predicted state, and predicted reward from step S23, the loss is calculated using the joint loss function L and the model parameters are updated. Predicted action 1 and predicted action 2 are added to obtain the merged predicted action. The merged predicted action is transformed into a probability distribution layer through a linear layer. The activation layer selects the action with the largest probability of the merged predicted action as the final action. After multiple iterations until the model converges (reward converges), the final decision model is obtained.

[0063] The joint loss function is:

[0064] ,

[0065] The first two terms are the cross-entropy loss functions for predicting action 1 and predicting action 2. , The last term is the MSE loss function for state and reward. Where K represents the length of the trajectory sequence, and These are prediction action 1 and prediction action 2, respectively. It is a predicted state. It predicts the reward.

[0066] In the decision-making and reasoning stage, the DCC model is deployed in the base station environment. The data collected from the sensors is processed in step S1 and then input into the DCC model. The model finally outputs a predicted action for decision-making.

[0067] To verify the authenticity of this invention, a simulation experiment was conducted, such as... Figure 3 As shown, the DCC model is trained using collected offline data, and the trained model interacts with the environment to make decisions. Figure 4 This is a simulation diagram of the training of a 5G base station energy-saving decision-making method based on the causal DecisionConvFormer model. It can be seen from the simulation diagram that the DCC model can converge stably to a certain level during training. Figure 5 This is a comparison chart of the generalization of a 5G base station energy-saving decision-making method based on the causal Decision ConvFormer model. It can be seen that the DCC model performs well in multiple scenarios. Figure 6This is a comparison chart of the training time overhead of a 5G base station energy-saving decision-making method based on the causal Decision ConvFormer model. The chart shows that the dimension segmentation processing of the DCC model effectively reduces the time complexity increased by improving decision accuracy.

[0068] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the program, when executed on the processor, implements the steps in the 5G base station energy-saving decision method based on the causal Decision ConvFormer model as described above.

[0069] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps in the 5G base station energy-saving decision-making method based on the causal Decision ConvFormer model as described above.

[0070] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0071] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the modules described above can be referred to the corresponding process in the aforementioned method implementation, and will not be repeated here.

[0072] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0073] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in a combination of hardware and software functional modules.

[0074] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer system (which may be a personal computer, server, or network system, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A 5G base station energy saving decision method based on a causal Decision ConvFormer model, characterized in that, The method comprises the following steps: S1, sample collection: Collecting sample data including rewards, actions and states from a 5G base station-photovoltaic-cell environment, and calculating cumulative reward returns; replacing the rewards in the sample data with the cumulative reward returns to form a trajectory sequence including cumulative reward returns, actions and states for training; S2, model training: Based on the trajectory sequence in step S1, the causal Decision ConvFormer model is trained to obtain a base station energy-saving decision prediction model; S3: decision reasoning: Using the base station energy-saving decision prediction model, the predicted action for energy-saving decision is obtained.

2. The method of claim 1, wherein, The state in the step S1 includes base station traffic, photovoltaic power generation, battery state, battery effective capacity, battery discharge depth and electricity price; the action includes battery charging state and electric power value; the reward is: , is the reward at t time step, , , are respectively base station power consumption, photovoltaic power generation and battery power supply at t time step, is the electricity price at t time step, is the device loss of the battery at t time step.

3. The method of claim 1, wherein, The calculation formula of the cumulative reward return in step S1 is: ， wherein, represents the cumulative reward return at time step t, T is the total time steps of the trajectory sequence, represents the reward at time step t.

4. The method of claim 1, wherein, The step S2 comprises: S21, converting the trajectory sequence into a vector form, and embedding position encoding information into the vector; S22, dividing the vector embedded with position encoding information into two dimensional parts, using convolution kernels with different preset window sizes to move and calculate multi-scale feature information on the two parts respectively, and adding the calculated multi-scale feature information along the third dimension, the sequence length obtained is unchanged from the input sequence length; S23, inputting the sequence length obtained in step S22 into the causal Decision ConvFormer model, outputting the first predicted action, the predicted state and the predicted reward, and calculating the loss of each with the corresponding true value; S24, the predicted state and the predicted reward are connected together to form new feature information, the new feature information is input into the multi-layer perception layer, the second predicted action is output, and the loss of the second predicted action and the corresponding true action is calculated, and the loss calculated in S23 is added as a joint loss to update the parameters of the causal Decision ConvFormer model; S25, after adding the probability distribution of the first predicted action and the second predicted action, the final predicted action is obtained through the activation layer.

5. The method of claim 4, wherein, In step S22, the vector embedded with position encoding information is dimensionally divided in a 1:1 ratio.

6. The method of claim 4, wherein, In step S22, a local and two windows are set, a filter is set for cumulative returns, actions and rewards in each window, the window moves on the sequence and calculates the related information in the current window, each window calculates a sequence equal to the input sequence length, then the two sequences are added to obtain the final sequence with multi-scale feature information; wherein the size of the local window is smaller than the size of the global window.

7. An electronic device comprising a memory and a processor, characterized in that The memory stores a computer program executable on the processor, and the processor executes the program to implement the steps in the method of any one of claims 1-6.

8. A storage medium storing a computer program, characterized by The computer program is executed by the processor to implement the steps in the method of any one of claims 1-6.

Images