5g base station energy consumption dynamic optimization adjustment system based on deep reinforcement learning

Abstract

The present application relates to the technical field of wireless communication, especially to a 5G base station energy consumption dynamic optimization and adjustment system based on deep reinforcement learning, comprising a feature perception module, used for collecting operation indexes and wireless environment logs of each 5G base station in a collaborative area, performing space-time alignment and normalization processing on the operation indexes and the wireless environment logs, and generating a state feature map containing the connection number, power consumption value and adjacent area coverage overlap degree of each 5G base station. In the present application, the adjacent area coverage overlap degree between base stations is extracted through a graph convolution network, and a deep reinforcement learning model is combined to make a collaborative decision, the mutual offset effect between the target base station hibernation energy saving and the adjacent base station power compensation cost is calculated when formulating an adjustment strategy, the invalid collaboration phenomenon caused by the rapid increase of energy consumption of surrounding base stations due to local single station hibernation is reduced, and the overall energy efficiency performance of the collaborative area is improved.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and in particular to a 5G base station energy consumption dynamic optimization and adjustment system based on deep reinforcement learning. Background Technology

[0002] With the large-scale deployment of 5G communication networks, the construction density of base station equipment and the energy consumption of a single station are increasing. In order to reduce the daily operating costs of the network, the industry usually adopts a base station hibernation mechanism to shut down the radio frequency resources of some base stations during the low-end of network services, and relies on the surrounding adjacent base stations to appropriately increase the transmission power to compensate for the signal coverage of the hibernation area, thereby achieving energy saving and consumption reduction of the communication network.

[0003] Existing base station energy-saving solutions are mostly limited to single-site perspective optimization when formulating sleep strategies, lacking comprehensive consideration of coverage coupling relationship and energy transfer effect between base stations. As a result, after the target base station enters sleep mode, neighboring base stations increase transmission power to fill signal blind spots. The extra power consumption generated by this compensatory behavior can easily offset the energy-saving benefits brought by the target base station's sleep mode, resulting in ineffective collaboration with little overall energy efficiency improvement in the collaborative area. Summary of the Invention

[0004] To address the above shortcomings, this invention provides a 5G base station energy consumption dynamic optimization and adjustment system based on deep reinforcement learning, aiming to improve the problem that existing base station energy-saving solutions lack comprehensive consideration of the coverage coupling relationship and energy transfer effect between base stations.

[0005] This invention provides the following technical solution: a 5G base station energy consumption dynamic optimization and adjustment system based on deep reinforcement learning, comprising:

[0006] The feature perception module is used to collect the operation indicators and wireless environment logs of each 5G base station in the collaborative area, perform spatiotemporal alignment and normalization processing on the operation indicators and wireless environment logs, and generate a state feature map containing the connection number, power consumption value and neighboring cell coverage overlap of each 5G base station.

[0007] The traffic prediction module, based on the state feature map, uses a long short-term memory network to extract the spatiotemporal feature vectors from the state feature map and generates a spatiotemporal load prediction matrix within a time window.

[0008] The graph feature aggregation module, based on the spatiotemporal load prediction matrix, uses a graph convolutional network to perform topology node mapping and edge weight calculation on the spatiotemporal load prediction matrix, extracts the correlation data between each of the 5G base stations, and generates a coupled feature matrix.

[0009] The collaborative decision-making module, based on the coupling feature matrix, inputs the coupling feature matrix into a deep reinforcement learning model with a centralized training distributed execution architecture for forward propagation calculation, and generates an initial policy action set containing the reconfiguration actions of radio frequency resources of each 5G base station in the collaborative area;

[0010] The constraint filtering module calculates the predicted network parameters of each 5G base station based on the initial policy action set, compares the predicted network parameters with the network constraint threshold, eliminates radio frequency resource reconfiguration actions that do not meet the network constraint threshold, and generates control commands.

[0011] The execution feedback module, based on the control command, sends the control command to the baseband processing unit and radio frequency unit of each of the 5G base stations, controls each of the 5G base stations to perform the radio frequency resource reconfiguration action, and collects the network status feedback value after the radio frequency resource reconfiguration action and sends it back to the feature perception module.

[0012] Preferably, in the feature perception module, generating a state feature map containing the connection count, power consumption value, and neighboring cell coverage overlap of each 5G base station specifically includes the following steps:

[0013] The operational metrics after spatiotemporal alignment and normalization are analyzed to extract the connection count and power consumption of each 5G base station.

[0014] Read the user equipment measurement report in the wireless environment log after the spatiotemporal alignment and normalization processing, calculate the signal overlap ratio between each of the 5G base stations that have an adjacent relationship based on the received signal reference power value in the user equipment measurement report, and generate the neighbor cell coverage overlap.

[0015] The number of connections, the power consumption value, and the neighboring cell coverage overlap input channel splicing layer are used to perform tensor splicing calculations to generate a feature matrix, and the feature matrix is ​​used as a state feature map.

[0016] Preferably, in the traffic prediction module, generating the spatiotemporal load prediction matrix within the time window specifically includes the following steps:

[0017] Multiple state feature maps collected within the acquisition period are spliced ​​together in chronological order to construct a state sequence;

[0018] The state sequence is input into the long short-term memory network, and hidden layer state updates and matrix multiplication operations are performed through the forget gate, input gate and output gate inside the long short-term memory network to output a spatiotemporal feature vector.

[0019] The spatiotemporal feature vectors are input into a fully connected network layer for linear mapping calculation to generate a spatiotemporal load prediction matrix within a time window.

[0020] Preferably, in the graph feature aggregation module, generating the coupled feature matrix specifically includes the following steps:

[0021] The spatiotemporal load prediction matrix is ​​analyzed, each 5G base station is mapped to a graph node of a graph network structure, and the predicted load data corresponding to each 5G base station in the spatiotemporal load prediction matrix is ​​extracted as the feature vector of each graph node to generate a graph node feature matrix.

[0022] Based on the adjacency relationship between the 5G base stations, establish the connection edges between the graph nodes, calculate the edge weight of each connection edge, and generate a graph adjacency matrix;

[0023] The graph node feature matrix and the graph adjacency matrix are input into the graph convolutional layer of the graph convolutional network for matrix multiplication and feature aggregation operations to extract the correlation data between the 5G base stations.

[0024] The associated data is subjected to an activation function operation, and the coupling feature matrix is ​​output.

[0025] Preferably, in the collaborative decision-making module, generating an initial strategy action set containing the RF resource reconfiguration actions of each of the 5G base stations within the collaborative area specifically includes the following steps:

[0026] The coupled feature matrix is ​​used as the input to the forward propagation calculation. Weight matrix multiplication and nonlinear activation operations are performed in the policy network of the deep reinforcement learning model to generate hidden layer feature vectors.

[0027] The hidden layer feature vector is input into the Softmax function layer in the policy network for normalization mapping calculation to generate action probability distribution data corresponding to each of the 5G base stations in the cooperative area;

[0028] Based on the action probability distribution data, the radio frequency resource reconfiguration action corresponding to each 5G base station is determined using the probability maximum extraction rule, and the determined radio frequency resource reconfiguration action is encapsulated into an initial strategy action set.

[0029] Preferably, the step of determining the radio frequency resource reconfiguration action corresponding to each of the 5G base stations using the probability maximum extraction rule, and encapsulating each determined radio frequency resource reconfiguration action into the initial strategy action set specifically includes the following steps:

[0030] Based on the probability maximum extraction rule, the numerical traversal comparison calculation is performed on the action probability vector corresponding to each 5G base station in the action probability distribution data to extract the maximum probability value in each action probability vector and the action sequence index corresponding to the maximum probability value.

[0031] Based on the action sequence index, query the action space mapping table, extract the action identifier code that matches the action sequence index, and use the action identifier code as the radio frequency resource reconfiguration action corresponding to each of the 5G base stations;

[0032] Collect the radio frequency resource reconfiguration actions corresponding to each 5G base station in the collaborative area, and serialize and splice each radio frequency resource reconfiguration action according to the topology node number of each 5G base station to generate an initial policy action set.

[0033] Preferably, in the constraint filtering module, the generation of control instructions specifically includes the following steps:

[0034] Parse the initial policy action set and extract the radio frequency resource reconfiguration action corresponding to each 5G base station;

[0035] The radio frequency resource reconfiguration actions are input into the network simulation model for calculation, and the predicted load value and predicted signal coverage area of ​​each 5G base station are output. The predicted load value and the predicted signal coverage area are then used to construct the predicted network parameters of each 5G base station.

[0036] The network constraint threshold is analyzed to extract the upper limit value of the capacity corresponding to the predicted load value and the lower limit value of the coverage area corresponding to the predicted signal coverage area.

[0037] The predicted load value is compared with the upper limit of the capacity value as a first value, and the predicted signal coverage area is compared with the lower limit of the coverage area value as a second value;

[0038] The radio frequency resource reconfiguration actions where the predicted load value is greater than the upper limit of the capacity value are extracted as the first rejection actions, and the radio frequency resource reconfiguration actions where the predicted signal coverage area is less than the lower limit of the coverage area value are extracted as the second rejection actions.

[0039] The first action to be removed and the second action to be removed are removed from the initial policy action set. The remaining policy action set after removing the first action to be removed and the second action to be removed is extracted and the remaining policy action set is compiled into control instructions.

[0040] Preferably, in the execution feedback module, the step of sending the control command to the baseband processing unit and radio frequency unit of each 5G base station based on the control command, controlling each 5G base station to perform the radio frequency resource reconfiguration action, and collecting the network status feedback value after performing the radio frequency resource reconfiguration action and sending it back to the feature perception module, specifically includes the following steps:

[0041] The control command is parsed, the configuration parameters corresponding to each 5G base station are extracted, and the configuration parameters are encapsulated into a driver message.

[0042] The driver message is sent to the baseband processing unit and the radio frequency unit of each 5G base station through the interface protocol, and the control interface of each 5G base station is called to modify the register value of each 5G base station and execute the radio frequency resource reconfiguration action.

[0043] Read the status registers and counters of each 5G base station, obtain the power consumption value and number of connections after the radio frequency resource reconfiguration action is performed, and construct the power consumption value and the number of connections into a network status feedback value;

[0044] The network status feedback value is transmitted to the feature perception module through the communication interface.

[0045] The present invention has the following beneficial effects:

[0046] 1. In this invention, the coverage overlap between neighboring base stations is extracted by graph convolutional network and combined with deep reinforcement learning model for collaborative decision-making. When formulating adjustment strategy, the mutual offsetting effect between the energy saving of target base station dormancy and the power compensation cost of adjacent base stations is calculated. This reduces the ineffective collaboration caused by the surge in energy consumption of surrounding base stations due to local single-station dormancy, and improves the overall energy efficiency of the collaborative area.

[0047] 2. In this invention, an action filtering mechanism based on network simulation model and constraint threshold comparison is introduced. Before issuing control commands, the predicted load and predicted signal coverage area of ​​each base station after undertaking the compensation coverage task are verified. Adjustment actions that may cause the capacity of surrounding compensation base stations to be overloaded or produce local coverage blind spots are replaced. While controlling global energy consumption, the safety and stability of basic communication experience are ensured.

[0048] 3. In this invention, a long short-term memory network is used to capture the spatiotemporal load evolution pattern and construct a closed-loop link for physical state feedback. This enables the system to predict the spatial shift trend of traffic caused by base station state adjustment, and to obtain feedback data on actual power consumption and load changes after the action is executed. Based on the dynamically changing network environment, the system continuously corrects the energy consumption coordination strategy among multiple base stations. Attached Figure Description

[0049] Figure 1 This is an architecture diagram of the 5G base station energy consumption dynamic optimization and adjustment system based on deep reinforcement learning proposed in this invention. Detailed Implementation

[0050] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] This invention provides a 5G base station energy consumption dynamic optimization and adjustment system based on deep reinforcement learning, such as... Figure 1 As shown, it includes:

[0052] The feature perception module is used to collect the operation indicators and wireless environment logs of each 5G base station in the collaborative area, perform spatiotemporal alignment and normalization processing on the operation indicators and wireless environment logs, and generate a state feature map containing the number of connections, power consumption value and coverage overlap of each 5G base station.

[0053] Furthermore, in the feature perception module, generating a state feature map containing the connection count, power consumption value, and coverage overlap of each 5G base station specifically includes the following steps:

[0054] Analyze the operational metrics after spatiotemporal alignment and normalization, and extract the connection count and power consumption of each 5G base station;

[0055] Read the user equipment measurement report from the wireless environment log after spatiotemporal alignment and normalization, calculate the signal overlap ratio between adjacent 5G base stations based on the received signal reference power value in the user equipment measurement report, and generate the neighbor cell coverage overlap.

[0056] The number of connections, power consumption, and neighboring cell coverage overlap are input to the channel splicing layer for tensor splicing calculation to generate a feature matrix, which is then used as a state feature map.

[0057] Specifically, the feature perception module first continuously collects operational metrics and wireless environment logs from multiple 5G base stations within the collaborative area. Due to differences in data reporting cycles among different devices, the system performs spatiotemporal alignment processing on the data at a fixed time step, using a linear interpolation algorithm to uniformly map asynchronously arriving operational metrics and wireless environment logs to the same time and corresponding base station spatial nodes. Subsequently, operational data with large numerical fluctuations is processed, the spatiotemporally aligned operational metrics are analyzed, and the first... 5G base stations Initial number of connections at time 1 Compared with the initial power consumption value The system uses a maximum-minimum algorithm for normalization calculation. The normalization formula for the number of connections is as follows: The normalized formula for power consumption is: . in the formula and These represent the normalized connection count and power consumption values, respectively. and These represent the maximum and minimum number of connections historically recorded for this base station, respectively. and These represent the maximum and minimum power consumption values ​​recorded in historical statistics, respectively.

[0058] Next, the system reads the user equipment measurement report from the wireless environment log after spatiotemporal alignment and normalization. The system extracts the reference power values ​​of the received signals from the user equipment to the serving base station and all neighboring base stations recorded in the report. These values ​​are used to calculate the signal overlap ratio among adjacent 5G base stations, thereby generating the neighboring cell coverage overlap. Let the... Each 5G base station serves as a service base station, and the total number of user devices under this service base station is: And the service base station has a total of The number of adjacent base stations. For the first serving base station... Each user equipment, its measurements from the serving base station The received signal reference power value is denoted as , from the The received signal reference power value of each adjacent base station is denoted as Define the signal overlap determination function. ,when and The system determines that the user equipment is located within the overlapping coverage area of ​​the serving base station and the adjacent base station. The value is 1 otherwise, 0. The set effective signal coverage strength threshold. Serving base station. With the The formula for calculating the coverage overlap between neighboring base stations is as follows: Based on this, the overlap of all adjacent base stations is further averaged to calculate the base station... The average neighboring cell coverage overlap is calculated using the following formula: .

[0059] After extracting and calculating various feature data, the system inputs the obtained connection count, power consumption value, and neighboring cell coverage overlap into the channel stitching layer of the deep learning framework for tensor stitching calculation. Assume the cooperative region contains a total of... For each 5G base station, the system constructs a first feature channel matrix using the normalized connection count of all base stations, a second feature channel matrix using the normalized power consumption value, and a third feature channel matrix using the average neighboring cell coverage overlap of each base station. In the channel stitching layer, following the data input format requirements of the deep learning network, these three feature matrices are stitched and fused along the channel dimensions to generate a matrix with a dimension of [missing information]. The system uses the fused feature matrix as the state feature map representing the overall network operating environment at the current time step.

[0060] Through rigorous mathematical mapping and tensor concatenation, the underlying operational data of multi-source heterogeneous base stations is transformed into a structurally standardized state feature matrix. The output state feature map is directly used as the basic data frame for continuous time series data and input into the traffic prediction module of the next stage, allowing the Long Short-Term Memory network to read and extract spatiotemporal features.

[0061] The traffic prediction module, based on the state feature map, uses a long short-term memory network to extract the spatiotemporal feature vectors from the state feature map and generate a spatiotemporal load prediction matrix within the time window.

[0062] Furthermore, in the traffic prediction module, generating the spatiotemporal load prediction matrix within the time window specifically includes the following steps:

[0063] Multiple state feature maps collected within the acquisition period are spliced ​​together in chronological order to construct a state sequence;

[0064] The state sequence is input into the Long Short-Term Memory (LSTM) network. The hidden state is updated and matrix multiplication is performed through the forget gate, input gate and output gate inside the LSM network, and the spatiotemporal feature vector is output.

[0065] The spatiotemporal feature vectors are input into the fully connected network layer for linear mapping calculation to generate the spatiotemporal load prediction matrix within the time window.

[0066] Specifically, the traffic prediction module first receives the dimension output by the feature perception module, which is... The state feature diagram, in which This indicates the number of 5G base stations within the collaborative area. The system sets up a range containing... The acquisition cycle consists of several consecutive time steps. Multiple state feature maps generated sequentially within this acquisition cycle at fixed time steps are then stitched together in chronological order. Let the th... The state feature map of the input at each time step is as follows: To adapt to the input format requirements of Long Short-Term Memory (LSTM) networks, the system will use a matrix... Flattened to a length of one-dimensional input vector The system then serializes and stacks these input vectors in the time dimension to construct a state sequence that represents the historical temporal evolution characteristics of each 5G base station in the collaborative area.

[0067] The constructed state sequence is sequentially input into the Long Short-Term Memory (LSTM) network. The system first initializes the initial hidden state vector of the LTM network. With the initial cell state vector This is a vector whose value is zero. At each time step of the Long Short-Term Memory network... The system receives the input vector at the current moment. Compared with the hidden state vector of the previous time step Vector concatenation is performed. Internally, the network uses a forget gate to calculate and decide which historical feature information to discard. The formula for calculating the forget gate output vector is as follows: Next, the input gate filters out the new features that need to be retained. The formula for calculating the input gate output vector is as follows: Simultaneously, candidate cell state vectors are generated. Based on the outputs of the forget gate and the input gate, the system combines the cell state vector from the previous time step. The current cell state vector is updated using the following formula: Finally, the output feature of the current hidden state vector is controlled by the output gate. The formula for calculating the output vector of the output gate is as follows: And update the hidden state vector at the current time step. In the above formula This represents the Sigmoid non-linear activation function. This represents the hyperbolic tangent activation function. , , , These represent the weight parameter matrices for the corresponding gating units. , , , These represent the corresponding bias parameter vectors. This represents matrix multiplication. This represents the Hadamard product operation, which multiplies corresponding elements. After a complete loop iteration at each time step, the Long Short-Term Memory (LSTM) network outputs the hidden state vector of the last time step. The system uses the hidden state vector as the extracted spatiotemporal feature vector containing the spatiotemporal evolution law.

[0068] After extracting the spatiotemporal feature vectors, the system inputs them into a fully connected network layer for linear mapping calculation. The fully connected network layer transforms the high-dimensional spatiotemporal feature vectors, mapping them to predicted values ​​within the future target time window. The formula for linear mapping is as follows: , in the formula This represents a one-dimensional vector output by a linear mapping. This represents the weight matrix of the fully connected layer. This represents the bias vector of the fully connected layer. To correspond to the spatial topology, the system will compute the one-dimensional vector output. Reshaping to a dimension of The matrix, where This represents the number of load metrics predicted for a single base station. The reshaped matrix is ​​the spatiotemporal load prediction matrix generated within the time window, which contains the estimated load data of each 5G base station in the cooperative area for future time periods.

[0069] By introducing a gating mechanism, the time-series network accurately captures the time-dependent relationships of the underlying operational features of the base station, transforming the basic state input from the feature perception module into time-forward-looking estimated load data, and seamlessly connecting to the topology node prediction input requirements of the next-stage graph feature aggregation module.

[0070] The graph feature aggregation module, based on the spatiotemporal load prediction matrix, uses a graph convolutional network to perform topology node mapping and edge weight calculation on the spatiotemporal load prediction matrix, extracts the correlation data between each 5G base station, and generates a coupled feature matrix.

[0071] Furthermore, in the graph feature aggregation module, generating the coupled feature matrix specifically includes the following steps:

[0072] The spatiotemporal load prediction matrix is ​​analyzed, each 5G base station is mapped to a graph node in a graph network structure, and the predicted load data corresponding to each 5G base station in the spatiotemporal load prediction matrix is ​​extracted as the feature vector of each graph node to generate a graph node feature matrix.

[0073] Based on the adjacency relationships between each 5G base station, establish the connection edges between each graph node, calculate the edge weight of each connection edge, and generate a graph adjacency matrix;

[0074] The graph node feature matrix and graph adjacency matrix are input into the graph convolutional layer of the graph convolutional network for matrix multiplication and feature aggregation operations to extract the correlation data between each 5G base station.

[0075] Activation function operations are performed on the associated data to output the coupling feature matrix.

[0076] Specifically, the graph feature aggregation module first receives the output of the traffic prediction module, which has a dimension of [missing value]. The spatiotemporal load prediction matrix, where The number of 5G base stations in the coordinated area. This represents the number of load indicators predicted for a single base station. The system analyzes this spatiotemporal load prediction matrix and integrates the load indicators within the cooperative area. Each 5G base station is mapped to a set of graph nodes in a graph network structure. For any graph node in this set... The system will use the corresponding number in the spatiotemporal load prediction matrix. The length of each 5G base station is The predicted load data is extracted as the initial feature vector for the graph node. The system arranges the initial feature vectors of all graph nodes row by row according to the topology node number, generating a dimension of [missing information]. Graph node feature matrix This matrix directly carries the communication connection and resource demand prediction attributes of each base station for future time windows.

[0077] The system then establishes connection edges between graph nodes based on the physical location and signal adjacency of each 5G base station, and calculates the edge weight of each connection edge. The system uses the neighboring cell coverage overlap calculated in the feature perception module as a quantitative indicator of the spatial coupling strength between base stations. Let the base station... With base station If adjacency exists, the system establishes a graph node. With graph nodes Connect the edges between them and cover the overlap of their neighboring regions. The edge weight of the connecting edge is directly assigned. If two base stations are not adjacent, the edge weight is assigned a value of zero, thus constructing a dimension of [missing value]. The original adjacency matrix To preserve the base station's own predicted feature information during feature aggregation, the system adds self-loops to the original adjacency matrix, introducing a dimension of [missing information]. identity matrix Generate a self-loop adjacency matrix Next, the system calculates the degree matrix of the self-loop adjacency matrix. The degree matrix is ​​a diagonal matrix, and the formula for calculating the elements on its main diagonal is: The system uses the degree matrix to perform symmetric normalization on the self-loop adjacency matrix to generate a graph adjacency matrix. . in the formula Degree matrix The inverse square root matrix has its main diagonal elements being the negative 1 / 2 power of the corresponding elements on the main diagonal of the degree matrix. The system uses this to eliminate the numerical suppression error caused by core base station nodes with large degrees to surrounding nodes during feature aggregation.

[0078] After the parameter matrix is ​​constructed, the system will generate the graph node feature matrix. Adjacency matrix of graph The graph convolutional layer of the input graph convolutional network performs matrix multiplication and feature aggregation operations to extract the correlation data between 5G base stations. The graph convolutional layer first aggregates the predicted load features of adjacent base stations with overlapping coverage to the target base station node according to edge weights through multiplication of the graph adjacency matrix and the graph node feature matrix. Then, it multiplies by the trainable weight matrix inside the graph convolutional layer to perform feature space transformation. The formula for calculating the correlation data is as follows: , in the formula This represents the associated data matrix output after aggregation calculation. This represents matrix multiplication. This represents the feature map weight matrix of the graph convolutional layer. The dimension size is ,in This sets the output feature dimension values ​​for the graph convolutional layer. After a linear transformation of this weight matrix, the associated data matrix... The dimension is rigorously mapped as .

[0079] After extracting the associated data, the system processes the associated data matrix. The elements in the matrix are activated using activation functions to introduce nonlinear features, resulting in a coupled feature matrix. The system uses a modified linear unit activation function for calculation, and the activation calculation formula is as follows: . in the formula Representation of associated data matrix A matrix of all zeros with exactly the same dimensions. This operation compares the values ​​at corresponding positions in two matrices element by element and takes the maximum value. This represents the coupled feature matrix output by the activation function, whose dimension remains constant. .

[0080] This step introduces a graph network algorithm to perform deep computational fusion of isolated future load prediction data from base stations with the spatial adjacency coverage structure of the existing network. The generated coupling feature matrix not only includes the base station's own temporal load trend, but also accurately quantifies the spatial interference and traffic diversion coordination effects of surrounding base stations through spatial mapping of feature dimensions. This provides a high-dimensional graph topology input foundation for the subsequent collaborative decision-making module to evaluate the global chain reaction of radio frequency resource reconfiguration actions.

[0081] The collaborative decision-making module, based on the coupled feature matrix, inputs the coupled feature matrix into a deep reinforcement learning model with a centralized training distributed execution architecture for forward propagation calculation, generating an initial policy action set containing the reconfiguration actions of radio frequency resources of each 5G base station in the collaborative area;

[0082] Furthermore, in the collaborative decision-making module, generating the initial policy action set containing the RF resource reconfiguration actions of each 5G base station within the collaborative area specifically includes the following steps:

[0083] The coupled feature matrix is ​​used as the input for forward propagation computation. Weight matrix multiplication and nonlinear activation operations are performed in the policy network of the deep reinforcement learning model to generate hidden layer feature vectors.

[0084] The hidden feature vectors are input into the Softmax function layer in the policy network for normalization mapping calculation, generating action probability distribution data corresponding to each 5G base station in the cooperative area;

[0085] Based on the action probability distribution data, the probability maximum extraction rule is used to determine the radio frequency resource reconfiguration action corresponding to each 5G base station, and the determined radio frequency resource reconfiguration action is encapsulated into an initial policy action set.

[0086] Furthermore, the radio frequency resource reconfiguration actions corresponding to each 5G base station are determined using the probability maximum extraction rule, and the determined radio frequency resource reconfiguration actions are encapsulated into an initial policy action set, specifically including the following steps:

[0087] Based on the probability maximum extraction rule, the numerical traversal comparison calculation is performed on the action probability vectors corresponding to each 5G base station in the action probability distribution data to extract the maximum probability value in each action probability vector and the action sequence index corresponding to the maximum probability value.

[0088] Based on the action sequence index, query the action space mapping table, extract the action identifier code that matches the action sequence index, and use the action identifier code as the radio frequency resource reconfiguration action corresponding to each 5G base station.

[0089] Collect the radio frequency resource reconfiguration actions corresponding to each 5G base station in the collaborative area, and serialize and splice each radio frequency resource reconfiguration action according to the topology node number of each 5G base station to generate an initial policy action set.

[0090] Specifically, the collaborative decision-making module first receives the dimensionality output by the graph feature aggregation module. The coupling characteristic matrix, where The number of 5G base stations in the coordinated area. This represents the feature dimension value output by the graph network. The system uses this coupled feature matrix as the forward propagation input for the deep reinforcement learning model under a centralized training distributed execution architecture. In the policy network of the deep reinforcement learning model, the system multiplies the coupled feature matrix with the first weight matrix inside the policy network, adds the first bias matrix, and then performs a nonlinear activation operation. This matrix multiplication operation is physically and logically equivalent to multiple 5G base stations within a collaborative region sharing the weight parameters of the same policy network and performing parallel distributed action inference based on their respective feature row vectors to generate hidden layer feature vectors. The specific calculation formula for the hidden layer feature vectors is as follows: . in the formula Indicates the dimension size as The hidden layer feature matrix, which is composed of The hidden layer feature vectors corresponding to each 5G base station are arranged in rows. The number of hidden layer nodes set for the policy network. This represents the input coupling feature matrix. Indicates the dimension size as The first weight matrix, It indicates that the length is The one-dimensional bias vector is copied along the base station dimension. The first bias matrix generated by the stacking. This represents matrix multiplication. This represents the modified linear unit activation function.

[0091] The system then inputs the hidden layer feature matrix into the Softmax function layer of the policy network for normalization mapping calculation, generating action probability distribution data corresponding to each 5G base station within the cooperative area. The system first maps the hidden layer feature matrix to the action space dimension through a linear transformation, calculated using the following formula: . in the formula Indicates the dimension size as The action state matrix, The total number of radio frequency resource reconfiguration actions set for the collaborative region. Indicates the dimension size as The second weight matrix, It indicates that the length is The one-dimensional bias vector is copied along the base station dimension. The second bias matrix is ​​generated by the stacking process. Next, the system uses a Softmax function layer to perform row-wise normalized mapping calculations on the action state matrix. Let the index of the base station within the cooperative area be... ,in The value range is 1 to The index of the action sequence is ,in The value range is 1 to For the first The first 5G base station corresponding to the The probability formula for selecting an action is as follows: . in the formula Indicates the first The first 5G base station performs the first The probability value of each action. and These represent the first and second positions in the action state matrix, respectively. Line 1 Column and the Line 1 The numerical values ​​of the column elements. To sum the variables, This represents exponential operations with the natural constant as the base. The action probability distribution data output by this calculation consists of all... The elements constitute a quantitative characterization of the tendency of each base station to implement various radio frequency adjustment strategies under the current network environment.

[0092] Based on action probability distribution data, the system uses probability maximum extraction rules to determine the radio frequency resource reconfiguration actions corresponding to each 5G base station. The system assigns a length of [length missing] to each 5G base station in the action probability distribution data. The action probability vectors are numerically traversed and compared to calculate the maximum probability value in each action probability vector, as well as the index of the action sequence containing that maximum probability value. The formula for calculating the action sequence index corresponding to each 5G base station is: . in the formula Indicates the variable being set Value range 1 to Find the index of the action sequence that maximizes the subsequent probability value. The system uses this formula to accurately extract the action sequence index with the highest selection probability for each base station. .

[0093] The system queries a pre-set action space mapping table based on the extracted action sequence index. This mapping table records the mapping relationship between action sequence indexes and actual physical control commands. The system extracts action identifier codes that strictly match the action sequence indexes, such as specific antenna channel sleep modes or RF transmit power adjustment levels, and uses these action identifier codes as the RF resource reconfiguration actions corresponding to each 5G base station. Finally, the system collects all data within the collaborative area. For each 5G base station, the radio frequency resource reconfiguration action is serialized and spliced ​​according to the topology node number order of each 5G base station to generate a data set containing global network adjustment and control requirements, and this data set is encapsulated into an initial policy action set.

[0094] By transforming the high-dimensional topological coupling characteristics into a probability distribution of specific executable radio frequency control strategies, and establishing the optimal action combination for each base station through extreme value optimization rules, a pre-selected scheme for global network energy consumption optimization to be verified is provided for the next stage constraint filtering module.

[0095] The constraint filtering module calculates the predicted network parameters of each 5G base station based on the initial policy action set, compares the predicted network parameters with the network constraint threshold, eliminates radio frequency resource reconfiguration actions that do not meet the network constraint threshold, and generates control commands.

[0096] Furthermore, in the constraint filtering module, generating control commands specifically includes the following steps:

[0097] Parse the initial policy action set and extract the radio frequency resource reconfiguration actions corresponding to each 5G base station;

[0098] The reconfiguration of each radio frequency resource is input into the network simulation model for calculation, and the predicted load value and predicted signal coverage area of ​​each 5G base station are output. The predicted load value and predicted signal coverage area are then used to construct the predicted network parameters of each 5G base station.

[0099] The network constraint threshold is analyzed to extract the upper limit of capacity corresponding to the predicted load value and the lower limit of coverage area corresponding to the predicted signal coverage area.

[0100] The predicted load value is compared with the upper limit of the capacity value as the first comparison, and the predicted signal coverage area is compared with the lower limit of the coverage area as the second comparison;

[0101] Radio resource reconfiguration actions with predicted load values ​​greater than the capacity limit are extracted as the first to be eliminated actions, and radio resource reconfiguration actions with predicted signal coverage areas less than the coverage area lower limit are extracted as the second to be eliminated actions.

[0102] Remove the first and second actions to be removed from the initial policy action set, extract the remaining policy action set after removing the first and second actions, and compile the remaining policy action set into control instructions.

[0103] Specifically, the constraint filtering module first receives the initial policy action set output by the collaborative decision-making module. This initial policy action set contains the radio frequency resource reconfiguration actions of multiple 5G base stations arranged in topology node number order within the collaborative area. The system performs serialization parsing operations on the initial policy action set to extract the first... Radio frequency resource reconfiguration for each 5G base station ,in The value ranges from 1 to the total number of 5G base stations in the collaborative area. .

[0104] The system inputs the extracted RF resource reconfiguration actions into a preset network simulation model for physical parameter mapping calculations. The system first parses the RF resource reconfiguration actions. The corresponding underlying physical configuration item is directly converted into a capacity adjustment ratio coefficient by querying a preset physical parameter mapping table. Transmit power adjustment ratio coefficient Simultaneously, the system calls the spatiotemporal load prediction matrix output by the traffic prediction module to extract the first... The feature values ​​in the row vector corresponding to each 5G base station are used as the benchmark for predicting the load. The formula for calculating the predicted load value is as follows: . in the formula Indicates the first Predicted load values ​​for 5G base stations This represents the capacity adjustment ratio of the action mapping. Indicates the baseline predicted load, This indicates a scalar multiplication operation. Further, the system reads the static engineering parameter files stored locally on each 5G base station and extracts the initial reference signal coverage area of ​​that base station under the current physical settings such as antenna height and downtilt angle. The formula for calculating the predicted signal coverage area is as follows: . in the formula Indicates the first The predicted signal coverage area of ​​a 5G base station This represents the transmit power adjustment ratio for the action mapping. This represents the initial reference signal coverage area. The system will calculate the output predicted load value. With the predicted signal coverage area Together, they form the predicted network parameters for each 5G base station.

[0105] The system then parses the preset network constraint rule file, targeting the first... For each 5G base station, extract the maximum capacity value corresponding to its equipment hardware capabilities. And the minimum coverage area corresponding to the basic communication guarantee requirements. The system will use its predicted load values. With the maximum capacity value The first numerical comparison is performed to determine whether it meets the network security capacity requirements, and the predicted signal coverage area is also determined. Compared with the lower limit of coverage area A second numerical comparison is performed to determine whether it meets the requirements for blind-spot-free communication coverage.

[0106] Based on the numerical comparison results, the system extracts those that meet the judgment criteria. RF resource reconfiguration As the first action to be eliminated, extract those that meet the judgment criteria. RF resource reconfiguration As the second action to be removed, the system directly deletes both the first and second actions from the initial policy action set. To ensure a strict correspondence between the action sequence dimension and the base station topology nodes, the system replaces the action identifier code at the corresponding position with a safety baseline action code representing maintaining the current operating state. The system extracts the remaining policy action set after performing the deletion and replacement operations and compiles this remaining policy action set into machine-readable control instructions according to the communication protocol specifications of the base station's underlying equipment.

[0107] By introducing a deterministic physical parameter mapping mechanism and dual hard threshold comparison, the model-generated action sequences were accurately eliminated for violations and baseline data was replaced, ensuring the safety and reliability of the control commands output by the compiler in the live network environment.

[0108] The execution feedback module, based on control commands, sends control commands to the baseband processing unit and radio frequency unit of each 5G base station, controls each 5G base station to perform radio frequency resource reconfiguration, and collects network status feedback values ​​after performing radio frequency resource reconfiguration and sends them back to the feature perception module.

[0109] Furthermore, in the execution feedback module, based on control commands, control commands are sent to the baseband processing unit and radio frequency unit of each 5G base station to control each 5G base station to perform radio frequency resource reconfiguration, and the network status feedback value after the radio frequency resource reconfiguration is collected and sent back to the feature perception module. Specifically, this includes the following steps:

[0110] Parse control commands, extract configuration parameters corresponding to each 5G base station, and encapsulate the configuration parameters into driver messages;

[0111] The driver message is sent to the baseband processing unit and radio frequency unit of each 5G base station through the interface protocol, and the control interface of each 5G base station is called to modify the register values ​​of each 5G base station and perform radio frequency resource reconfiguration.

[0112] Read the status registers and counters of each 5G base station, obtain the power consumption value and number of connections after the RF resource reconfiguration action, and construct the power consumption value and number of connections into a network status feedback value;

[0113] The network status feedback value is transmitted to the feature perception module through the communication interface.

[0114] Specifically, the execution feedback module first receives machine-readable control commands compiled and issued by the constraint filtering module. The system then performs parsing operations on these control commands at the base station node level, extracting the first... The underlying configuration parameters corresponding to each 5G base station, among which The value ranges from 1 to the total number of 5G base stations in the collaborative area. This configuration parameter specifically includes the target transmit power attenuation and the antenna channel sleep status indicator. After extracting the configuration parameters, the system encapsulates them into a standard format driver message according to the base station equipment management protocol specification.

[0115] The system sends the encapsulated driver message to the first... through the operation, maintenance, and management communication interface. The baseband processing unit and radio frequency unit of a 5G base station. After receiving the driver message, the underlying agent program of the base station equipment calls the system control interface of the baseband processing unit and radio frequency unit, and directly writes the values ​​in the configuration parameters into the corresponding hardware physical registers. By modifying the values ​​of the transmit power control register and the channel enable status register, the system physically triggers and controls each 5G base station to perform the corresponding radio frequency resource reconfiguration action.

[0116] After each 5G base station completes its radio frequency resource reconfiguration and a preset network status stabilization delay has elapsed, the system reads the hardware status registers and baseband performance counters of each 5G base station via a polling mechanism. The system then obtains the [data from these registers]. The actual operational status data of each 5G base station at the new time step is used to extract the actual number of connections after the execution of actions. Compared with actual power consumption value The system will construct the obtained values ​​into the first... The network state feedback vector of a 5G base station is constructed using the following formula: . in the formula Indicates the first The state feedback vector corresponding to each 5G base station and These represent the extracted actual number of connections and the actual power consumption, respectively. Square brackets indicate a mathematical construct that combines the contained scalar values ​​into a one-dimensional row vector, and commas indicate the physical separation of elements within the vector. The system will coordinate all within the region... The network state feedback vectors of 5G base stations are stacked row by row according to the topology node sequence, generating a dimension of [missing value]. The global network state feedback matrix.

[0117] Finally, the system transmits the global network status feedback matrix to the feature perception module via the internal data communication bus. Upon receiving the matrix, the feature perception module calculates the actual connection count... Compared with actual power consumption value It is strictly mapped to the initial number of connections and the initial power consumption value of the next time step, and is used to combine the wireless environment logs collected in the new time step to start a new round of data normalization processing and state feature map generation calculation.

[0118] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A 5G base station energy consumption dynamic optimization adjustment system based on deep reinforcement learning, characterized in that, include: The feature perception module is used to collect the operation indicators and wireless environment logs of each 5G base station in the collaborative area, perform spatiotemporal alignment and normalization processing on the operation indicators and wireless environment logs, and generate a state feature map containing the connection number, power consumption value and neighboring cell coverage overlap of each 5G base station. The traffic prediction module, based on the state feature map, uses a long short-term memory network to extract the spatiotemporal feature vectors from the state feature map and generates a spatiotemporal load prediction matrix within a time window. The graph feature aggregation module, based on the spatiotemporal load prediction matrix, uses a graph convolutional network to perform topology node mapping and edge weight calculation on the spatiotemporal load prediction matrix, extracts the correlation data between each of the 5G base stations, and generates a coupled feature matrix. The collaborative decision-making module, based on the coupling feature matrix, inputs the coupling feature matrix into a deep reinforcement learning model with a centralized training distributed execution architecture for forward propagation calculation, and generates an initial policy action set containing the reconfiguration actions of radio frequency resources of each 5G base station in the collaborative area; The constraint filtering module calculates the predicted network parameters of each 5G base station based on the initial policy action set, compares the predicted network parameters with the network constraint threshold, eliminates radio frequency resource reconfiguration actions that do not meet the network constraint threshold, and generates control commands. The execution feedback module, based on the control command, sends the control command to the baseband processing unit and radio frequency unit of each of the 5G base stations, controls each of the 5G base stations to perform the radio frequency resource reconfiguration action, and collects the network status feedback value after the radio frequency resource reconfiguration action and sends it back to the feature perception module. 2.The 5G base station energy consumption dynamic optimization adjustment system based on deep reinforcement learning of claim 1, wherein, In the feature perception module, generating a state feature map containing the connection count, power consumption value, and neighboring cell coverage overlap of each 5G base station specifically includes the following steps: The operational metrics after spatiotemporal alignment and normalization are analyzed to extract the connection count and power consumption of each 5G base station. Read the user equipment measurement report in the wireless environment log after the spatiotemporal alignment and normalization processing, calculate the signal overlap ratio between each of the 5G base stations that have an adjacent relationship based on the received signal reference power value in the user equipment measurement report, and generate the neighbor cell coverage overlap. The number of connections, the power consumption value, and the neighboring cell coverage overlap input channel splicing layer are used to perform tensor splicing calculations to generate a feature matrix, and the feature matrix is ​​used as a state feature map. 3.The deep reinforcement learning based 5G base station energy consumption dynamic optimization adjustment system of claim 1, wherein, In the traffic prediction module, generating the spatiotemporal load prediction matrix within the time window specifically includes the following steps: Multiple state feature maps collected within the acquisition period are spliced ​​together in chronological order to construct a state sequence; The state sequence is input into the long short-term memory network, and hidden layer state updates and matrix multiplication operations are performed through the forget gate, input gate and output gate inside the long short-term memory network to output a spatiotemporal feature vector. The spatiotemporal feature vectors are input into a fully connected network layer for linear mapping calculation to generate a spatiotemporal load prediction matrix within a time window. 4.The 5G base station energy consumption dynamic optimization adjustment system based on deep reinforcement learning of claim 1, wherein, In the graph feature aggregation module, generating the coupled feature matrix specifically includes the following steps: The spatiotemporal load prediction matrix is ​​analyzed, each 5G base station is mapped to a graph node of a graph network structure, and the predicted load data corresponding to each 5G base station in the spatiotemporal load prediction matrix is ​​extracted as the feature vector of each graph node to generate a graph node feature matrix. Based on the adjacency relationship between the 5G base stations, establish the connection edges between the graph nodes, calculate the edge weight of each connection edge, and generate a graph adjacency matrix; The graph node feature matrix and the graph adjacency matrix are input into the graph convolutional layer of the graph convolutional network for matrix multiplication and feature aggregation operations to extract the correlation data between the 5G base stations. The associated data is subjected to an activation function operation, and the coupling feature matrix is ​​output.

5. The deep reinforcement learning based 5G base station energy consumption dynamic optimization adjustment system of claim 1, wherein, In the collaborative decision-making module, generating an initial policy action set containing the 5G base station radio frequency resource reconfiguration actions within the collaborative area specifically includes the following steps: The coupled feature matrix is ​​used as the input to the forward propagation calculation. Weight matrix multiplication and nonlinear activation operations are performed in the policy network of the deep reinforcement learning model to generate hidden layer feature vectors. The hidden layer feature vector is input into the Softmax function layer in the policy network for normalization mapping calculation to generate action probability distribution data corresponding to each of the 5G base stations in the cooperative area; Based on the action probability distribution data, the radio frequency resource reconfiguration action corresponding to each 5G base station is determined using the probability maximum extraction rule, and the determined radio frequency resource reconfiguration action is encapsulated into an initial strategy action set.

6. The 5G base station energy consumption dynamic optimization and adjustment system based on deep reinforcement learning according to claim 5, characterized in that, The step of determining the radio frequency resource reconfiguration action corresponding to each 5G base station using the probability maximum extraction rule, and encapsulating each determined radio frequency resource reconfiguration action into the initial policy action set, specifically includes the following steps: Based on the probability maximum extraction rule, the numerical traversal comparison calculation is performed on the action probability vector corresponding to each 5G base station in the action probability distribution data to extract the maximum probability value in each action probability vector and the action sequence index corresponding to the maximum probability value. Based on the action sequence index, query the action space mapping table, extract the action identifier code that matches the action sequence index, and use the action identifier code as the radio frequency resource reconfiguration action corresponding to each of the 5G base stations; Collect the radio frequency resource reconfiguration actions corresponding to each 5G base station in the collaborative area, and serialize and splice each radio frequency resource reconfiguration action according to the topology node number of each 5G base station to generate an initial policy action set.

7. The deep reinforcement learning based 5G base station energy consumption dynamic optimization adjustment system of claim 1, wherein, In the constraint filtering module, the generation of control instructions specifically includes the following steps: Parse the initial policy action set and extract the radio frequency resource reconfiguration action corresponding to each 5G base station; The radio frequency resource reconfiguration actions are input into the network simulation model for calculation, and the predicted load value and predicted signal coverage area of ​​each 5G base station are output. The predicted load value and the predicted signal coverage area are then used to construct the predicted network parameters of each 5G base station. The network constraint threshold is analyzed to extract the upper limit value of the capacity corresponding to the predicted load value and the lower limit value of the coverage area corresponding to the predicted signal coverage area. The predicted load value is compared with the upper limit of the capacity value as a first value, and the predicted signal coverage area is compared with the lower limit of the coverage area value as a second value; The radio frequency resource reconfiguration actions where the predicted load value is greater than the upper limit of the capacity value are extracted as the first rejection actions, and the radio frequency resource reconfiguration actions where the predicted signal coverage area is less than the lower limit of the coverage area value are extracted as the second rejection actions. The first action to be removed and the second action to be removed are removed from the initial policy action set. The remaining policy action set after removing the first action to be removed and the second action to be removed is extracted and the remaining policy action set is compiled into control instructions. 8.The 5G base station energy consumption dynamic optimization adjustment system based on deep reinforcement learning of claim 1, wherein, In the execution feedback module, based on the control command, the control command is sent to the baseband processing unit and radio frequency unit of each 5G base station to control each 5G base station to perform the radio frequency resource reconfiguration action, and the network status feedback value after the radio frequency resource reconfiguration action is collected and sent back to the feature perception module. Specifically, this includes the following steps: The control command is parsed, the configuration parameters corresponding to each 5G base station are extracted, and the configuration parameters are encapsulated into a driver message. The driver message is sent to the baseband processing unit and the radio frequency unit of each 5G base station through the interface protocol, and the control interface of each 5G base station is called to modify the register value of each 5G base station and execute the radio frequency resource reconfiguration action. Read the status registers and counters of each 5G base station, obtain the power consumption value and number of connections after the radio frequency resource reconfiguration action is performed, and construct the power consumption value and the number of connections into a network status feedback value; The network status feedback value is transmitted to the feature perception module through the communication interface.

Images