Antenna downward inclination angle optimization method and system based on flow prediction and reinforcement learning

Abstract

The invention discloses an antenna downward inclination angle optimization method and system based on flow prediction and reinforcement learning, and relates to the technical field of wireless communication. Constructing a space-time event cube data set of the network traffic data of the base stations in the target area; obtaining a traffic prediction model according to the space-time event cube data set; predicting a future traffic value of each base station in the target area through a traffic prediction model; and according to the future flow value and the three-dimensional position information of each base station, after carrying out clustering analysis on whether the base station participates in antenna downward inclination angle adjustment, determining the base station which needs to be subjected to antenna downward inclination angle adjustment. According to the method, after accurate network flow prediction is obtained, clustering analysis of antenna downward inclination angle adjustment is carried out based on the predicted flow and the geographic position of the user, collaborative optimization adjustment is carried out on the antenna downward inclination angle between the base stations through a deep reinforcement learning algorithm, and therefore reasonable scheduling and utilization of mobile communication network resources are effectively achieved.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a method and system for optimizing antenna downtilt angle based on traffic prediction and reinforcement learning. Background Art

[0002] With the rapid development of mobile communication technology, users' demand for high-speed data transmission and seamless coverage continues to increase. Mobile services change dynamically in time and space, and different events will also affect mobile data traffic, resulting in a tidal effect in base station network traffic. As a key component of the mobile communication system, the downtilt angle setting of the antenna directly affects the coverage, signal quality and network capacity of the base station. Reasonable antenna downtilt angle can not only optimize signal coverage, but also effectively reduce interference between cells and improve overall network performance. The adjustment of antenna downtilt angle is an important research direction in the field of wireless communication. It aims to expand or reduce cell coverage, improve communication service quality, and balance network performance between different base stations by reasonably adjusting wireless network parameters. Its goal is to improve the resource utilization efficiency of base stations, achieve reliable transmission and balance network capacity. This process requires comprehensive consideration of multiple factors, including user communication needs, base station radio environment, and user geographic location. The traditional antenna downtilt angle setting mainly relies on engineers' experience and field testing. However, with the expansion of network scale and the complexity of the environment, this method has been difficult to meet the needs of efficient and accurate network optimization. Therefore, how to scientifically and automatically optimize the antenna downtilt angle has become an important research direction in the field of mobile communications.

[0003] By properly adjusting the antenna downtilt angle, users' communication experience can be improved, operating costs can be effectively reduced, and the stability and efficient operation of the mobile communication network can be ensured. In addition, this optimization helps the network adapt to the growing communication needs and the introduction of new technologies. Common strategies for antenna downtilt angle include simulation-based optimization, measurement data-based, machine learning-based, multi-objective optimization, dynamic optimization, etc. By using these strategies alone or in combination, traffic can be dynamically allocated according to network scale, performance requirements, and application scenarios, ensuring efficient, stable, and reliable operation of the mobile communication network.

[0004] In mobile communication networks, a variety of antenna downtilt optimization strategies are adopted, and the common goal of these strategies is to achieve the best balance between network performance, availability and resource utilization. These strategies include simulation-based methods, which use wireless propagation models and network simulation tools to simulate the impact of different downtilt angles on signal coverage and interference; measurement data-based methods, which analyze actual network conditions through drive tests and network performance indicators to guide downtilt adjustment; machine learning-based methods, which build data-driven prediction models and apply intelligent optimization algorithms to find the best configuration; multi-objective optimization strategies, which make trade-offs in coverage, system capacity and interference management to seek the overall optimal solution; and dynamic optimization methods, which dynamically adjust the downtilt angle according to real-time user distribution and environmental changes to achieve adaptive optimization of the network. By applying these strategies alone or in combination, the performance of mobile communication networks can be effectively improved to meet the growing communication needs.

[0005] At present, traffic prediction of base station networks mainly relies on simple historical data analysis, which fails to fully consider the complex and changeable spatial characteristics and event characteristics in reality. The limitations of this method lead to low accuracy of traffic prediction. At the same time, although there are antenna downtilt optimization methods based on reinforcement learning, their effects are not ideal due to the lack of accurate traffic prediction as support. Therefore, how to effectively optimize the antenna downtilt angle of mobile communication networks has become an important issue that needs to be solved urgently. Summary of the invention

[0006] In view of the defects existing in the prior art, the technical problem solved by the present invention is: how to effectively adjust the antenna downtilt angle of a mobile communication network.

[0007] To achieve the above objectives, in a first aspect, an embodiment of the present application provides an antenna downtilt angle optimization method based on traffic prediction and reinforcement learning, the method comprising the following steps:

[0008] Construct a spatiotemporal event cube dataset of network traffic data of base stations in the target area;

[0009] In the same dimension, the space, time and event data in the spatiotemporal event cube data set are fused; the corresponding features are extracted in the time domain, space domain and event domain respectively, and the extracted features are cross-fused to achieve the fusion of multi-domain information and obtain the traffic prediction model;

[0010] The future traffic value of each base station in the target area is predicted by the traffic prediction model; based on the future traffic value and three-dimensional location information of each base station, a cluster analysis is performed on the base stations to determine whether they participate in the antenna downtilt angle adjustment, and the base stations that need to adjust the antenna downtilt angle are determined;

[0011] According to the future traffic value of the base station for which the antenna downtilt angle adjustment is required, the antenna downtilt angle adjustment scheme of the current base station is calculated to complete the adjustment of the antenna downtilt angle.

[0012] In conjunction with the first aspect, in one implementation, the process of constructing a spatiotemporal event cube dataset of network traffic data of base stations in a target area includes:

[0013] Set the target area and network traffic data collection period;

[0014] Collect network traffic data of base stations in the target area according to the collection period;

[0015] Based on the time, space, event dimensions and flow values ​​of network traffic data, a spatiotemporal event cube dataset of network traffic data is constructed.

[0016] In combination with the first aspect, in one implementation, after the cube data set is constructed, the following steps are further included: slicing the cube data set in the time domain and classifying it through a classification algorithm to obtain a cube data set that meets the requirements.

[0017] In combination with the first aspect, in one implementation, the process of acquiring the traffic prediction model includes:

[0018] Perform full connection layer processing on the data in the spatiotemporal event cube data to achieve the fusion of space, time and event data in the same dimension;

[0019] In the spatial domain, the trend feature extraction module and node feature extraction module with attention mechanism are used to analyze the correlation of data and obtain spatial domain features.

[0020] In the time domain, the trend feature extraction module, period feature extraction module and noise feature extraction module with attention mechanism are used to perform correlation analysis on the time domain data to obtain the time domain features.

[0021] In the event domain, data association analysis is performed through the trend feature extraction module and node feature extraction module with attention mechanism to obtain event domain features;

[0022] By cross-fusion, the spatial domain features, time domain features and event domain features are integrated and fused to obtain the corresponding three vectors;

[0023] After the three vectors are subjected to two layers of identical multi-domain feature extraction and cross-fusion operations, they are passed through two layers of fully connected networks to obtain the traffic prediction results.

[0024] In combination with the first aspect, in one embodiment, the process of obtaining a traffic prediction result by subjecting the three vectors to two layers of identical multi-domain feature extraction and cross-fusion operations, and then passing them through two layers of fully connected networks includes: subjecting the three vectors to two layers of identical multi-domain feature extraction and cross-fusion operations to obtain three fused vectors; flattening the three fused vectors, and fusing and extracting features through two layers of fully connected networks to obtain a traffic prediction result.

[0025] In combination with the first aspect, in one implementation, the process of determining a base station that needs to adjust the antenna downtilt angle includes:

[0026] The base stations are sorted in a specified order according to the future traffic values ​​of the base stations, and the designated base stations are determined as cluster centers according to the sorting results;

[0027] The base stations are clustered by clustering algorithm, and the new cluster center is calculated according to the average distance within the cluster. The algorithm is iterated continuously until convergence, and the base stations that need to adjust the antenna downtilt angle are obtained.

[0028] In combination with the first aspect, in one implementation, the average distance is defined in three-dimensional space, and the specific calculation formula is:

[0029] v s =(l s ,s s ,n s )

[0030] Among them, v s represents the three-dimensional space distance of the sth cell, l s Indicates the traffic volume of the current cell, s s Indicates the ratio of the number of users in the middle and lower layers of the current cell's three-dimensional space to the total number of users in the cell, n s Indicates the ratio of the number of users in the three-dimensional space of the current cell to the total number of users in the cell; s s and n s Obtained according to the corresponding three-dimensional position information.

[0031] In combination with the first aspect, in one implementation, the state information of the deep reinforcement learning algorithm is:

[0032]

[0033] Among them, state represents the state information of the deep reinforcement learning algorithm, R represents the number of cells in the area of ​​interest, represents the predicted traffic of the Rth cell, S R It represents the ratio of the number of low- and medium-level users in the Rth cell to the total number of users, N R It represents the ratio of the number of nearby users in the Rth cell to the total number of users;

[0034] The actions of the deep reinforcement learning algorithm are:

[0035] action=(a1,a2,...,a R )

[0036] Among them, action represents the action of the deep reinforcement learning algorithm, a R Indicates the antenna downtilt angle value of the Rth cell.

[0037] In combination with the first aspect, in one implementation, the process of determining the three-dimensional position information includes: determining the three-dimensional coordinate position of the user according to the two-dimensional coordinate position of the user.

[0038] In a second aspect, an embodiment of the present application provides an antenna downtilt angle optimization system based on traffic prediction and reinforcement learning, which is used to implement the steps of the method described in any one of claims 1 to 9.

[0039] Compared with the prior art, the advantages of the present invention are:

[0040] The spatial domain neural network based on the attention mechanism of the present invention can accurately extract the spatial features of location information, break through geographical barriers, and capture subtle correlations in space. The introduction of event domain feature extraction makes the model more suitable for actual scenarios and can accurately capture the impact of events on traffic, especially showing extremely high accuracy when responding to temporary emergencies.

[0041] At the same time, the present invention can perform fusion synchronously in the process of feature extraction through the combination of cross fusion and fully connected layers, avoiding the performance loss that may be caused by directly performing low-dimensional temporal feature processing after extracting high-dimensional spatial features, thereby improving the robustness of the model.

[0042] On this basis, after obtaining accurate network traffic prediction, the present invention performs cluster analysis on antenna downtilt adjustment based on the predicted traffic and the user's geographic location, and collaboratively optimizes and adjusts the antenna downtilt angles between base stations through a deep reinforcement learning algorithm, thereby effectively realizing the rational scheduling and utilization of mobile communication network resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1It is a framework flow chart of the antenna downtilt angle optimization method based on traffic prediction and reinforcement learning in an embodiment of the present invention;

[0045] Figure 2 is a detailed flow chart of an antenna downtilt angle optimization method based on traffic prediction and reinforcement learning in an embodiment of the present invention;

[0046] Figure 3 The figure is a schematic diagram of the hardware structure of the antenna downtilt angle optimization device based on traffic prediction and reinforcement learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0048] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0049] On the first aspect, an embodiment of the present application provides an antenna downtilt angle optimization method based on traffic prediction and reinforcement learning, which belongs to the cross-technology of the Chinese invention patent "A load balancing optimization method and device based on traffic prediction and reinforcement learning" with publication number CN117479231A.

[0050] See also Figure 1 As shown, the method comprises the following steps:

[0051] Step A: Obtain the location information and periodic network traffic data of the base stations in the target area, and then construct a spatiotemporal event cube dataset of the network traffic data of the base stations in the target area.

[0052] Step B: In the same dimension, the space, time and event data in the spatiotemporal event cube dataset are fused; the corresponding features are extracted in the time domain, space domain and event domain respectively, and the extracted features are cross-fused to achieve the fusion of multi-domain information and obtain the traffic prediction model.

[0053] Step C: Predict the future traffic value of each base station in the target area through the traffic prediction model; perform cluster analysis on whether the base stations participate in antenna downtilt adjustment based on the future traffic value of each base station and the three-dimensional location information of the user, and determine the base stations that need to adjust the antenna downtilt angle.

[0054] Step D: According to the future traffic value of the base station for which the antenna downtilt angle adjustment is required, a deep reinforcement learning algorithm (such as DQN, DDPG, PPO, A3C, SAC) is used to calculate the antenna downtilt angle adjustment scheme of the current base station to complete the adjustment of the antenna downtilt angle.

[0055] In one embodiment, the process of step A includes:

[0056] Step A01: Set the target area (area of ​​interest or designated area) and the collection period of network traffic data (eg, 15 seconds).

[0057] Step A02: Obtain event data of a specific location in the future through an information website; for example, relevant event information of a location at a specific time point in the future may be obtained through a weather website, a map service, or a ticket purchasing website.

[0058] Step A03: Collect network traffic data of base stations in the target area according to a collection period, and save it through a time series database and a storage device.

[0059] Step A04: Construct a spatiotemporal event cube dataset of network traffic data based on the time, space, event dimensions and traffic value of the network traffic data.

[0060] Furthermore, after constructing the cube data set, step A04 also includes the following steps: slicing the spatiotemporal event cube data set in the time domain, and classifying the data set using an SVM (support vector machine) classification algorithm; and removing contaminated data that does not meet the requirements after classification through a data clipping method, thereby obtaining a "pure" cube data set.

[0061] Specifically, the data will be sliced ​​by time domain, such as weekly units, and then the SVM classification algorithm will be used to divide the data into two categories: pure data and contaminated data. The contaminated data that does not meet the requirements (such as pulsed data or seriously abnormal data) will be deleted through trimming methods.

[0062] In one embodiment, the process of step B includes:

[0063] Step B01: Perform full connection layer processing on the data in the spatiotemporal event cube data to achieve the fusion of space, time and event data in the same dimension.

[0064] The present invention uses a multi-domain feature extraction and cross-fusion method to analyze and predict network traffic data. The clean data is first operated through a fully connected layer to initially fuse the data at different time points to form a four-dimensional feature space of time-space-event-traffic. That is, the spatiotemporal event cube data set is used to record the traffic value corresponding to a certain time, a specific spatial location, and event information.

[0065] Step B02: In the spatial domain, the trend feature extraction module and node feature extraction module with attention mechanism are used to perform data correlation analysis to obtain spatial domain features. For example, the traffic characteristics of CBD and Zhongguancun during working hours may be similar, and the module with attention mechanism can help capture these correlations.

[0066] The trend feature extraction module of the attention mechanism enhances the importance of fixed trend data by giving it a higher weight. For example, when predicting the traffic flow of a community at 10 am, the traffic flow from 8 to 10 am, the traffic flow in the same time period yesterday, and the traffic flow in the same period last week will be given a greater weight. In the four-dimensional feature space, the geographical location is first fixed, and the other three-dimensional information forms an index table and is input into the attention module to efficiently extract features. Then, the spatial domain information is further extracted through the graph convolutional neural network and the convolutional network. If the attention mechanism is not used, a large amount of redundant information may slow down the learning process and cannot effectively process very long time series data. The extracted spatial domain features are then input into the time domain and event domain for fusion. If multi-domain cross-fusion is not performed, the combination of low-dimensional information is not sufficient, which may lead to a decrease in prediction accuracy.

[0067] Step B03: In the time domain, by using a trend feature extraction module with an attention mechanism, a period feature extraction module, and a noise feature extraction module, correlation analysis of the time domain data is performed to obtain time domain features.

[0068] Similar to the trend feature extraction module in the spatial domain, the trend data in the time domain will also be enhanced by the attention mechanism. However, network traffic in the time domain is often periodic, so it is necessary to introduce a periodic feature extraction module (for example, data from Monday to Friday constitutes a period) and add a noise feature extraction module to remove the impact of the period. After that, the information will be further extracted through the graph convolutional neural network and the convolutional network. The extracted time domain features will be passed as input to the spatial domain and event domain. If the periodic attention mechanism is not used, the periodic characteristics of network traffic may not be effectively captured, resulting in reduced prediction accuracy.

[0069] Step B04: In the event domain, data association analysis is performed through the trend feature extraction module and the node feature extraction module with the attention mechanism to obtain event domain features.

[0070] Event domain feature extraction focuses on the analysis of weather conditions, traffic flow, large-scale events such as concerts, and holidays. The attention mechanism is used to strengthen the extraction of trend features and node features, similar to the operation in the spatial domain. After further processing by the graph convolutional neural network and convolutional network, the event domain features will be input into the spatial domain and time domain.

[0071] Step B05: Integrate and fuse the spatial domain features, temporal domain features and event domain features by cross-fusion to obtain three corresponding vectors (spatial domain vector, temporal domain vector and event domain vector).

[0072] Step B06: After the three vectors are subjected to two layers of the same multi-domain feature extraction and cross-fusion operations, they are passed through two layers of fully connected networks to obtain the traffic prediction results. At this time, the training of the traffic prediction model can be completed and the traffic prediction model can be obtained.

[0073] Specifically, the process of step B06 includes:

[0074] The three vectors are subjected to two layers of the same multi-domain feature extraction and cross-fusion operations to obtain three fused vectors: that is, the first layer performs multi-domain feature extraction and cross-fusion operations on the three vectors to obtain three new vectors, and the second layer performs multi-domain feature extraction and cross-fusion operations on the three new vectors to obtain three fused vectors (spatial domain fusion vectors corresponding to spatial domain vectors, time domain fusion vectors corresponding to time domain vectors, and event domain fusion vectors corresponding to event domain vectors).

[0075] The three fused vectors are flattened and fused through two layers of fully connected networks to extract features, and then the traffic prediction result is obtained.

[0076] It should be noted that: after completing the training of the spatiotemporal event cross-fusion network feature extraction model, that is, completing the training of the prediction model, it is necessary to iterate and verify repeatedly on the test set to ensure that the average prediction accuracy of the prediction model is lower than the preset threshold. The model needs to be tested on the test set to ensure that it meets the generalization requirements. When the prediction error of the model is less than the specified accuracy standard (for example, the root mean square error is lower than the set value), the model is saved for future use.

[0077] In one embodiment, the process of determining the three-dimensional position information in step C includes: locating the user's two-dimensional coordinate position through TDOA (Time Difference of Arrival), GPS, and Wi-Fi assisted technologies, and determining the user's three-dimensional coordinate position through high-precision three-dimensional building maps and base station signal beam data, thereby forming a high, medium, and low-level distribution cognition of the user.

[0078] In one embodiment, the process of step C includes:

[0079] Step C01: sorting the base stations in a specified order (from high to low) according to the predicted future traffic values ​​of the base stations in the target area;

[0080] Step C02: Select the first and last cells in the sorting results as initial cluster centers;

[0081] Step C03: Apply the K-Means clustering algorithm to perform cluster analysis on the base stations, and then calculate the new cluster center based on the average distance within the cluster. After continuous iteration until the algorithm converges, the base stations that need to adjust the antenna downtilt angle and the base stations that do not need to adjust the antenna downtilt angle are obtained. The above average distance is defined in three-dimensional space, and the specific calculation formula is:

[0082] v s =(l s ,s s ,n s )

[0083] Among them, v s represents the three-dimensional space distance of the sth cell, l s Indicates the traffic volume of the current cell, s s Indicates the ratio of the number of users in the middle and lower layers of the current cell's three-dimensional space to the total number of users in the cell, n s Indicates the ratio of the number of users in the three-dimensional space of the current cell to the total number of users in the cell; s s and n s Obtained according to the corresponding three-dimensional position information.

[0084] In one embodiment, the state information of the deep reinforcement learning algorithm in step D is:

[0085]

[0086] Among them, state represents the state information of the deep reinforcement learning algorithm, R represents the number of cells in the area of ​​interest, represents the predicted traffic of the Rth cell, S R It represents the ratio of the number of low- and medium-level users in the Rth cell to the total number of users, N R It indicates the ratio of the number of nearby users in the Rth cell to the total number of users.

[0087] The actions of the deep reinforcement learning algorithm are:

[0088] action=(a1,a2,...,a R )

[0089] Among them, action represents the action of the deep reinforcement learning algorithm, a R Indicates the antenna downtilt angle value of the Rth cell;

[0090] The reward for the deep reinforcement learning algorithm is the total system throughput in the target area.

[0091] On this basis, the specific process of using the deep reinforcement learning algorithm to calculate the antenna downtilt angle adjustment plan of the current base station in step D includes: according to the location ratio of the cell terminal equipment (obtained according to the geographical location) and the future traffic value, the antenna downtilt angle adjustment parameters of the current base station are obtained as actions through the deep reinforcement learning algorithm. The final goal is to optimize the system throughput.

[0092] At the same time, before step D is calculated by the deep reinforcement learning algorithm, it also includes the following steps: establishing a handshake communication with the current base station. Successful handshake communication means that the current base station can perform antenna downtilt operation.

[0093] In summary, the present invention proposes an antenna downtilt optimization method based on traffic prediction and reinforcement learning. This method fully considers multi-dimensional factors such as time, space and events, adopts more efficient feature extraction and fusion technology, and further optimizes the antenna downtilt strategy after traffic prediction. This method can effectively solve the problem of resource optimization configuration in current mobile communication networks, while ensuring transmission accuracy, improving energy efficiency and saving costs.

[0094] Preferably, firstly, full connection operations are performed in the time domain, space domain and event domain respectively to fuse the respective trend information. Then, in the space domain, feature extraction is performed using the trend feature extraction module and the node feature extraction module with the attention mechanism, and then spatiotemporal features are extracted through the graph convolution network and the convolutional neural network. After completion, the results are respectively input into the time domain and event domain modules for cross fusion.

[0095] Similarly, in the time domain, the trend feature extraction module, period feature extraction module and noise feature extraction module with attention mechanism are used for feature extraction, and then the graph convolution network and convolutional neural network are used for spatiotemporal feature extraction. After completion, they are passed to the space domain and event domain modules for cross-fusion.

[0096] Again, in the event domain, the trend feature extraction module and node feature extraction module with attention mechanism are used for feature extraction, and then the spatiotemporal feature extraction is performed through the graph convolutional network and convolutional neural network. After completion, the results are sent to the time domain and space domain modules for cross fusion. In this way, a layer of time-space-event feature cross fusion is completed, and then the same operation is repeated for two layers. Finally, after flattening the features, they are connected to the two-layer fully connected network for the final information fusion to obtain the prediction results.

[0097] It should be pointed out that if we only rely on the attention mechanism, we may ignore the subtle correlations between multiple domains. The current feature extraction method usually extracts features from the time domain first, and then extracts features from the spatial domain, lacking the cross-fusion of multi-domain features between high-dimensional and low-dimensional domains.

[0098] In the activation function of all neural networks, the Leaky ReLU function (i.e., an activation function) is used to avoid the problem of "dead neurons". The loss function uses the root mean square error (RMSE) to accurately evaluate the degree of prediction deviation.

[0099] After the model meets the accuracy requirements, it is deployed and the predicted traffic of the cell is calculated. The communication scenario considered is that the cell is overloaded and key coverage is achieved by adjusting the downtilt angle of the cell antenna. According to the predicted traffic, the cells are clustered using the K-Means clustering algorithm, and the cells are clustered into cells that need to participate in the antenna downtilt adjustment and cells that do not need to participate in the antenna downtilt adjustment. The cells are queried in a "handshake" manner to determine the final list of cells that participate in the antenna downtilt adjustment.

[0100] Based on the cell list, traffic prediction results, user terminal device location and other environmental information, the antenna downtilt angle value of the cell is given through a deep reinforcement learning algorithm to adjust the antenna downtilt angle and dynamically meet the wireless network traffic demand.

[0101] The following combination Figure 2 , the antenna downtilt angle optimization method based on traffic prediction and reinforcement learning of the present invention is specifically described:

[0102] S101: receiving a target area and a time period for traffic collection input by a user, and proceeding to S102;

[0103] The central server prepares to adjust the antenna downtilt angle according to the area specified by the user and sets the time interval for traffic data collection (for example: 1 second / 10 seconds / 1 minute / 30 minutes, etc.).

[0104] S102: The central server establishes communication with the base station. The base station will upload its network traffic data to the central server within the set time period to form a spatiotemporal event cube dataset. At the same time, the central server integrates Internet data (for example, event information on weekdays or weekends in the calendar, meteorological event information on weather websites, traffic flow event information on map websites, group activity information on concert websites, etc.) to jointly build a complete spatiotemporal event cube dataset and enter S103;

[0105] S103: Use the SVM classification algorithm to perform binary classification on the collected data, distinguishing them into pure data and polluted data, and then determine whether the spatiotemporal event cube dataset is a pure dataset; if so, proceed to S105, if not, proceed to S104; remove polluted data that does not meet the requirements by clipping (for example, there are pulse data, polluted data, etc.). The expression of SVM for binary classification is as follows:

[0106]

[0107] Among them, x represents the input sample to be classified; x i represents the i-th training sample; y i Indicates the label of the i-th training sample, with a value of +1 or -1; α i represents the Lagrange multiplier, representing the importance of the sample; K(x i ,x) represents the kernel function, which is used to calculate the similarity between samples; b represents the bias term; represents the symbolic function, which determines the output category; n represents the number of training samples.

[0108] S104: After confirming that the data of interest is contaminated, a pure spatiotemporal event cube dataset is obtained by a clipping method, and then proceeding to S105;

[0109] S105: After obtaining a clean data set, first perform a full connection operation, then perform feature extraction in the spatial domain, and enter S106. Preferably, a node feature extraction module with an attention mechanism is used to perform correlation analysis on the spatial domain.

[0110] Subsequently, the frequency domain graph convolution method is used to extract spatial features, that is, the convolution operation is realized by linearly diagonalizing the operator in the Fourier domain to replace the traditional convolution operator. At the same time, the convolutional neural network is used to extract temporal features.

[0111] Similarly, in the process of feature extraction in the time domain, a trend feature extraction module with an attention mechanism, a period feature extraction module, and a noise feature extraction module are included. The attention mechanism, multi-head attention feature extraction formula, and graph convolution feature extraction formula of these modules are the same as those in the spatial domain.

[0112] Similarly, in the feature extraction of the event domain, there are trend feature extraction modules and node feature extraction modules with attention mechanisms. Its attention mechanism, multi-head attention feature extraction formula, and graph convolution feature extraction formula are also consistent with those in the spatial domain.

[0113] S106: Use the extracted features to predict traffic. After completing feature extraction in the spatial domain, continue to extract features in the time domain and event domain; similarly, after completing feature extraction in the time domain, enter the spatial domain and event domain for further feature extraction; after extracting features in the event domain, enter the time domain and spatial domain again to extract features. Through this cycle, information fusion of multi-domain features at different high and low dimensions is achieved. Finally, access the two-layer fully connected network to complete the fusion of global information and enter S107.

[0114] The activation function of the neural network uses the Leaky ReLU function, which is expressed as follows:

[0115] f(x)=max(αx,x)

[0116] Where α is a decimal less than 1, usually set to 0.01. When x is greater than 0, the Leaky ReLU function is the same as the normal ReLU function, both are linear functions; when x is less than 0, the Leaky ReLU function is a nonlinear function. The LeakyReLU function makes the training process of the neural network more robust by solving the "neuron death" problem in the standard ReLU function. By using the fully connected layer, the subtle connections between different domains can be captured.

[0117] S107: Check whether the prediction accuracy meets the requirements; if yes, go to S108; if not, return to S105. After fusing multi-domain features, the root mean square error (RMSE) is used as the loss function to accurately evaluate the degree of prediction deviation. Its expression is as follows:

[0118]

[0119] in, It is i The predicted value of samples, It is i The true value of samples, n is the number of samples. The unit of RMSE is the same as that of the predicted value and the actual value. The smaller the RMSE value, the smaller the gap between the predicted result and the true value, and the higher the prediction accuracy. By calculating the RMSE, it is judged whether it is lower than the preset threshold to determine whether the prediction accuracy meets the requirements. If not, continue to perform multi-domain feature extraction and cross-fusion calculations until the required prediction accuracy is finally achieved.

[0120] S108: After the central processor has deployed the traffic prediction model, it calculates the predicted traffic of each cell and enters S109.

[0121] S109: After the CPU obtains the predicted traffic data, it uses a two-stage clustering algorithm to divide the cells into two categories: cells that need to participate in antenna downtilt adjustment and cells that do not need to participate in antenna downtilt adjustment, and then enters S110. The two-stage clustering algorithm includes: 1) calculating the network traffic of each cell and sorting them from high to low according to the traffic; 2) using the K-Means clustering algorithm to perform cluster analysis on the cells. Distance is a three-dimensional space v s =(l s ,s s ,n s ) is the distance defined in .

[0122] S110: After obtaining the traffic prediction data and the antenna downtilt angle list, the central processor communicates with each cell base station again to confirm whether the antenna downtilt angle adjustment operation can be performed; if so, it enters S111; if not, it returns to S101 to avoid being unable to adjust resources in the current period.

[0123] S111: The central processing unit calculates an adjustment plan for the antenna downtilt angle through a deep reinforcement learning algorithm based on the predicted network traffic and the user's three-dimensional location information and other states, and executes the adjustment of the antenna downtilt angle.

[0124] The state information of the deep reinforcement learning algorithm is:

[0125]

[0126] Among them, state represents the state information of the deep reinforcement learning algorithm, R represents the number of cells in the area of ​​interest, represents the predicted traffic of the Rth cell, S R It represents the ratio of the number of low- and medium-level users in the Rth cell to the total number of users, N R It represents the ratio of the number of nearby users in the Rth cell to the total number of users;

[0127] The actions of the deep reinforcement learning algorithm are:

[0128] action=(a1,a2,...,a R )

[0129] Among them, action represents the action of the deep reinforcement learning algorithm, a R Indicates the antenna downtilt angle value of the Rth cell;

[0130] The reward of the deep reinforcement learning algorithm is the total system throughput in the area of ​​interest. The antenna downtilt parameters of each cell are given as actions through five classic deep reinforcement learning algorithms: DQN, DDPG, PPO, A3C, and SAC. The final goal is to optimize the system throughput.

[0131] In a second aspect, an embodiment of the present invention further provides an antenna downtilt angle optimization system based on traffic prediction and reinforcement learning, which is used to implement the steps of the above-mentioned antenna downtilt angle optimization method.

[0132] Specifically, the system includes:

[0133] The base station module is used to upload the network traffic data of the base station and perform the antenna downtilt angle adjustment operation;

[0134] The data construction module is used to execute the process of step A in the above method. Specifically, the module includes:

[0135] The data collection module is used to collect network traffic data of base stations in the target area according to a set fixed period;

[0136] The data preprocessing module is used to filter out polluted network traffic data using the SVM classification algorithm to obtain a pure data set.

[0137] The traffic prediction model training module is used to: execute the process of step B in the above method. Specifically, the module includes:

[0138] The spatiotemporal event feature extraction module is used to extract the features of network traffic data from three dimensions: time, space and events;

[0139] The cross fusion and traffic prediction module is used to: fuse multi-domain features and generate traffic prediction results.

[0140] The antenna downtilt angle adjustment cell clustering module is used to: execute step C in the above method; that is, perform cluster analysis on the cells to determine whether antenna downtilt angle adjustment is required based on the traffic prediction result and location information.

[0141] The antenna downtilt angle adjustment reinforcement learning module is used to: execute step D in the above method; that is, use a deep reinforcement learning algorithm to calculate the antenna downtilt angle adjustment plan of the cell, and send the plan to the base station module to perform antenna downtilt angle adjustment.

[0142] The timing relationship between each module includes: the base station module sends base station traffic data, base station location, user three-dimensional location and other information to the data acquisition module; the data acquisition module passes the collected traffic data to the data preprocessing module; the preprocessed data is sent to the spatiotemporal event feature extraction module; then, the data enters the multi-domain feature cross-fusion and traffic prediction module for traffic prediction; the prediction results are passed to the antenna downtilt angle adjustment cell clustering module to cluster the antenna downtilt angles of the cells; finally, the antenna downtilt angle adjustment reinforcement learning module calculates the optimization plan for the antenna downtilt angle based on the deep reinforcement learning algorithm, and sends it to each base station module to achieve dynamic optimization and utilization of base station resources in the region.

[0143] The present invention not only integrates information from the time domain and the space domain, but also further considers the impact of different events on traffic prediction. In addition, the results of different domains are cross-fused in the middle layer, which enhances the ability to capture the time-space-event correlation between various high and low dimensions. By designing a graph convolutional neural network, the location-related features between any two base stations are effectively extracted.

[0144] At the same time, the present invention performs a cluster analysis of the antenna downtilt angles of the cells, identifies the cells that are necessary and qualified to participate in the antenna downtilt angle adjustment, and avoids excessive switching. With the help of deep reinforcement learning, the environment is learned and the antenna downtilt angle parameters of the cells are adjusted, maximizing the system throughput and reducing costs. By adjusting the antenna downtilt angle in advance through traffic prediction, it can respond to demand more quickly than solutions based only on reinforcement learning.

[0145] In addition, the present invention adopts an antenna downtilt angle adjustment mechanism that combines traffic prediction and reinforcement learning to predict network traffic in real time and perform antenna downtilt angle adjustment operations in real time, which is more adaptable to the actual needs of base stations intelligently sensing the environment and dynamically adjusting, thereby improving the efficiency of resource utilization.

[0146] On the third aspect, an embodiment of the present application provides an antenna downtilt angle optimization device based on traffic prediction and reinforcement learning. The antenna downtilt angle optimization device based on traffic prediction and reinforcement learning can be a personal computer (PC), a laptop computer, a server, and other devices with data processing functions.

[0147] Reference Figure 3 , Figure 3 The hardware structure diagram of the antenna downtilt optimization device based on traffic prediction and reinforcement learning involved in the embodiment of the present application is shown in FIG. In the embodiment of the present application, the antenna downtilt optimization device based on traffic prediction and reinforcement learning may include a processor, a memory, a communication interface, and a communication bus.

[0148] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0149] The communication interface includes an input / output (I / O) interface, a physical interface, and a logical interface, etc., which are used to interconnect the devices inside the antenna downtilt optimization device based on traffic prediction and reinforcement learning, and an interface used to interconnect the antenna downtilt optimization device based on traffic prediction and reinforcement learning with other devices (such as other computing devices or user equipment). The physical interface can be an Ethernet interface, an optical fiber interface, an ATM interface, etc.; the user equipment can be a display, a keyboard, etc.

[0150] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0151] The processor may be a general-purpose processor, which may call the antenna downtilt optimization program based on traffic prediction and reinforcement learning stored in the memory, and execute the antenna downtilt optimization method based on traffic prediction and reinforcement learning provided in the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the antenna downtilt optimization program based on traffic prediction and reinforcement learning is called may refer to the various embodiments of the antenna downtilt optimization method based on traffic prediction and reinforcement learning of the present application, which will not be repeated here.

[0152] Those skilled in the art will understand that Figure 3 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0153] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0154] The computer-readable storage medium of the present application stores an antenna downtilt angle optimization program based on traffic prediction and reinforcement learning, wherein when the antenna downtilt angle optimization program based on traffic prediction and reinforcement learning is executed by a processor, the steps of the antenna downtilt angle optimization method based on traffic prediction and reinforcement learning as described above are implemented.

[0155] Among them, the method implemented when the antenna downtilt angle optimization program based on traffic prediction and reinforcement learning is executed can refer to the various embodiments of the antenna downtilt angle optimization method based on traffic prediction and reinforcement learning of the present application, and will not be repeated here.

[0156] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0158] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0159] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0160] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0161] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0162] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0163] The above are only specific implementations of the embodiments of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the embodiments of the present invention, and these modifications or replacements should be included in the protection scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for optimizing antenna downtilt angle based on traffic prediction and reinforcement learning, characterized in that: The method comprises the following steps: Construct a spatiotemporal event cube dataset of network traffic data of base stations in the target area; In the same dimension, the space, time and event data in the spatiotemporal event cube data set are fused; the corresponding features are extracted in the time domain, space domain and event domain respectively, and the extracted features are cross-fused to achieve the fusion of multi-domain information and obtain the traffic prediction model; The future traffic value of each base station in the target area is predicted by the traffic prediction model; based on the future traffic value and three-dimensional location information of each base station, a cluster analysis is performed on the base stations to determine whether they participate in the antenna downtilt angle adjustment, and the base stations that need to adjust the antenna downtilt angle are determined; According to the future traffic value of the base station for which the antenna downtilt angle adjustment is required, the antenna downtilt angle adjustment scheme of the current base station is calculated to complete the adjustment of the antenna downtilt angle.

2. The antenna downtilt angle optimization method based on traffic prediction and reinforcement learning as claimed in claim 1, characterized in that: The process of constructing a spatiotemporal event cube dataset of network traffic data of base stations in the target area includes: Set the target area and network traffic data collection period; Collect network traffic data of base stations in the target area according to the collection period; Based on the time, space, event dimensions and flow values ​​of network traffic data, a spatiotemporal event cube dataset of network traffic data is constructed.

3. The antenna downtilt angle optimization method based on traffic prediction and reinforcement learning as described in claim 2, characterized in that: After the cube data set is constructed, the following steps are also included: slicing the cube data set in the time domain and classifying it through a classification algorithm to obtain a cube data set that meets the requirements.

4. The antenna downtilt angle optimization method based on traffic prediction and reinforcement learning as claimed in claim 1, characterized in that: The process of obtaining the traffic prediction model includes: Perform full connection layer processing on the data in the spatiotemporal event cube data to achieve the fusion of space, time and event data in the same dimension; In the spatial domain, the trend feature extraction module and node feature extraction module with attention mechanism are used to analyze the correlation of data and obtain spatial domain features. In the time domain, the trend feature extraction module, period feature extraction module and noise feature extraction module with attention mechanism are used to perform correlation analysis on the time domain data to obtain the time domain features. In the event domain, data association analysis is performed through the trend feature extraction module and node feature extraction module with attention mechanism to obtain event domain features; By cross-fusion, the spatial domain features, temporal domain features and event domain features are integrated and fused to obtain the corresponding three vectors; After the three vectors are subjected to two layers of identical multi-domain feature extraction and cross-fusion operations, they are passed through two layers of fully connected networks to obtain the traffic prediction results.

5. The antenna downtilt angle optimization method based on traffic prediction and reinforcement learning as described in claim 4 is characterized in that: The process of obtaining the traffic prediction result by performing two layers of identical multi-domain feature extraction and cross-fusion operations on the three vectors and then performing two layers of fully connected networks includes: performing two layers of identical multi-domain feature extraction and cross-fusion operations on the three vectors to obtain three fused vectors; flattening the three fused vectors and fusing and extracting features through two layers of fully connected networks to obtain the traffic prediction result.

6. The antenna downtilt angle optimization method based on traffic prediction and reinforcement learning as claimed in claim 1, characterized in that: The process of determining the base station that needs to adjust the antenna downtilt angle includes: The base stations are sorted in a specified order according to the future traffic values ​​of the base stations, and the designated base stations are determined as cluster centers according to the sorting results; The base stations are clustered by clustering algorithm, and the new cluster center is calculated according to the average distance within the cluster. The algorithm is iterated continuously until convergence, and the base stations that need to adjust the antenna downtilt angle are obtained.

7. The antenna downtilt angle optimization method based on traffic prediction and reinforcement learning as claimed in claim 6, characterized in that: The average distance is defined in three-dimensional space, and the specific calculation formula is: v s =(l s ,s s ,n s ) Among them, v s represents the three-dimensional space distance of the sth cell, l s Indicates the traffic volume of the current cell, s s Indicates the ratio of the number of users in the middle and lower layers of the current cell's three-dimensional space to the total number of users in the cell, n s Indicates the ratio of the number of users in the three-dimensional space of the current cell to the total number of users in the cell; s s and n s Obtained according to the corresponding three-dimensional position information.

8. The antenna downtilt angle optimization method based on traffic prediction and reinforcement learning as claimed in claim 1, characterized in that: The state information of the deep reinforcement learning algorithm is: Among them, state represents the state information of the deep reinforcement learning algorithm, R represents the number of cells in the area of ​​interest, represents the predicted traffic of the Rth cell, S R It represents the ratio of the number of low- and medium-level users in the Rth cell to the total number of users, N R It represents the ratio of the number of nearby users in the Rth cell to the total number of users; The actions of the deep reinforcement learning algorithm are: action=(a1,a2,...,a R ) Among them, action represents the action of the deep reinforcement learning algorithm, a R Indicates the antenna downtilt angle value of the Rth cell.

9. The antenna downtilt angle optimization method based on traffic prediction and reinforcement learning according to any one of claims 1 to 8, characterized in that: The process of determining the three-dimensional position information includes: determining the three-dimensional coordinate position of the user according to the two-dimensional coordinate position of the user. 10.An antenna downtilt angle optimization system based on traffic prediction and reinforcement learning, characterized in that: The system is used to implement the steps of the method described in any one of claims 1 to 9.

Images