Massive MIMO antenna parameter adjustment method and device

Abstract

The embodiment of the present invention provides a method and device for adjusting parameters of a large-scale MIMO antenna, the method comprising: determining the adjustment direction of the adjustment parameters of the antenna based on an expert experience method according to the current actual network state of the antenna; selecting an adjustment action from a Q-table table according to the adjustment direction, and finding the current Q value corresponding to each selected adjustment action of the current actual network state from the Q-table table; selecting a maximum value from the current Q values ​​found, and adjusting the antenna according to the adjustment action corresponding to the maximum value; calculating the reward function of the antenna according to the actual network state of the adjusted antenna, and updating the maximum value based on the reward function based on the Q-learning method until the performance index of the antenna meets the preset conditions. The embodiment of the present invention improves the accuracy of antenna parameter adjustment, requires fewer training samples, and has interpretability.

Description

Technical Field

[0001] The present invention relates to the technical field of massive MIMO antennas, and in particular to a massive MIMO antenna parameter adjustment method and device. Background Art

[0002] Traditional antenna feed optimization, such as 4G networks, mainly adjusts the azimuth and downtilt angles of the antenna based on MR (Measurement Report) data to achieve dynamic balanced optimization of network capacity, coverage, interference and other indicators. However, manual operation cannot accurately control the changes in mechanical azimuth and downtilt angles, and cannot achieve the expected results. The 5G network is more densely built, and the network environment coexists with the 4G network, which will make the optimization of network coverage, interference and other aspects more complicated.

[0003] Massive MIMO (Multi Input Multi Output) antenna technology is a key technology to improve system capacity and spectrum utilization, and introduces more remotely adjustable parameters, such as the number of beams, azimuth, downtilt, horizontal width, and vertical width. The traditional method of adjusting antennas based on engineer experience and tower workers is no longer applicable, and a set of intelligent solutions is needed to automatically optimize the antenna adjustment parameters in the later stage.

[0004] At present, most of the solutions for large-scale MIMO antenna adjustment are based on propagation model simulation and neural network learning based on MR data. Among them, the former performs well in free space scenarios, but in real scenarios, the environment is extremely complex, with tall buildings and different building materials. In addition, the impact of seasonal changes on the vegetation on both sides of the road and in the park makes the propagation model no longer reliable. Due to the reflection, diffraction and interference of wireless environment signals, MR data cannot accurately reflect the signal characteristics of a certain area, so the optimization effect of antenna adjustment parameters is poor. At the same time, neural networks are also a type of algorithm with poor model interpretability, resulting in low credibility of the final optimization solution. Not only that, neural networks require a large amount of training sample data for training. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for adjusting parameters of a large-scale MIMO antenna, which are used to solve the defects of the prior art that the optimization effect of the large-scale MIMO antenna adjustment parameters is poor, the interpretability is poor, and a large number of training samples are required for training, so as to achieve accurate optimization of the large-scale MIMO antenna adjustment parameters, which is interpretable and does not require a large number of training samples.

[0006] An embodiment of the present invention provides a method for adjusting parameters of a large-scale MIMO antenna, including:

[0007] According to the current actual network state of the antenna, determining the adjustment direction of the adjustment parameter of the antenna based on the expert experience method; wherein the current actual network state includes the value of the network indicator of the antenna;

[0008] Selecting an adjustment action from the Q-table of the antenna according to the adjustment direction, and searching the Q-table for the Q value of the last iteration update corresponding to each selected adjustment action of the current actual network state; wherein the adjustment action includes the value to be adjusted of the adjustment parameter;

[0009] Selecting a maximum value from the Q values ​​updated in the last iteration corresponding to each selected adjustment action of the current actual network state, and adjusting the antenna according to the adjustment action corresponding to the maximum value;

[0010] The reward function of the antenna is calculated according to the adjusted actual network state of the antenna, the maximum value is updated based on the Q-learning method according to the reward function, and the adjusted actual network state of the antenna is iterated as the current actual network state until the performance indicator of the antenna meets the preset conditions.

[0011] According to an embodiment of the present invention, the method for adjusting parameters of a massive MIMO antenna further includes:

[0012] Gridding the area covered by the antenna;

[0013] Calculate the current actual network status of the antenna in each grid according to the current MR data and KPI data of each grid;

[0014] An average value of the current actual network status of the antenna in all grids is calculated, and the average value is used as the current actual network status of the antenna.

[0015] According to a method for adjusting parameters of a massive MIMO antenna according to an embodiment of the present invention, the network indicators of the antenna include network coverage, network overlap coverage and resource utilization;

[0016] Accordingly, according to the current actual network state of the antenna, the step of determining the adjustment direction of the adjustment parameter of the antenna based on the expert experience method includes:

[0017] If the resource utilization rate is greater than a first preset threshold, the beam width in the adjustment parameter is reduced or the downtilt angle is reduced;

[0018] If the resource utilization rate is less than the second preset threshold, determining whether the network coverage rate is less than a third preset threshold;

[0019] If the network coverage is less than a third preset threshold, shrinking the beam width or pressing down the downtilt angle in the adjustment parameter;

[0020] If the network coverage is greater than the fourth preset threshold, determining whether the network overlap coverage is greater than the fifth preset threshold;

[0021] If the network overlap coverage is greater than a fifth preset threshold, shrinking the beam width or pressing down the downtilt angle in the adjustment parameter;

[0022] If the network overlap coverage is less than a sixth preset threshold, the bandwidth in the adjustment parameter is expanded or the downtilt angle is increased.

[0023] According to a method for adjusting parameters of a massive MIMO antenna according to an embodiment of the present invention, before the step of searching the Q-table table for the Q value updated in the last iteration corresponding to each selected adjustment action of the current actual network state, the method further includes:

[0024] Determining a value of each network indicator according to a preset range and a preset value step of each network indicator of the antenna;

[0025] Select any one value from the values ​​of each network indicator to combine and generate the network state of the antenna;

[0026] Determining a value to be adjusted for each adjustment parameter according to a preset range and a preset value step of each adjustment parameter of the antenna;

[0027] Selecting any one value from the values ​​to be adjusted of each adjustment parameter for combination to generate an adjustment action for the antenna;

[0028] Construct the Q-table according to the generated network status and adjustment action of the antenna;

[0029] Each row of the Q-table represents the Q value of any network state corresponding to each adjustment action, and each column of the Q-table represents the Q value of any adjustment action corresponding to each network state.

[0030] According to a method for adjusting parameters of a massive MIMO antenna according to an embodiment of the present invention, the step of selecting a maximum value from the Q value updated in the last iteration corresponding to each selected adjustment action of the current actual network state comprises:

[0031] Based on the ε-greedy algorithm, a maximum value is selected from the Q value updated in the last iteration corresponding to each selected adjustment action of the current actual network state.

[0032] According to a method for adjusting parameters of a massive MIMO antenna according to an embodiment of the present invention, the reward function of the antenna is calculated according to the actual network status of the adjusted antenna by the following formula:

[0033] R = x*wc+y*oc+z*prb;

[0034] Among them, R is the reward function, x, y and z are weight coefficients, wc is the network coverage rate in the actual network state of the antenna after adjustment, oc is the network overlap coverage rate in the actual network state of the antenna after adjustment, and prb is the resource utilization rate in the actual network state of the antenna after adjustment.

[0035] According to a method for adjusting parameters of a massive MIMO antenna according to an embodiment of the present invention, the maximum value is updated based on the reward function based on the Q-learning method by the following formula:

[0036]

[0037] Among them, Q′(S t ,A t ) indicates S t Corresponding to A t The updated Q value, S t represents the actual network status of the antenna after the adjustment parameter is adjusted for the tth time, A t represents the adjustment action corresponding to the maximum value obtained after the adjustment parameter is adjusted for the tth time, Q(S t ,A t )S t Corresponding to A t The Q value before the update, α is the learning rate, R is the reward function, γ is the discount factor, Indicates S t+1 Corresponding to S t+1 The maximum value among the Q values ​​of the selected adjustment action a.

[0038] The embodiment of the present invention further provides a large-scale MIMO antenna parameter adjustment device, including:

[0039] A determination module, configured to determine an adjustment direction of an adjustment parameter of the antenna based on an expert experience method according to a current actual network state of the antenna; wherein the current actual network state includes a value of a network indicator of the antenna;

[0040] A search module, configured to select an adjustment action from a Q-table of the antenna according to the adjustment direction, and search the Q-table for the Q value of the current actual network state corresponding to each selected adjustment action after the last iteration update; wherein the adjustment action includes the value to be adjusted of the adjustment parameter;

[0041] An adjustment module, configured to select a maximum value from the Q values ​​updated in the last iteration corresponding to each selected adjustment action of the current actual network state, and adjust the antenna according to the adjustment action corresponding to the maximum value;

[0042] An updating module is used to calculate a reward function of the antenna according to the adjusted actual network state of the antenna, update the maximum value based on the reward function based on the Q-learning method, and iterate the adjusted actual network state of the antenna as the current actual network state until the performance indicator of the antenna meets a preset condition.

[0043] An embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned large-scale MIMO antenna parameter adjustment methods are implemented.

[0044] An embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of any of the above-mentioned methods for adjusting the parameters of large-scale MIMO antennas are implemented.

[0045] The large-scale MIMO antenna parameter adjustment method and device provided by the embodiment of the present invention, by filtering out bad adjustment actions based on the actual network status fed back from the previous iteration and relying on expert experience, accelerates the convergence of iterations, reduces the demand for samples, and improves the accuracy of antenna parameter adjustment; the antenna parameters are adjusted in a small-step and gradual iterative manner, which is safe and reliable and ensures the stability of the network; the experience learning of historical adjustments is realized through the Q-table table, and an experience reference is formed in the next action adjustment, which reduces the number of iterations; and the feedback of each adjustment is recorded with the help of the Q-table table, which has logical interpretability and provides a reference for antenna parameter optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 It is a flowchart of a method for adjusting parameters of a large-scale MIMO antenna provided by an embodiment of the present invention;

[0048] Figure 2It is a flowchart of the judgment logic of the expert experience method in a large-scale MIMO antenna parameter adjustment method provided by an embodiment of the present invention;

[0049] Figure 3 It is a structural schematic diagram of a large-scale MIMO antenna parameter adjustment device provided by an embodiment of the present invention;

[0050] Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Combine the following Figure 1 A flow chart of a method for adjusting parameters of a massive MIMO antenna according to an embodiment of the present invention is described, the method comprising: S101, determining an adjustment direction of an adjustment parameter of the antenna based on an expert experience method according to a current actual network state of the antenna; wherein the current actual network state includes a value of a network indicator of the antenna;

[0053] Among them, the antenna in this embodiment is a large-scale MIMO antenna, and the current actual network status of the antenna includes the values ​​of network indicators of one or more antennas, such as the network coverage rate and network overlap coverage of the antenna. The value of the network indicator can be obtained based on KPI (Key Performance Indicators) data and MR data. The current actual network status of the antenna is compared with the preset threshold value through the expert experience method, and the adjustment direction of each or part of the adjustment parameters of the antenna is determined according to the comparison result. The adjustment parameters of the antenna include beam width, azimuth and downtilt angle in the horizontal and vertical directions.

[0054] S102, selecting an adjustment action from the Q-table of the antenna according to the adjustment direction, and searching the Q-table for the Q value of the current actual network state corresponding to each selected adjustment action after the last iteration update; wherein the adjustment action includes the value to be adjusted of the adjustment parameter;

[0055] The Q-table stores multiple network states, multiple adjustment actions, and the Q value corresponding to the combination of any network state and any adjustment action. Each row in the Q-table corresponds to a network state, each column corresponds to an adjustment action, and each value corresponds to a network state and an adjustment action. If the antenna adjustment parameters are adjusted for the first time, the initial value of the Q value is set to 0. The Q value reflects the network performance of the antenna after adjustment, and the Q value in the Q-table is updated at each iteration.

[0056] This embodiment determines the adjustment range of the adjustment parameter according to the adjustment direction of the adjustment parameter and the current value of the adjustment parameter. For example, if the adjustment direction of the downtilt angle is downward, the adjustment range of the downtilt angle is less than the current downtilt angle of the antenna. According to the adjustment range of the adjustment parameter, an adjustment action that satisfies the adjustment parameter range is selected from the Q-table of the antenna.

[0057] According to the actual current network status S of the antenna t You can determine the row in the Q-table, that is, S t Corresponding to the adjustment action set A t {a1,a2,…,a n}, where t is the sequence index of the antenna adjustment parameters and n is the number of all adjustment actions in the Q-table. The direction of the antenna adjustment action is filtered by the expert experience method, and the selected adjustment action set is A f t {a f1 ,a f2 ,…,a fm}. Where f is the action label filtered by the expert experience method, m is the number of selected adjustment actions, and m is less than n. Find S from the Q-table t Corresponding to A f t {a f1 ,a f2 ,…,a fm}. It can be seen that filtering greatly reduces the amount of data and requires fewer training samples.

[0058] S103, selecting a maximum value from the Q values ​​updated in the last iteration corresponding to each selected adjustment action of the current actual network state, and adjusting the antenna according to the adjustment action corresponding to the maximum value;

[0059] From S t Corresponding to A f t {a f1 ,a f2 ,…,a fm}. The present embodiment is not limited to the method for selecting the maximum value. The adjustment action corresponding to the maximum value is output as the optimal adjustment action, and the optimal adjustment action is executed to adjust the adjustment parameters of the antenna. Thus, the shape of the antenna beam is optimized to meet the requirements of dynamic balanced optimization of indicators such as network capacity, coverage, and interference, so as to improve the spectrum efficiency of the wireless environment, achieve the purpose of stimulating traffic, and realize the operator's expectations of incremental growth and increased revenue.

[0060] S104, calculating the reward function of the antenna according to the adjusted actual network state of the antenna, updating the maximum value based on the reward function based on the Q-learning method, and iterating the adjusted actual network state of the antenna as the current actual network state until the performance indicator of the antenna meets the preset conditions.

[0061] After the optimal adjustment action is executed for one cycle, the actual network state S of the antenna is re-extracted t+1 The reward function of the antenna is calculated according to the actual network status of the antenna. The reward function of the antenna reflects the impact of the execution of the optimal adjustment action on the network status of the antenna. The Q-learning method is a temporal difference method of reinforcement learning. In this embodiment, the reward function of the antenna is substituted into the Q-learning method to calculate the S t The Q value corresponding to the optimal adjustment action is updated. t+1 As the current actual network state, the parameters of the antenna are continued to be adjusted until the performance indicators of the antenna meet the preset conditions, such as the utilization rate of PRB (Physical Resource Block) is less than 50%. The performance indicators of the antenna include uplink and downlink PRB utilization, number of RRC (Radio Resource Control) connections, uplink and downlink traffic, downlink CCE (Control Channel Element) utilization, switching success rate, drop rate, multi-user pairing success rate, RRC (Radio Resource Control) establishment success rate, PRB interference noise, CQI (Channel Quality Indication) mean, etc.

[0062] This embodiment uses the actual network status fed back from the previous iteration and expert experience to filter out bad adjustment actions, thereby accelerating the convergence of iterations, reducing the demand for samples, and improving the accuracy of antenna parameter adjustment. The antenna parameters are adjusted in a small-step and gradual iterative manner, which is safe and reliable and ensures the stability of the network. The Q-table is used to implement experience learning of historical adjustments, forming an experience reference for the next action adjustment, reducing the number of iterations, and the Q-table is used to record the feedback of each adjustment, which has logical interpretability and provides a reference for antenna parameter optimization.

[0063] On the basis of the above-mentioned embodiment, the present embodiment further includes the step of determining the adjustment direction of the adjustment parameters of the antenna based on the expert experience method according to the current actual network status of the antenna before: rasterizing the area covered by the antenna; calculating the current actual network status of the antenna in each grid according to the current MR data and KPI data of each grid; calculating the average value of the current actual network status of the antenna in all grids, and using the average value as the current actual network status of the antenna.

[0064] Specifically, this embodiment rasterizes the area covered by the antenna based on the electronic map, such as rasterizing it into a geographic grid of 30*30 meters. The grid identification, building height, and whether it is covered by water of each grid are saved in the geographic grid feature table. According to the MR data and KPI data of each grid, and in combination with the building height and whether it is covered by water in each grid, the current actual network status of the antenna in each grid is calculated. This embodiment is not limited to a specific calculation method. The average value of the current actual network status of all grids in the antenna coverage area is used as the current actual network status of the antenna.

[0065] On the basis of the above embodiments, the network indicators of the antenna in this embodiment include network coverage, network overlap coverage and resource utilization; accordingly, according to the current actual network status of the antenna, the step of determining the adjustment direction of the adjustment parameters of the antenna based on the expert experience method includes: if the resource utilization is greater than the first preset threshold, the bandwidth in the adjustment parameter is shrunk or the downtilt angle is lowered; if the resource utilization is less than the second preset threshold, it is determined whether the network coverage is less than the third preset threshold; if the network coverage is less than the third preset threshold, the bandwidth in the adjustment parameter is shrunk or the downtilt angle is lowered; if the network coverage is greater than the fourth preset threshold, it is determined whether the network overlap coverage is greater than the fifth preset threshold; if the network overlap coverage is greater than the fifth preset threshold, the bandwidth in the adjustment parameter is shrunk or the downtilt angle is lowered; if the network overlap coverage is less than the sixth preset threshold, the bandwidth in the adjustment parameter is expanded or the downtilt angle is raised.

[0066] Among them, the first preset threshold is greater than the second preset threshold, the third preset threshold is less than the fourth preset threshold, and the fifth preset threshold is greater than the sixth preset threshold. The expert experience method adopts the decision tree judgment logic representation, and the judgment logic flow chart of the expert experience method is as follows Figure 2 As shown. Among them, the judgment node, that is, the non-leaf node can be increased, and the leaf node determines the adjustment direction of the antenna. For example, if the resource utilization of the MM antenna is less than 30%, the network coverage is greater than 95%, and the network overlap coverage is less than 5%, the downtilt angle of the antenna is raised to absorb more services.

[0067] On the basis of the above embodiments, before the step of searching the Q-table table for the Q value of the last iteration updated for each selected adjustment action corresponding to the current actual network state, the present embodiment further includes: determining the value of each network indicator according to the preset range and preset value step of each network indicator of the antenna; selecting any one value from the values ​​of each network indicator for combination to generate the network state of the antenna;

[0068] Suppose the network state set S = {s|s = (wc, oc, prb)} of the antenna, where wc∈(0.70, 0.99) represents the network coverage of the antenna; oc∈(0.0, 0.50) represents the network overlap coverage of the antenna; prb∈(0.30, 0.75) represents the resource utilization of the antenna. Since the three network indicators wc, oc and prb are all continuous variables, their step sizes can be defined as 0.05. If the preset range is defined, an alarm is output directly. The preset range of each network indicator can be determined according to the specification parameters of the antenna. For example, there are 6 values ​​for wc, 9 values ​​for oc, and 8 values ​​for prb. The number of combinations obtained by selecting any one value from the values ​​of each network indicator is 6*8*9=432, that is, there are 432 generated network states.

[0069] Determine the value to be adjusted of each adjustment parameter according to the preset range and the preset value step of each adjustment parameter of the antenna; select any one value from the values ​​to be adjusted of each adjustment parameter to combine and generate an adjustment action of the antenna;

[0070] The antenna adjustment action set A = {a|a = (VBW, HBW, Az, T)}, where VBW is the horizontal beam width to be adjusted, such as the preset value step size is 3 degrees; HBW is the vertical beam width to be adjusted, such as the preset value step size is 3 degrees; Az is the azimuth angle to be adjusted, such as the preset value step size is 5 degrees; T is the downtilt angle to be adjusted, such as the preset value step size is 3 degrees. The preset range of each adjustment parameter is not only limited by the antenna performance of the equipment manufacturer, but also limited by the mutual influence between the adjustment parameters.

[0071] The Q-table is constructed according to the generated network state and adjustment action of the antenna; wherein each row of the Q-table represents the Q value of any network state corresponding to each adjustment action, and each column of the Q-table represents the Q value of any adjustment action corresponding to each network state.

[0072] Among them, the column direction of the Q-table is the elements in the antenna's network state set S, and the row direction is the elements in the antenna's adjustment action set A. The number of units in the Q-table is the product of the number of elements in the antenna's network state set S and the number of elements in the antenna's adjustment action set A. After the Q-table is constructed, the Q value of each unit is initialized to 0. The schematic diagram of the Q-table is shown in Table 1. For the convenience of representation, its network state selects two network indicators: network coverage and network overlap coverage. Among them, H represents the horizontal beam width, V represents the vertical beam width, T represents the downtilt angle, A represents the azimuth angle, C represents the network coverage rate, and O represents the network overlap coverage. The numbers after the letters represent the numbers, for example, A1-A6 represent 6 different azimuth values.

[0073] Table 1 Q-table

[0074]

[0075] Based on the above embodiments, the step of selecting the maximum value from the Q value after the last iterative update of the current actual network state corresponding to each selected adjustment action in this embodiment includes: selecting the maximum value from the Q value after the last iterative update of the current actual network state corresponding to each selected adjustment action based on the ε-greedy algorithm.

[0076] Specifically, this embodiment defines strategy π according to the ε-greedy method, and the formula is as follows:

[0077]

[0078] Where m is the number of all possible adjustment actions a filtered by the expert experience method under the current state s. ε defaults to 1e-4, which is the probability of exploratory learning. t Select the optimal adjustment action A t and execute.

[0079] On the basis of the above embodiments, in this embodiment, the reward function of the antenna is calculated according to the actual network status of the antenna after adjustment by the following formula:

[0080] R = x*wc+y*oc+z*prb;

[0081] Among them, R is the reward function, x, y and z are weight coefficients, wc is the network coverage rate in the actual network state of the antenna after adjustment, oc is the network overlap coverage rate in the actual network state of the antenna after adjustment, and prb is the resource utilization rate in the actual network state of the antenna after adjustment.

[0082] Specifically, according to the optimal adjustment action A t After adjusting the antenna parameters, the antenna network status S is calculated based on the latest MR data and KPI data fed back. t+1 According to S t+1 The wc, oc and prb in the calculation result are used to calculate the value R of the reward function. According to experience, the weight coefficients of the three network indicators wc, oc and prb can be set to x = 0.3, y = 0.3, and z = 0.4.

[0083] Based on the above embodiments, this embodiment updates the maximum value according to the reward function based on the Q-learning method through the following formula:

[0084]

[0085] Among them, Q′(S t ,A t ) indicates S t Corresponding to A t The updated Q value, S t represents the actual network status of the antenna after the adjustment parameter is adjusted for the tth time, A t represents the adjustment action corresponding to the maximum value obtained after the adjustment parameter is adjusted for the tth time, Q(S t ,A t )S t Corresponding to A t The Q value before the update, α is the learning rate, which can be set to 0.1, R is the reward function, γ is the discount factor, which can be set to 0.8, Indicates S t+1 Corresponding to S t+1 The maximum value among the Q values ​​of the selected adjustment action a.

[0086] The following describes a massive MIMO antenna parameter adjustment device provided by an embodiment of the present invention. The massive MIMO antenna parameter adjustment device described below and the massive MIMO antenna parameter adjustment method described above can refer to each other.

[0087] like Figure 3 As shown, the device includes a determination module 301, a search module 302, an adjustment module 303 and an update module 304, wherein:

[0088] The determination module 301 is used to determine the adjustment direction of the adjustment parameter of the antenna based on the expert experience method according to the current actual network state of the antenna; wherein the current actual network state includes the value of the network indicator of the antenna;

[0089] Among them, the antenna in this embodiment is a massive MIMO antenna, and the current actual network state of the antenna includes the value of one or more network indicators of the antenna, such as the network coverage rate and network overlap coverage of the antenna. The value of the network indicator can be obtained based on the KPI data and the MR data. The current actual network state of the antenna is compared with the preset threshold value through the expert experience method, and the adjustment direction of each or part of the adjustment parameters of the antenna is determined according to the comparison result.

[0090] The search module 302 is used to select an adjustment action from the Q-table of the antenna according to the adjustment direction, and search the Q value of the last iteration update corresponding to each selected adjustment action of the current actual network state from the Q-table; wherein the adjustment action includes the value to be adjusted of the adjustment parameter;

[0091] According to the adjustment direction of the adjustment parameter and the current value of the adjustment parameter, the adjustment range of the adjustment parameter is determined. According to the adjustment range of the adjustment parameter, an adjustment action that satisfies the adjustment parameter range is selected from the Q-table of the antenna. The latest Q value corresponding to each selected adjustment action of the current actual network state is found from the Q-table.

[0092] The adjustment module 303 is used to select a maximum value from the Q values ​​updated in the last iteration corresponding to each selected adjustment action of the current actual network state, and adjust the antenna according to the adjustment action corresponding to the maximum value;

[0093] The maximum value is selected from the current Q value corresponding to each selected adjustment action of the current actual network state. This embodiment is not limited to the method for selecting the maximum value. The adjustment action corresponding to the maximum value is output as the optimal adjustment action, and the optimal adjustment action is executed to adjust the adjustment parameters of the antenna. Thereby, the shape of the antenna beam is optimized to meet the requirements of dynamic balanced optimization of indicators such as network capacity, coverage, and interference, so as to improve the spectrum efficiency of the wireless environment, achieve the purpose of stimulating traffic, and realize the operator's expectations of incremental growth and increased revenue.

[0094] The update module 304 is used to calculate the reward function of the antenna according to the adjusted actual network state of the antenna, update the maximum value based on the Q-learning method according to the reward function, and iterate the adjusted actual network state of the antenna as the current actual network state until the performance indicator of the antenna meets the preset conditions.

[0095] After the optimal adjustment action is executed for one cycle, the actual network state of the antenna is re-extracted. The reward function of the antenna is calculated based on the actual network state of the antenna. The reward function of the antenna reflects the impact of the execution of the optimal adjustment action on the network state of the antenna. The Q-learning method is a temporal difference method of reinforcement learning. In this embodiment, the reward function of the antenna is substituted into the Q-learning method to update the Q value of the optimal adjustment action until the performance index of the antenna meets the preset conditions.

[0096] This embodiment uses the actual network status fed back from the previous iteration and expert experience to filter out bad adjustment actions, thereby accelerating the convergence of iterations, reducing the demand for samples, and improving the accuracy of antenna parameter adjustment. The antenna parameters are adjusted in a small-step and gradual iterative manner, which is safe and reliable and ensures the stability of the network. The Q-table is used to implement experience learning of historical adjustments, forming an experience reference for the next action adjustment, reducing the number of iterations, and the Q-table is used to record the feedback of each adjustment, which has logical interpretability and provides a reference for antenna parameter optimization.

[0097] Based on the above embodiment, the present implementation also includes a rasterization module, which is used to rasterize the area covered by the antenna; calculate the current actual network status of the antenna in each grid based on the current MR data and KPI data of each grid; calculate the average value of the current actual network status of the antenna in all grids, and use the average value as the current actual network status of the antenna.

[0098] On the basis of the above embodiments, the network indicators of the antenna in this embodiment include network coverage, network overlapping coverage and resource utilization; accordingly, the determination module is specifically used to: if the resource utilization is greater than the first preset threshold, the bandwidth in the adjustment parameter is shrunk or the downtilt angle is lowered; if the resource utilization is less than the second preset threshold, it is determined whether the network coverage is less than the third preset threshold; if the network coverage is less than the third preset threshold, the bandwidth in the adjustment parameter is shrunk or the downtilt angle is lowered; if the network coverage is greater than the fourth preset threshold, it is determined whether the network overlapping coverage is greater than the fifth preset threshold; if the network overlapping coverage is greater than the fifth preset threshold, the bandwidth in the adjustment parameter is shrunk or the downtilt angle is lowered; if the network overlapping coverage is less than the sixth preset threshold, the bandwidth in the adjustment parameter is expanded or the downtilt angle is raised.

[0099] On the basis of the above embodiments, the present embodiment also includes a construction module, which is used to determine the value of each network indicator according to the preset range and preset value step of each network indicator of the antenna; select any one value from the values ​​of each network indicator for combination to generate the network state of the antenna; determine the value to be adjusted of each adjustment parameter according to the preset range and preset value step of each adjustment parameter of the antenna; select any one value from the values ​​to be adjusted of each adjustment parameter for combination to generate the adjustment action of the antenna; construct the Q-table according to the generated network state and adjustment action of the antenna; wherein each row of the Q-table represents the Q value of each adjustment action corresponding to any network state, and each column of the Q-table represents the Q value of each adjustment action corresponding to any network state.

[0100] On the basis of the above embodiments, the adjustment module in this embodiment is specifically used to: select the maximum value from the Q value updated in the last iteration corresponding to each selected adjustment action of the current actual network state based on the ε-greedy algorithm.

[0101] On the basis of the above embodiments, the updating module in this embodiment is specifically used to: calculate the reward function of the antenna according to the actual network state of the antenna after adjustment by the following formula:

[0102] R = x*wc+y*oc+z*prb;

[0103] Among them, R is the reward function, x, y and z are weight coefficients, wc is the network coverage rate in the actual network state of the antenna after adjustment, oc is the network overlap coverage rate in the actual network state of the antenna after adjustment, and prb is the resource utilization rate in the actual network state of the antenna after adjustment.

[0104] On the basis of the above embodiments, the updating module in this embodiment is specifically used to update the maximum value according to the reward function based on the Q-learning method through the following formula:

[0105]

[0106] Among them, Q′(S t ,A t ) indicates S t Corresponding to A t The updated Q value, S t represents the actual network status of the antenna after the adjustment parameter is adjusted for the tth time, A t represents the adjustment action corresponding to the maximum value obtained after the adjustment parameter is adjusted for the tth time, Q(S t ,A t )St Corresponding to A t The Q value before the update, α is the learning rate, R is the reward function, γ is the discount factor, Indicates S t+1 Corresponding to S t+1 The maximum value among the Q values ​​of the selected adjustment action a.

[0107] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the large-scale MIMO antenna parameter adjustment method, which includes: determining the adjustment direction of the adjustment parameters of the antenna based on the expert experience method according to the current actual network state of the antenna; selecting an adjustment action from the Q-table table according to the adjustment direction, and searching the Q-table table for the current actual network state corresponding to each selected adjustment action; selecting the maximum value from the current Q value found, and adjusting the antenna according to the adjustment action corresponding to the maximum value; calculating the reward function of the antenna according to the actual network state of the adjusted antenna, and updating the maximum value based on the Q-learning method according to the reward function until the performance index of the antenna meets the preset conditions.

[0108] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0109] On the other hand, an embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the large-scale MIMO antenna parameter adjustment method provided by the above-mentioned method embodiments, the method including: determining the adjustment direction of the adjustment parameters of the antenna based on an expert experience method according to the current actual network status of the antenna; selecting an adjustment action from a Q-table according to the adjustment direction, and searching the Q-table for the current Q value corresponding to each selected adjustment action of the current actual network status; selecting a maximum value from the current Q values ​​found, and adjusting the antenna according to the adjustment action corresponding to the maximum value; calculating the reward function of the antenna according to the actual network status of the adjusted antenna, and updating the maximum value based on the Q-learning method according to the reward function until the performance index of the antenna meets the preset conditions.

[0110] On the other hand, an embodiment of the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the large-scale MIMO antenna parameter adjustment method provided in the above-mentioned embodiments, the method comprising: determining an adjustment direction of the adjustment parameters of the antenna based on an expert experience method according to the current actual network status of the antenna; selecting an adjustment action from a Q-table according to the adjustment direction, and searching the Q-table for a current Q value corresponding to each selected adjustment action of the current actual network status; selecting a maximum value from the current Q values ​​found, and adjusting the antenna according to the adjustment action corresponding to the maximum value; calculating a reward function of the antenna according to the actual network status of the adjusted antenna, and updating the maximum value based on the Q-learning method according to the reward function until the performance index of the antenna meets a preset condition.

[0111] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for adjusting parameters of a large-scale MIMO antenna, characterized in that: include: According to the current actual network state of the antenna, determining the adjustment direction of the adjustment parameter of the antenna based on the expert experience method; wherein the current actual network state includes the value of the network indicator of the antenna; Selecting an adjustment action from the Q-table of the antenna according to the adjustment direction, and searching the Q-table for the Q value of the last iteration update corresponding to each selected adjustment action of the current actual network state; wherein the adjustment action includes the value to be adjusted of the adjustment parameter; Selecting a maximum value from the Q values ​​updated in the last iteration corresponding to each selected adjustment action of the current actual network state, and adjusting the antenna according to the adjustment action corresponding to the maximum value; The reward function of the antenna is calculated according to the adjusted actual network state of the antenna, the maximum value is updated based on the Q-learning method according to the reward function, and the adjusted actual network state of the antenna is iterated as the current actual network state until the performance indicator of the antenna meets the preset conditions.

2. The method for adjusting parameters of a massive MIMO antenna according to claim 1, wherein: Before the step of determining the adjustment direction of the adjustment parameter of the antenna based on the expert experience method according to the current actual network state of the antenna, the method further includes: Gridding the area covered by the antenna; Calculate the current actual network status of the antenna in each grid according to the current MR data and KPI data of each grid; An average value of the current actual network status of the antenna in all grids is calculated, and the average value is used as the current actual network status of the antenna.

3. The method for adjusting parameters of a massive MIMO antenna according to claim 1, wherein: The network indicators of the antenna include network coverage, network overlap coverage and resource utilization; Accordingly, according to the current actual network state of the antenna, the step of determining the adjustment direction of the adjustment parameter of the antenna based on the expert experience method includes: If the resource utilization rate is greater than a first preset threshold, the beam width in the adjustment parameter is reduced or the downtilt angle is reduced; If the resource utilization rate is less than the second preset threshold, determining whether the network coverage rate is less than a third preset threshold; If the network coverage is less than a third preset threshold, shrinking the beam width or pressing down the downtilt angle in the adjustment parameter; If the network coverage is greater than the fourth preset threshold, determining whether the network overlap coverage is greater than the fifth preset threshold; If the network overlap coverage is greater than a fifth preset threshold, shrinking the beam width or pressing down the downtilt angle in the adjustment parameter; If the network overlap coverage is less than a sixth preset threshold, the bandwidth in the adjustment parameter is expanded or the downtilt angle is increased.

4. The method for adjusting parameters of a massive MIMO antenna according to any one of claims 1 to 3, characterized in that: Before the step of searching the Q-table table for the Q value of the current actual network state corresponding to the last iterative update of each selected adjustment action, the step further includes: Determining a value of each network indicator according to a preset range and a preset value step of each network indicator of the antenna; Select any one value from the values ​​of each network indicator to combine and generate the network state of the antenna; Determining a value to be adjusted for each adjustment parameter according to a preset range and a preset value step of each adjustment parameter of the antenna; Selecting any one value from the values ​​to be adjusted of each adjustment parameter for combination to generate an adjustment action for the antenna; Construct the Q-table according to the generated network status and adjustment action of the antenna; Each row of the Q-table represents the Q value of any network state corresponding to each adjustment action, and each column of the Q-table represents the Q value of any adjustment action corresponding to each network state.

5. The method for adjusting parameters of a massive MIMO antenna according to any one of claims 1 to 3, characterized in that: The step of selecting the maximum value from the Q value updated in the last iteration corresponding to each selected adjustment action of the current actual network state comprises: Based on the ε-greedy algorithm, a maximum value is selected from the Q value updated in the last iteration corresponding to each selected adjustment action of the current actual network state.

6. The method for adjusting parameters of a massive MIMO antenna according to any one of claims 1 to 3, characterized in that: The reward function of the antenna is calculated according to the actual network status of the antenna after adjustment by the following formula: R = x*wc+y*oc+z*prb; Among them, R is the reward function, x, y and z are weight coefficients, wc is the network coverage rate in the actual network state of the antenna after adjustment, oc is the network overlap coverage rate in the actual network state of the antenna after adjustment, and prb is the resource utilization rate in the actual network state of the antenna after adjustment.

7. The method for adjusting parameters of a massive MIMO antenna according to any one of claims 1 to 3, characterized in that: The maximum value is updated based on the Q-learning method according to the reward function using the following formula: Among them, Q′(S t ,A t ) indicates S t Corresponding to A t The updated Q value, S t represents the actual network status of the antenna after the adjustment parameter is adjusted for the tth time, A t represents the adjustment action corresponding to the maximum value obtained after the adjustment parameter is adjusted for the tth time, Q(S t ,A t )S t Corresponding to A t The Q value before the update, α is the learning rate, R is the reward function, γ is the discount factor, Indicates S t+1 Corresponding to S t+1 The maximum value among the Q values ​​of the selected adjustment action a.

8. A large-scale MIMO antenna parameter adjustment device, characterized in that: include: A determination module, configured to determine an adjustment direction of an adjustment parameter of the antenna based on an expert experience method according to a current actual network state of the antenna; wherein the current actual network state includes a value of a network indicator of the antenna; A search module, configured to select an adjustment action from a Q-table of the antenna according to the adjustment direction, and search the Q-table for the Q value of the current actual network state corresponding to each selected adjustment action after the last iteration update; wherein the adjustment action includes the value to be adjusted of the adjustment parameter; An adjustment module, configured to select a maximum value from the Q values ​​updated in the last iteration corresponding to each selected adjustment action of the current actual network state, and adjust the antenna according to the adjustment action corresponding to the maximum value; An updating module is used to calculate a reward function of the antenna according to the adjusted actual network state of the antenna, update the maximum value based on the reward function based on the Q-learning method, and iterate the adjusted actual network state of the antenna as the current actual network state until the performance indicator of the antenna meets a preset condition.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the large-scale MIMO antenna parameter adjustment method according to any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for adjusting parameters of a large-scale MIMO antenna are implemented as described in any one of claims 1 to 7.

Images