Communication operator network optimization method and device and readable storage medium

Abstract

The invention provides a communication operator network optimization method and device and a readable storage medium, and relates to the technical field of networks, and the method comprises the steps: obtaining at least one region which needs to be expanded in an application scene; calculating the priority of each region according to the current people flow density, the predicted user number, the current network load, the user experience score and the service type score of each region in the at least one region needing to be expanded; based on the sequence of the priorities, path planning is conducted on at least one area needing to be expanded through an A star algorithm, so that the optimal path of movement of the signal optimization trolley is obtained; and in the process that the signal optimization trolley travels according to the optimal path, a reinforcement learning RL method is adopted to determine whether to continue capacity expansion or leave the scene in the current area. According to the method, the device and the readable storage medium, the problems that an existing communication operator network optimization method is insufficient in flexibility, low in efficiency and difficult to cope with human traffic changes in a scene, and consequently capacity expansion is inaccurate and not timely can be solved.

Description

Technical Field

[0001] The present invention relates to the field of network technology, and in particular to a communication operator network optimization method, device and readable storage medium. Background Art

[0002] With the advancement of network technology, the network has been deeply integrated into people's daily lives. However, when using WLAN networks in public places, users often have the following concerns: the security of completely open public WLAN networks is worrying because data transmission is easily stolen; public WLAN networks that require mobile phone number verification have the risk of leaking personal information such as mobile phone numbers and are difficult to trust; private WLAN networks are inconvenient because they require passwords, and their security is not completely reliable. Therefore, the networks of communication operators are still the first choice for most users in public places.

[0003] In scenes such as cinemas, theaters, or shopping malls with uneven crowd density, the network of telecommunications operators may experience extreme situations of overload or underload. Specifically, in areas with less traffic, the network load is normal and the user experience is good; in areas with concentrated traffic, the network load is too large, which is prone to problems such as video freezes, network congestion, and dropped volte calls, resulting in a poor user experience.

[0004] The existing network optimization methods of communication operators mainly install fixed indoor distributed base stations in high-traffic areas to increase capacity. However, once installed, such base stations cannot be moved, resulting in idle network equipment and waste of resources. In addition, these methods are single-means, inflexible, and inefficient. They are difficult to cope with changes in traffic within the scene, resulting in inaccurate and untimely capacity expansion, which affects the user's network experience. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a communication operator network optimization method, device and readable storage medium in response to the above-mentioned deficiencies in the prior art, so as to solve the problem that the existing communication operator network optimization method mainly installs fixed indoor distributed base stations in high-density areas to increase capacity, which has the disadvantages of insufficient flexibility, low efficiency, and difficulty in coping with changes in the flow of people in the scene, resulting in inaccurate and untimely capacity expansion, affecting the user's network experience.

[0006] In a first aspect, the present invention provides a communication operator network optimization method, which is applied to a signal optimization vehicle, and the method comprises:

[0007] Obtain at least one area that needs to be expanded in the application scenario;

[0008] Calculate the priority of each area according to the current traffic density, predicted number of users, current network load, user experience score and service type score of each area of ​​at least one area requiring capacity expansion;

[0009] Based on the order of the priorities, the A-star algorithm is used to perform path planning for at least one area that needs to be expanded, so as to obtain an optimal path for the signal optimization vehicle to move;

[0010] When the signal optimization vehicle is traveling along the optimal path, a reinforcement learning (RL) method is used to decide whether to continue capacity expansion or leave the current area.

[0011] Furthermore, the obtaining of at least one area that needs to be expanded in the application scenario specifically includes:

[0012] Get the area where expansion needs to be started in the application scenario;

[0013] Obtaining predicted physical resource block PRB utilization and predicted downlink traffic in the next preset time period for the area where capacity expansion needs to be started;

[0014] The area where capacity expansion needs to be started and whose predicted PRB utilization is greater than the first capacity expansion threshold and whose predicted downlink traffic is greater than the second capacity expansion threshold is determined as the area where capacity expansion needs to be started, and at least one area where capacity expansion needs to be started is obtained.

[0015] Furthermore, the obtaining of the area in the application scenario where expansion needs to be initiated specifically includes:

[0016] Obtain historical crowd density data of multiple areas in the application scenario from the crowd density information database of the monitoring device;

[0017] Analyze the historical crowd density data of each area through the time series model to predict the crowd density data at future moments;

[0018] Selecting an area where the predicted crowd density data exceeds a crowd flow threshold from a plurality of areas;

[0019] The relevant capacity index of the selected area at the current moment is obtained, and the area where capacity expansion needs to be started is determined according to the relevant capacity index.

[0020] Further, obtaining the predicted physical resource block PRB utilization and predicted downlink traffic in the next preset time period for the area where the capacity expansion needs to be started specifically includes:

[0021] For the area where capacity expansion needs to be started, the current number of users, current PRB utilization, current downlink traffic, and current proportion of video service users are collected;

[0022] Obtaining a predicted PRB utilization rate for the next preset time period based on the current number of users, the current PRB utilization rate, and the current video service user ratio;

[0023] The predicted downlink traffic in the next preset time period is obtained based on the current number of users and the current downlink traffic.

[0024] Further, the obtaining the predicted PRB utilization rate for the next preset time period based on the current number of users, the current PRB utilization rate and the current video service user ratio specifically includes:

[0025] Calculating a user growth rate based on the current number of users and the predicted number of users in the next preset time period;

[0026] Calculating the current number of video service users and the predicted number of video service users according to the current video service user ratio;

[0027] Define the service type influencing factor corresponding to the video service according to the demand of the video service for PRB;

[0028] Calculate the average number of PRBs required per user according to the current PRB utilization, the current number of users and the total number of PRBs;

[0029] Calculate the number of PRBs required by video service users according to the average number of PRBs required per user and the service type impact factor corresponding to the video service;

[0030] Calculate the number of PRBs required by other service users according to the average number of PRBs required by each user;

[0031] The predicted PRB utilization is calculated according to the predicted number of video service users, the number of PRBs required by video service users, the predicted number of other service users, and the number of PRBs required by other service users.

[0032] Further, the obtaining the predicted downlink traffic in the next preset time period based on the current number of users and the current downlink traffic specifically includes:

[0033] Calculate the average traffic demand per user according to the current downlink traffic and the current number of users;

[0034] Define the traffic demand influencing factor corresponding to the video service according to the traffic demand of the video service;

[0035] Calculate the traffic demand of video service users according to the average traffic demand of each user and the traffic demand influencing factor corresponding to the video service;

[0036] Calculate the traffic demand of other service users according to the average traffic demand of each user;

[0037] The predicted downlink traffic is calculated according to the predicted number of video service users, the traffic demand number of video service users, the predicted number of other service users, and the traffic demand number of other service users.

[0038] Furthermore, the priority of each area is calculated according to the current flow density, predicted number of users, current network load, user experience score and service type score of each area in the at least one area that needs to be expanded, and is specifically calculated according to the following formula:

[0039]

[0040] Where P is the priority, W1, W2, W3, W4, and W5 are weight coefficients, D is the current flow density in the current area, and D SUM is the current total flow density of at least one area that needs to be expanded, N is the predicted number of users in the current area, and N SUM is the predicted total number of users in at least one area that needs to be expanded, L is the current network load in the current area, E is the user experience score, and T is the service type score.

[0041] Furthermore, before calculating the priority of each area according to the current flow density, predicted number of users, current network load, user experience score and service type score of each area in the at least one area requiring capacity expansion, the method further includes:

[0042] Obtaining the network delay, network packet loss rate and network jitter of each of the areas within a preset time period in the past;

[0043] Perform a comprehensive weighted calculation on the network delay, network packet loss rate and network jitter in each of the areas within a preset time period in the past to obtain a user experience score corresponding to each of the areas;

[0044] Obtain scores corresponding to different types of services in each of the areas in a preset time period in the past, where the different types of services include at least two of the following: video streaming, voice calls, online games, web browsing, emails, and file downloads / uploads;

[0045] A comprehensive weighted calculation is performed on the scores corresponding to different types of businesses in each of the regions within a preset time period in the past to obtain a business type score corresponding to each of the regions.

[0046] Furthermore, the signal optimization vehicle uses a reinforcement learning (RL) method to decide whether to continue capacity expansion or leave the current area during the driving process according to the optimal path, specifically including:

[0047] Initialize the Q value table or neural network;

[0048] Obtain the current status S = {D, U, E} in real time, where D represents the current traffic density, U represents the current PRB utilization rate, and E represents the user experience score;

[0049] Select the optimal action A according to the current state S and the Q value table or the neural network, where action A is to continue to expand capacity or leave the site;

[0050] Execute the selected action A;

[0051] Get the new state S′={D′,U′,E′} and reward R of the environment feedback;

[0052] Update the Q value table or neural network parameters according to the new state S′ and reward R;

[0053] Based on the new state S′ and the preset stay / leave judgment condition, it is determined whether to continue capacity expansion or leave the site.

[0054] Furthermore, the stay / leave judgment condition includes:

[0055] When the current traffic density is greater than the first threshold, or the current PRB utilization is greater than the second threshold, or the user experience score is greater than the third threshold, and continues for a period of time T, take the action of continuing to expand capacity; and,

[0056] When the current crowd density is less than the fifth threshold, the current PRB utilization is less than the sixth threshold, and the user experience score is less than or equal to the third threshold, and the situation lasts for a period of time T, take the action of leaving the site;

[0057] Among them, the first threshold is greater than the fifth threshold, and the second threshold is greater than the sixth threshold.

[0058] In a second aspect, the present invention provides a communication operator network optimization device, which is arranged in a signal optimization vehicle, and the device comprises:

[0059] An expansion area acquisition module is used to acquire at least one area that needs to be expanded in an application scenario;

[0060] a priority calculation module, connected to the expansion area acquisition module, for calculating the priority of each area according to the current flow density, predicted number of users, current network load, user experience score and service type score of each area of ​​the at least one area requiring expansion;

[0061] A trolley path planning module, connected to the priority calculation module, is used to perform path planning for at least one area that needs to be expanded based on the order of the priorities using the A-star algorithm to obtain an optimal path for the movement of the signal optimization trolley;

[0062] The car network optimization module is connected to the car path planning module and is used to use the reinforcement learning RL method to decide whether to continue to expand capacity or leave the site in the current area when the signal optimization car is driving according to the optimal path.

[0063] In a third aspect, the present invention provides a communication operator network optimization device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the communication operator network optimization method described in the first aspect above.

[0064] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the communication operator network optimization method described in the first aspect is implemented.

[0065] The present invention provides a communication operator network optimization method, device and readable storage medium. First, obtain at least one area that needs to be expanded in the application scenario; then calculate the priority of each area according to the current flow density, predicted number of users, current network load, user experience score and service type score of each area in the at least one area that needs to be expanded; then use the A star algorithm to plan the path of the at least one area that needs to be expanded based on the order of the priorities to obtain the optimal path for the signal optimization car to move; finally, in the process of the signal optimization car driving according to the optimal path, the reinforcement learning RL method is used to decide whether to continue to expand the current area or leave the scene. The present invention uses a movable signal optimization car to intelligently identify at least one area that needs to be expanded in the application scenario, and perform priority sorting and path planning, so that the signal optimization car can move to densely populated areas in order of priority, dynamically realize network expansion, not only improve flexibility, but also ensure users' good network experience, and reduce user complaints. The solution solves the problem that the existing communication operators' network optimization method is mainly to install fixed indoor distributed base stations in high-traffic areas to increase capacity, which is insufficient in flexibility, low in efficiency, and difficult to cope with changes in traffic within the scene, resulting in inaccurate and untimely capacity expansion, affecting users' network experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a flow chart of a communication operator network optimization method according to Embodiment 1 of the present invention;

[0067] Figure 2 This is a schematic diagram of the structure of a communication operator network optimization device according to Embodiment 2 of the present invention;

[0068] Figure 3 This is a schematic diagram of the structure of a communication operator network optimization device according to Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0069] In order to enable those skilled in the art to better understand the technical solution of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0070] It should be understood that the specific embodiments and drawings described herein are only used to explain the present invention rather than to limit the present invention.

[0071] It can be understood that, in the absence of conflict, the various embodiments of the present invention and the various features in the embodiments can be combined with each other.

[0072] It can be understood that, for the convenience of description, the drawings of the present invention only show the parts related to the present invention, while the parts irrelevant to the present invention are not shown in the drawings.

[0073] It can be understood that the terms "first", "second", etc. in the embodiments of the present invention are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0074] It can be understood that, in the absence of conflict, the functions and steps marked in the flowcharts and block diagrams of the present invention may occur in an order different from that marked in the drawings.

[0075] It is understood that the flowcharts and block diagrams of the present invention illustrate the possible architectures, functions, and operations of the systems, devices, equipment, and methods according to the various embodiments of the present invention. Each box in the flowchart or block diagram may represent a unit, module, program segment, or code, which contains executable instructions for implementing the specified functions. Moreover, each box or combination of boxes in the block diagram and flowchart may be implemented by a hardware-based system that implements the specified functions, or may be implemented by a combination of hardware and computer instructions.

[0076] Embodiment 1:

[0077] This embodiment provides a communication operator network optimization method, which is applied to a signal optimization vehicle, such as Figure 1 As shown, the method includes:

[0078] Step S101: Acquire at least one area that needs to be expanded in an application scenario.

[0079] In this embodiment, the signal optimization vehicle intelligently identifies the density of human traffic and obtains at least one area that needs to be expanded in the application scenario.

[0080] Optionally, obtaining at least one area that needs to be expanded in the application scenario specifically includes:

[0081] Get the area where expansion needs to be started in the application scenario;

[0082] Obtaining predicted PRB (Physical Resource Block) utilization and predicted downlink traffic for the next preset time period for the area where capacity expansion needs to be started;

[0083] The area where capacity expansion needs to be started and whose predicted PRB utilization is greater than the first capacity expansion threshold and whose predicted downlink traffic is greater than the second capacity expansion threshold is determined as the area where capacity expansion needs to be started, and at least one area where capacity expansion needs to be started is obtained.

[0084] In an optional embodiment, obtaining the area in the application scenario where capacity expansion needs to be started specifically includes:

[0085] Obtain historical crowd density data of multiple areas in the application scenario from the crowd density information database of the monitoring device;

[0086] Analyze the historical crowd density data of each area through the time series model to predict the crowd density data at future moments;

[0087] Selecting an area where the predicted crowd density data exceeds a crowd flow threshold from a plurality of areas;

[0088] The relevant capacity index of the selected area at the current moment is obtained, and the area where capacity expansion needs to be started is determined according to the relevant capacity index.

[0089] In this embodiment, the monitoring device monitors the flow of people in the application scenario in real time, and establishes a database of crowd density information at each hourly granularity. The signal optimization car can obtain the crowd density information database of the monitoring device, and then analyze the historical crowd density data of each area to predict future changes in crowd density. The time series model preferably uses the ARIMA model. After the crowd flow prediction, the threshold is set, and the area where the predicted crowd density data exceeds the crowd flow threshold is selected. The signal optimization car extracts the relevant capacity indicators of these areas at the current time through the network management platform, including PRB utilization (that is, the average utilization of downlink PRB) and average downlink traffic. When both reach the high load threshold of the base station, it is determined that the corresponding area needs to start capacity expansion.

[0090] Optionally, obtaining a predicted physical resource block PRB utilization rate and a predicted downlink traffic volume in a next preset time period for the area where capacity expansion needs to be started specifically includes:

[0091] For the area where capacity expansion needs to be started, the current number of users, current PRB utilization, current downlink traffic, and current proportion of video service users are collected;

[0092] Obtaining a predicted PRB utilization rate for the next preset time period based on the current number of users, the current PRB utilization rate, and the current video service user ratio;

[0093] The predicted downlink traffic in the next preset time period is obtained based on the current number of users and the current downlink traffic.

[0094] In this embodiment, after determining the areas where capacity expansion needs to be started, it is then determined whether the base stations in these areas will meet the capacity expansion standards in the next preset time period (for example, one hour), and the predicted PRB utilization and predicted downlink traffic in the next preset time period are first obtained.

[0095] Optionally, the acquiring the predicted PRB utilization rate for the next preset time period based on the current number of users, the current PRB utilization rate, and the current video service user ratio specifically includes:

[0096] Calculating a user growth rate based on the current number of users and the predicted number of users in the next preset time period;

[0097] Calculating the current number of video service users and the predicted number of video service users according to the current video service user ratio;

[0098] Define the service type influencing factor corresponding to the video service according to the demand of the video service for PRB;

[0099] Calculate the average number of PRBs required per user according to the current PRB utilization, the current number of users and the total number of PRBs;

[0100] Calculate the number of PRBs required by video service users according to the average number of PRBs required per user and the service type impact factor corresponding to the video service;

[0101] Calculate the number of PRBs required by other service users according to the average number of PRBs required by each user;

[0102] The predicted PRB utilization is calculated according to the predicted number of video service users, the number of PRBs required by video service users, the predicted number of other service users, and the number of PRBs required by other service users.

[0103] In this embodiment, the user growth rate is calculated according to the following formula:

[0104]

[0105] Among them, U growthrate represents the user growth rate, U predicted Represents the predicted number of users, U current Indicates the current number of users.

[0106] In this embodiment, different services have different requirements for resources, and this requirement is defined as the service type influencing factor F. type For example, the video service has a higher demand for PRB, which is 150% of the ordinary text and voice service. Then the service type influencing factor is: F video prb = 1.5.

[0107] Average number of PRBs required per user = current PRB utilization / current number of users * total number of PRBs;

[0108] Number of PRBs required by video service users = average number of PRBs required per user * F video prb;

[0109] The number of PRBs required by other service users = the average number of PRBs required by each user;

[0110] Predicted PRB utilization rate = (predicted number of video service users * number of PRBs required by video service users + predicted number of other service users * number of PRBs required by other service users) / total number of PRBs.

[0111] Optionally, the acquiring predicted downlink traffic in the next preset time period based on the current number of users and the current downlink traffic specifically includes:

[0112] Calculate the average traffic demand per user according to the current downlink traffic and the current number of users;

[0113] Define the traffic demand influencing factor corresponding to the video service according to the traffic demand of the video service;

[0114] Calculate the traffic demand of video service users according to the average traffic demand of each user and the traffic demand influencing factor corresponding to the video service;

[0115] Calculate the traffic demand of other service users according to the average traffic demand of each user;

[0116] The predicted downlink traffic is calculated according to the predicted number of video service users, the traffic demand number of video service users, the predicted number of other service users, and the traffic demand number of other service users.

[0117] In this embodiment, assuming that the traffic demand of the video service is 200% of that of the common service, the traffic demand influencing factor is: F video traffic = 2, and this value can be flexibly set.

[0118] Average traffic demand per user = current downlink traffic / current number of users;

[0119] Video service user traffic demand = average traffic demand per user * F video traffic;

[0120] Traffic demand of other service users = average traffic demand of each user;

[0121] Predicted downlink traffic = predicted number of video service users * traffic demand of video service users + predicted number of other service users * traffic demand of other service users.

[0122] Step S102: Calculate the priority of each area in the at least one area that needs to be expanded according to the current traffic density, predicted number of users, current network load, user experience score and service type score of each area.

[0123] In this embodiment, the priority of each area is calculated according to the current traffic density, predicted number of users, current network load, user experience score and service type score of each area of ​​the at least one area that needs to be expanded, and is specifically calculated according to the following formula:

[0124]

[0125] Where P is the priority, W1, W2, W3, W4, and W5 are weight coefficients, D is the current flow density in the current area, and D SUM is the current total flow density of at least one area that needs to be expanded, N is the predicted number of users in the current area, and N SUM is the predicted total number of users in at least one area that needs to be expanded, L is the current network load in the current area, E is the user experience score, and T is the service type score.

[0126] Optionally, before calculating the priority of each area according to the current traffic density, predicted number of users, current network load, user experience score and service type score of each area in the at least one area requiring capacity expansion, the method further includes:

[0127] Obtaining the network delay, network packet loss rate and network jitter of each of the areas within a preset time period in the past;

[0128] Perform a comprehensive weighted calculation on the network delay, network packet loss rate and network jitter in each of the areas within a preset time period in the past to obtain a user experience score corresponding to each of the areas;

[0129] Obtain scores corresponding to different types of services in each of the areas in a preset time period in the past, where the different types of services include at least two of the following: video streaming, voice calls, online games, web browsing, emails, and file downloads / uploads;

[0130] A comprehensive weighted calculation is performed on the scores corresponding to different types of businesses in each of the regions within a preset time period in the past to obtain a business type score corresponding to each of the regions.

[0131] In this embodiment, the user experience score (E) is an indicator that reflects the overall satisfaction and experience quality of users using network services in a region. The score can be comprehensively evaluated and calculated based on network delay, network packet loss rate, and network jitter, for example:

[0132] E=W latency *E latency +W packet_loss *E packet_loss +W jitter **E jitter

[0133] In the formula, E latency Indicates network delay, E packet_loss Indicates the network packet loss rate, E jitter Indicates network jitter, W latency , W packet_loss , W jitter is the corresponding weight coefficient.

[0134] In this embodiment, the service type score (T) is a score based on the importance of different service types to network requirements. Different types of services have different requirements for network bandwidth, latency, stability, etc. The score can be comprehensively evaluated and calculated based on video streaming, voice calls, online games, web browsing, email, file download / upload. The higher the score standard, the more urgent the need for capacity expansion, for example:

[0135] T=W streaming * streaming +W voice *T voice +W gaming *T gaming

[0136] +W web *T web +W email *T email +W file *T file

[0137] Among them, T streamin is the rating of the video streaming; T voice is the rating of the voice call; T gaming is the rating of the online game; T web is the rating of web page browsing; T email is the email's rating; T file is the score of file download / upload; W streaming , W voice , W gaming , W web , W email , W fileIt is the weight coefficient of each business type. The result of this weight coefficient is equal to the proportion of requests of the corresponding business type in the area that needs to be expanded within a period of time.

[0138] Step S103: Based on the priority order, the A* algorithm is used to perform path planning for at least one area that needs to be expanded, so as to obtain the optimal path for the signal optimization vehicle to move.

[0139] In this embodiment, the signal optimization car uses the A* algorithm for intelligent path planning to ensure that the signal optimization car moves to the crowded area along the optimal path. Assume that the signal optimization car needs to move from the current coordinates (coordinate (0,0)) to the center coordinates of area A (coordinate (10,10)), and there are several obstacles in the environment. The A* algorithm is used for path planning, and the optimal path is obtained as follows:

[0140] Path = {(0,0),(2,1),(4,3),(6,6),(8,8),(10,10)}

[0141] Step S104: When the signal optimization vehicle is traveling along the optimal path, a reinforcement learning (RL) method is used to decide whether to continue to expand capacity in the current area or to leave the site.

[0142] In this embodiment, in order to design a more creative and intelligent method for determining whether to stay or leave, we can use the reinforcement learning (RL) method. Through the RL algorithm, the signal optimization car can continuously explore and learn to optimize the decision of whether to stay or leave, and decide whether to continue to expand or leave the site.

[0143] Optionally, the signal optimization vehicle uses a reinforcement learning (RL) method to decide whether to continue capacity expansion or leave the current area during the driving process according to the optimal path, specifically including:

[0144] Initialize the Q value table or neural network;

[0145] Obtain the current status S = {D, U, E} in real time, where D represents the current traffic density, U represents the current PRB utilization rate, and E represents the user experience score;

[0146] Select the optimal action A according to the current state S and the Q value table or the neural network, where action A is to continue to expand capacity or leave the site;

[0147] Execute the selected action A;

[0148] Obtain the new state S′={D′,U′,E′} and reward R of the environmental feedback; where D′ represents the new crowd density, U′ represents the new PRB utilization rate, and E′ represents the new user experience score;

[0149] Update the Q value table or neural network parameters according to the new state S′ and reward R;

[0150] Based on the new state S′ and the preset stay / leave judgment condition, it is determined whether to continue capacity expansion or leave the site.

[0151] In this embodiment, the calculation of the reward R is based on the following rules:

[0152] If the user experience score E improves, a positive reward is given.

[0153] If the current passenger flow density D and the current PRB utilization rate U decrease, a positive reward is given (indicating that demand is reduced and people can leave).

[0154] Otherwise a negative reward is given.

[0155] In this embodiment, the signal optimization car performs multiple learnings through a reinforcement learning algorithm (such as Q-learning) to optimize its stay / go decision. The following is a simplified learning process:

[0156] Initialization: Set the initial Q value table or neural network weights. Set the learning rate α and discount factor γ;

[0157] Multiple learning iterations:

[0158] First learning: state S1 = {D = 120, U = 80%, E = 3.8}; select action A1 (continue to expand); get the new state S1′ = {D = 110, U = 70%, E = 3.2} and reward R1 = 1; update the Q value table.

[0159] Second learning: state S2 = {D = 800, U = 40%, E = 4.5, F = 4.6}; select action A2 (leave the scene); get the new state S2′ = {D = 40, U = 30%, E = 2} and reward R2 = 1; update the Q value table.

[0160] (Repeat multiple studies)

[0161] Strategy optimization: Through multiple learning, the signal optimization car continuously updates and optimizes its Q value table or neural network parameters to form the optimal strategy. In each learning process, the signal optimization car will try different actions and adjust the strategy according to the reward.

[0162] Final decision: After multiple learnings, the signal optimization car can quickly make the best decision when encountering a specific state. For example, when the state S = {30, 40%, 2}, the car decides to leave the scene because leaving in this state can get higher long-term rewards.

[0163] Optionally, the stay / leave judgment condition includes:

[0164] When the current traffic density is greater than the first threshold, or the current PRB utilization is greater than the second threshold, or the user experience score is greater than the third threshold, and continues for a period of time T, take the action of continuing to expand capacity; and,

[0165] When the current crowd density is less than the fifth threshold, the current PRB utilization is less than the sixth threshold, and the user experience score is less than or equal to the third threshold, and the situation lasts for a period of time T, take the action of leaving the site;

[0166] Among them, the first threshold is greater than the fifth threshold, and the second threshold is greater than the sixth threshold.

[0167] In this embodiment, the conditions for continuing to expand capacity include high traffic density, high PRB utilization, and low user experience. In these cases, and for a period of time T, taking the action of "continue to expand capacity" will increase network resources. The conditions for leaving the site include low traffic density, low PRB utilization, and high user experience. In these cases, and for a period of time T, taking the action of "leaving the site".

[0168] In this embodiment, the first threshold is, for example, 100, the second threshold is, for example, 75%, and the third threshold is, for example, 2.5. For example, when the current traffic density D>100, or the current PRB utilization U>75%, or the current user experience score E>2.5, and it continues for a period of time T, it is determined to continue to expand the capacity.

[0169] In this embodiment, the fifth threshold is, for example, 50, and the sixth threshold is, for example, 50%. For example, when the current crowd density D<50, and the current network load U<50%, and the current user experience score E≤2.5, and lasts for a period of time T, it is judged to leave the scene.

[0170] It should be noted that the communication operator network optimization method provided by the present invention realizes efficient, accurate, intelligent and dynamic network enhancement through the intelligent identification of crowd density by the signal optimization car. At the same time, the use of the intelligent dynamic capacity expansion method can also save the power of the signal optimization car and ensure its long-term network enhancement effect, thereby ensuring a good network experience for users and reducing user complaints. This method has the potential for large-scale promotion and application.

[0171] In a specific embodiment, the communication operator network optimization method may include the following steps:

[0172] Step S1: Real-time crowd monitoring

[0173] The monitoring equipment monitors the flow of people in the application scene in real time, divides the application scene into multiple areas, and numbers them ki. The monitoring equipment uses its imaging function to send back information related to the density of people in each area in real time, including time, location, and flow of people, and establishes a database O of the density of people at each hour.

[0174] Step S2: Future passenger flow prediction

[0175] The signal optimization car can obtain the database O of the monitoring equipment, and then analyze the historical crowd density data of each area ki to predict the future changes in crowd density. The future crowd density can be predicted through the time series model. Commonly used models include the ARIMA model and the seasonal autoregressive moving average model (SARIMA). Here we choose the ARIMA model. The ARIMA model can predict the future trend of time series data, such as stock prices, economic indicators, etc. It is based on the autoregressive (AR) and moving average (MA) processes, and uses differential processing to make the data stable. The ARIMA model works better for stable data and can capture long-term trends and cyclical fluctuations.

[0176] Assuming it is Monday, the historical flow data of area ki for the past 7 days can be shown in Table 1:

[0177] Table 1: Historical flow data

[0178]

[0179]

[0180] Suppose it is 3 o'clock in the afternoon and you need to predict the traffic at the next 4 o'clock. You can use the data from the past 7 days and the ARIMA model to predict the average traffic at the next 4 o'clock.

[0181] Step S3: Expansion decision

[0182] After the flow of people is predicted in step S2, a threshold is set (assuming the threshold is 120), and the area where the predicted flow of people exceeds the flow threshold is selected from ki, recorded as ji. The signal optimization car extracts the relevant capacity indicators of area ji at the current time through the network management platform, including PRB utilization and average downlink traffic. When the two conditions in Table 2 are met at the same time, it is considered that the high load threshold of the base station has been reached and capacity expansion needs to be started.

[0183] Table 2: Reference standards for starting capacity expansion

[0184] Conditional Relationship index Reference Standards Condition 1 Average PRB utilization rate of the cell X Condition 2 Average downlink traffic in the cell Y

[0185] For example, for 4G: X is 80%, Y is 8.4G. That is, if the average PRB utilization rate of the cell is greater than 80%, and the average downlink traffic of the cell is greater than 8.4GB, then capacity expansion can be started. At this time, it is necessary to determine whether the base station in the area will meet the capacity expansion standard in the next hour. The judgment includes:

[0186] Data collection: current number of users, current PRB utilization, current downlink traffic, and current proportion of video service users.

[0187] User Growth Rate Forecast: Calculate the user growth rate.

[0188]

[0189] Among them, U growthrate represents the user growth rate, U predicted Represents the predicted number of users, U current Indicates the current number of users.

[0190] Business type ratio: Calculate the current and predicted number of business type users. In addition to the number of users and traffic, other factors can also be considered, such as the business type of each user. In order to more accurately reflect the impact of different business types on base station resources, the business type influencing factors can be refined. For example, a certain proportion of existing users and predicted new users use video services. To obtain the specific type ratio, the network traffic of the accessed users can be analyzed in the core network test, different types of business data packets (such as video, web browsing, social media, etc.) can be identified, and the traffic proportion of each type of business can be counted.

[0191] Business demand influencing factor: Different businesses have different demands for resources. This demand is defined as the business type influencing factor F. type For example, if the video service has a higher demand for PRB, which is 150% of the ordinary text and voice service, then F video prb = 1.5. If the video service has a higher demand for traffic than the ordinary service, then F video traffic = 2. This value can be set flexibly.

[0192] PRB utilization prediction: PRB utilization is calculated based on the user ratio and demand of each service type.

[0193] Average number of PRBs required per user = current PRB utilization / current number of users * total number of PRBs;

[0194] Number of PRBs required by video service users = average number of PRBs required per user * F video prb;

[0195] The number of PRBs required by other service users = the average number of PRBs required by each user;

[0196] Predicted PRB utilization rate = (predicted number of video service users * number of PRBs required by video service users + predicted number of other service users * number of PRBs required by other service users) / total number of PRBs.

[0197] Downlink traffic forecast: Calculates the downlink traffic based on the user ratio and demand of each service type.

[0198] Average traffic demand per user = current downlink traffic / current number of users;

[0199] Video service user traffic demand = average traffic demand per user * F video traffic;

[0200] Traffic demand of other service users = average traffic demand of each user;

[0201] Predicted downlink traffic = predicted number of video service users * traffic demand of video service users + predicted number of other service users * traffic demand of other service users;

[0202] Capacity expansion judgment: Compare the predicted data with the capacity expansion threshold to determine whether capacity expansion is needed.

[0203] Specific examples to illustrate the calculation steps and formulas:

[0204] Assume the following data: Current number of users: 80; Current PRB utilization: 60%; Current downlink traffic: 5GB; Predicted number of users in the next hour: 120;

[0205] Expansion threshold: The average PRB utilization rate of the cell reaches 80%; the average downlink traffic of the cell reaches 8.4GB;

[0206] Current proportion of video service users: 30%;

[0207] Predicted video service user ratio: 30% (assuming that the video service user ratio in the predicted number of users is the same as the video service user ratio in the current number of users, but may be different);

[0208] Factor affecting the PRB demand of video services: Assuming that the PRB demand of video services increases by 50%, then Ftype = 1.5 (assuming that the PRB demand of video service users is 1.5 times that of other service users);

[0209] Factor affecting traffic demand of video service: Assuming that the traffic demand of video service increases by 100%, then Ftype=2 (assuming that the traffic demand of video service users is twice that of other service users).

[0210] (1) Data collection and preprocessing

[0211] Collect current basic data, including the number of users, PRB utilization, downlink traffic, and the proportion of video service users.

[0212] (2) User growth forecast

[0213] Calculate future user growth rate:

[0214] User growth rate = (predicted number of users - current number of users) / current number of users × 100% = (120-80) / 80 × 100% = 50%

[0215] (3) Business type ratio

[0216] Current number of video service users: Current number of video service users = 80 × 0.3 = 24

[0217] Predicted number of video service users: Predicted number of video service users = 120 × 0.3 = 36

[0218] (4) PRB Utilization Prediction

[0219] Users are divided into video service users and other service users, and their PRB requirements are calculated respectively:

[0220] Current average PRB requirement per user: Average PRB requirement per user = current PRB utilization / current number of users * total number of PRBs = 60% * 100 / 80 = 0.75 (for 4G, the total number of PRBs is 100, and for 5G it is 273);

[0221] PRB requirement of video service users: PRB requirement of video service users = 0.75 × 1.5 = 1.125;

[0222] PRB requirement of other service users: PRB requirement of other service users = 0.75;

[0223] Predicted PRB utilization: predicted PRB utilization = (36 × 1.125 + (120-36) × 0.75) / total number of PRBs = 103.5%;

[0224] (5) Downlink Traffic Prediction

[0225] Users are divided into video service users and other service users, and their traffic demands are calculated respectively:

[0226] Current average traffic demand per user: Average traffic demand per user = 5GB / 80 = 0.0625GB / user;

[0227] Traffic demand of video service users: Traffic demand of video service users = 0.0625 GB / user × 2 = 0.125 GB / user;

[0228] Traffic demand of other business users: Traffic demand of other business users = 0.0625 GB / user;

[0229] Predicted downlink traffic: Predicted downlink traffic = (36 × 0.125 GB / user + 84 × 0.0625 GB / user) = 4.5 GB + 5.25 GB = 9.75 GB;

[0230] (6) Determine whether capacity expansion is needed:

[0231] Compare the predicted PRB utilization and downlink traffic with the capacity expansion threshold: Compare the predicted PRB utilization with the capacity expansion threshold: 103.5%>80% (reaching the capacity expansion threshold); Compare the predicted downlink traffic with the capacity expansion threshold: 9.75GB>8.4GB (reaching the capacity expansion threshold); therefore, capacity expansion is required.

[0232] Step S4: Assuming that there are multiple areas in the predicted area ji that need to be expanded, the signal optimization car is determined based on the priority to expand the area.

[0233] The priority is calculated according to the following formula:

[0234] Priority = W1*D / current total traffic density of multiple areas that need to be expanded + W2*N / predicted number of users in multiple areas that need to be expanded + W3*L+W4*E+W5*T

[0235] Where: D is the current flow density of people in the area;

[0236] N is the predicted number of users in the area;

[0237] L is the current network load in the area;

[0238] E is the user experience score (unit: points);

[0239] T is the business type score (unit: points);

[0240] W1, W2, W3, W4, and W5 are weight coefficients that reflect the importance of each factor.

[0241] Specifically, the user experience score (E) is an indicator that reflects the overall satisfaction and experience quality of users using network services in the region. The score can be comprehensively evaluated and calculated through the following aspects:

[0242] (1) Network delay (Latency):

[0243] Description: Refers to the time required for data to be sent from the user end to the receiving end, usually in milliseconds (ms).

[0244] Scoring criteria: The higher the latency, the worse the user experience. A latency of less than 50ms is scored as 1 point, 50-100ms is scored as 2 points, 100-150ms is scored as 3 points, 150-200ms is scored as 4 points, and greater than 200ms is scored as 5 points.

[0245] (2) Packet Loss Rate:

[0246] Description: Refers to the ratio of data packets lost during data transmission to the total data packets, usually expressed as a percentage (%).

[0247] Scoring criteria: The lower the packet loss rate, the better the user experience. A packet loss rate of less than 1% is worth 1 point, 1-2% is worth 2 points, 2-5% is worth 3 points, 5-10% is worth 4 points, and greater than 10% is worth 5 points.

[0248] (3) Network Jitter:

[0249] Description: Refers to the variation of network latency, usually measured in milliseconds (ms).

[0250] Scoring criteria: The smaller the jitter, the better the user experience. If the jitter is less than 10ms, 1 point is given; if it is 10-20ms, 2 points; if it is 20-30ms, 3 points; if it is 30-50ms, 4 points; and if it is greater than 50ms, 5 points is given.

[0251] The above data can be obtained from the network management, and the indicator time granularity can be the past 5 minutes.

[0252] In order to obtain the comprehensive user experience score (E) of region ji, the above factors can be comprehensively weighted. For example:

[0253] E=W latency *E latency +W packet_loss *E packet_loss +W jitter *E jitter

[0254] Where W latency , W packet_loss , W jitter is the corresponding weight coefficient.

[0255] Specifically, the service type score (T) is based on the importance of different service types to network requirements. Different types of services have different requirements for network bandwidth, latency, stability, etc. The score can be comprehensively evaluated and calculated based on the following aspects. The higher the score standard, the more urgent the need for capacity expansion:

[0256] (1) Streaming Video:

[0257] Description: Such as watching high-definition videos, online live broadcasts, etc.

[0258] Rating criteria: Since video streaming has high requirements for bandwidth and stability, the rating is high. The rating is 5 points.

[0259] (2) Voice Call:

[0260] Description: Such as VoIP calls, Internet voice chats, etc.

[0261] Rating criteria: Voice calls have high requirements for latency and jitter, and the rating is medium to high. The rating is 4 points.

[0262] (3) Online Gaming:

[0263] Description: Such as multiplayer online games, etc.

[0264] Rating criteria: Online games have extremely high requirements for latency and jitter, and the highest rating is 5 points.

[0265] (4) Web Browsing:

[0266] Description: Such as browsing the web, online shopping, etc.

[0267] Rating criteria: Web browsing has moderate bandwidth and latency requirements, and the rating is medium. The rating is 3 points.

[0268] (5) Email:

[0269] Description: Such as sending and receiving emails, downloading attachments, etc.

[0270] Scoring criteria: Email has low requirements for network bandwidth and latency and is rated the lowest, with a score of 1.

[0271] (6) File Download / Upload:

[0272] Description: Such as downloading large files, uploading data, etc.

[0273] Scoring criteria: File downloads and uploads have high bandwidth requirements, but low latency requirements, and are rated high. The score is 4 points.

[0274] In order to obtain the business type score (T) of region ji, the above factors can be comprehensively weighted:

[0275] T=W streaming *T streaming +W voice *T voice +W gaming*T gaming

[0276] +W web *T web +W email *T email +W file *T file

[0277] Among them, T streamin is the rating of the video streaming; T voice is the rating of the voice call; T gaming is the rating of the online game; T web is the rating of web page browsing; T email is the rating of the email; T file is the score of file download / upload; W streaming , W voice , W gaming , W web , W email , W file is the weight coefficient of each service type. The result of this weight coefficient is equal to the proportion of requests of the corresponding service type in the area to be expanded within a period of time. For example, in the past 10 minutes, the total number of service requests is 1000 times, and the video streaming service has been requested 100 times, then W streaming =100 / 1000=0.1.

[0278] Dynamically adjust weights: Assuming there are two different scenarios: ordinary daily life and major event days, different weights can be set. For example, in ordinary daily life, the weight coefficients are W1 = 0.4, W2 = 0.2, W3 = 0.3, W4 = 0.05, W5 = 0.05, while in major event days, the weight coefficients are W1 = 0.3, W2 = 0.4, W3 = 0.1, W4 = 0.1, W5 = 0.1.

[0279] Priority calculation example:

[0280] Assume that the data of three regions A, B, and C are as shown in Table 3:

[0281] Table 3: Regional data

[0282]

[0283] Assuming the weight coefficients are W1 = 0.4, W2 = 0.2, W3 = 0.2, W4 = 0.1, and W5 = 0.1, the priority of each area is calculated as follows:

[0284] Calculate the priority of region A:

[0285] A=0.4*150 / (150+180+160)+0.2*300 / (300+250+350)+0.2*70%+0.1*4.0+0.1*5.0=60 / 490+60 / 900+0.14+0.4+0.5=1.11

[0286] Calculate the priority of region B:

[0287] B=0.4*180 / (150+180+160)+0.2*250 / (300+250+350)+0.2*60%+0.1*3.5+0.1*4.5=72 / 490+50 / 900+0.12+0.35+0.4=0.93 Calculate the priority of area C:

[0288] C=0.4*160 / (150+180+160)+0.2*350 / (300+250+350)+0.2*80%+0.1*3.8+0.1*4.8=64 / 490+70 / 900+0.16+0.38+0.4=1.02

[0289] According to the priority calculation results, the signal optimization car should move to area A first.

[0290] Step S5: Signal optimization vehicle path planning

[0291] The signal optimization car uses the A* algorithm for intelligent path planning to ensure that the signal optimization car moves to crowded areas along the optimal path. The path planning formula of the A* algorithm is:

[0292] f(n)=g(n)+h(n)

[0293] Among them, g(n) is the cost from the starting point to the current node n, and h(n) is the estimated cost from the current node to the target node.

[0294] Assume that the signal optimization car needs to move from the current coordinate (coordinate (0,0)) to the center coordinate (coordinate (10,10)) of area A. There are several obstacles in the environment. The A* algorithm is used for path planning, and the optimal path is:

[0295] Path = {(0,0),(2,1),(4,3),(6,6),(8,8),(10,10)}

[0296] Step S6: Determine whether to stay or leave

[0297] In order to design a more creative and intelligent method for deciding whether to stay or leave, we can use the reinforcement learning (RL) method. Through the RL algorithm, the signal optimization car can continuously explore and learn to optimize the decision of whether to continue to expand or leave the site.

[0298] The definitions of states, actions, and rewards are as follows:

[0299] State (S): The current state of the environment, including the current traffic density D, downlink PRB average utilization (PRB utilization for short), user experience score, etc. The state can be expressed as a four-tuple: S = {D, U, E}

[0300] Action (A): The actions that the signal optimization car can take, including "continue to expand capacity" and "leave the scene". Continue to expand capacity: increase network resources to meet current needs and continue to support the needs of high-traffic areas. Leave the scene: stop expanding capacity and move to the next area.

[0301] Reward (R): When the signal optimization car takes an action, it will feedback a reward value based on the effect of the action. The reward value depends on whether the user experience score improves, as well as changes in traffic density and network utilization. Rewards or penalties are given based on the effect of the action to guide the learning process.

[0302] The specific steps are as follows:

[0303] The signal optimization car selects an action at each moment based on the current state and adjusts its strategy based on the effect of the action. This process can be achieved through reinforcement learning algorithms, such as Q-learning.

[0304] 1. Initialize the Q value table or neural network:

[0305] Q-value table: used to store the Q-value of each state-action pair, indicating the expected return of taking the action in that state.

[0306] Neural Networks: Used to approximate Q-value tables, which are more effective when the state space is large.

[0307] 2. Real-time monitoring status:

[0308] Get the current status S={D,U,E} in real time

[0309] 3. Choose the best action:

[0310] According to the current state S and the learning strategy (Q value table or neural network), select the optimal action A.

[0311] Action A can be "continue to expand capacity" or "leave the site".

[0312] 4. Execute the action:

[0313] The car performs the selected action A.

[0314] 5. Environmental feedback:

[0315] After executing the action, the environment will feedback the new state S′={D′,U′,E′} and reward R.

[0316] The calculation of the reward R is based on the following rules:

[0317] If the user experience score E improves, a positive reward is given.

[0318] If the passenger flow density D and PRB utilization rate U decrease, a positive reward is given (indicating that demand is reduced and they can leave).

[0319] Otherwise a negative reward is given.

[0320] 6. Update strategy:

[0321] Use the Q-learning algorithm to update the Q value table or neural network parameters:

[0322]

[0323] Among them, α is the learning rate and γ is the discount factor.

[0324] 7. Decision output:

[0325] Based on the new state S′ and the stay / go judgment condition, it is determined whether to continue to expand or leave the site.

[0326] Specifically, the criteria for deciding whether to stay or leave include:

[0327] 1. Continue to expand capacity:

[0328] High traffic density: The current traffic density D is higher than the threshold, and there are still a large number of users who need network resources.

[0329] High PRB utilization: The current network load U is high, indicating that the existing resources are insufficient to meet the demand.

[0330] Low user experience: The user experience score E is high, and network quality needs to be improved through capacity expansion.

[0331] In these cases, and for a period of time T, taking the action of "continue to expand capacity" will increase network resources.

[0332] 2. Leave the scene:

[0333] Low traffic density: The current traffic density D is lower than the threshold, indicating that the number of users has decreased and no additional network resources are needed.

[0334] Low PRB utilization: The current network load U is low and the existing resources are sufficient to meet the demand.

[0335] High user experience: A higher user experience score E indicates good network quality and satisfactory user experience.

[0336] In these cases, and for a period of time T, a "leave the scene" action is taken.

[0337] Among them, in order to realize the decision of staying or leaving based on reinforcement learning, it is also necessary to clarify the threshold of each indicator and how to make decisions based on these indicators.

[0338] The indicators and threshold settings can be referred to as follows, or they can be set separately.

[0339] (a) Real-time crowd density (D): high crowd density threshold: 100 people, low crowd density threshold: 50 people;

[0340] (b) Downlink PRB average utilization (U): (b) high utilization threshold: 75%, low utilization threshold: 50%;

[0341] The purpose of setting two thresholds, high crowd density threshold (first threshold) and low crowd density threshold (fifth threshold), high utilization threshold and low utilization threshold is to prevent frequent decision switching and ensure the stability and responsiveness of the system. This method is called hysteresis or hysteresis control in the control system. Specifically:

[0342] Prevent frequent switching: If only one threshold is set (for example, 100 people), then when the density of people fluctuates around this threshold, the car may frequently switch between expanding and leaving. For example, if the density of people changes from 99 to 101, the car will switch actions, which will lead to frequent and unnecessary operations.

[0343] Ensure system stability: Setting two thresholds ensures that the system has a clear buffer zone between high load and low load, thereby preventing frequent decision switching. For example, the high crowd density threshold is set to 100 people, and the low crowd density threshold is set to 50 people. Only when the crowd density is lower than 50 people will the car consider leaving, and when the crowd density is higher than 100 people, it will consider further expansion. In this way, the middle area (50 to 100 people) provides stability.

[0344] More flexible response: The two thresholds enable the system to respond more flexibly to different load conditions. Under high load conditions, the system can respond quickly and increase resources; under low load conditions, the system can release resources in time, and under medium load conditions, the system can remain stable.

[0345] It should be noted that although a single threshold (such as 100 people) can simplify the judgment conditions, it may cause the following problems: Frequent switching: As mentioned above, the system will frequently switch between expansion and departure, resulting in resource waste and reduced efficiency. Insufficient response: When approaching the threshold, the system may not be able to respond to load changes in a timely manner because it needs to wait until it exceeds or falls below the threshold before making a decision.

[0346] (c) User experience score (E): Low experience threshold: 2.5-5 points, High experience threshold: 0-2.5 points

[0347] Based on the threshold, the car determines whether to continue to expand or leave the site based on the following conditions:

[0348] (1) Conditions for continued expansion:

[0349] The current traffic density D>100, or the current network load (i.e., the current PRB utilization) U>75%, or the current user experience score E>2.5, for a period of time T

[0350] (2) Conditions for leaving the site:

[0351] The current traffic density D is less than 50, the current network load U is less than 50%, and the current user experience score E is less than or equal to 2.5, and it lasts for a period of time T.

[0352] Specific examples

[0353] Initial state: S = {D = 120, U = 80%, E = 3.8}

[0354] Action selection: According to the current Q value table or neural network, select the action "continue to expand" or "leave the scene". Assume that the signal optimization car chooses "continue to expand" according to the current strategy and increases network resources.

[0355] Environmental feedback: After executing the action, the new state is: S′={D=110,U=70%,E=3.0};

[0356] Rewards are given based on the user experience score: R = 1 (user experience score improvement)

[0357] Update strategy: Use Q-learning algorithm to update Q value:

[0358]

[0359] Decision output: Based on the new state S′ and the updated strategy, decide again whether to continue to expand capacity or leave the site.

[0360] Assume that after multiple learnings, the car finally makes a decision in the following situations:

[0361] Current status: S = {30, 40%, 2}

[0362] Action selection: Based on the above stay / leave conditions: crowd density 30 < 100; network utilization 40% < 50%; user experience score 2 less than 2.5; the above situation lasts for T; the conditions for leaving the site are met, so select "Leave the site".

[0363] It should be noted that in reinforcement learning, multiple learning is necessary because it enables the agent (in this case, the signal optimization car) to optimize its strategy and improve the accuracy and efficiency of its decision-making through continuous interaction with the environment. The core purpose of this process is to enable the car to make the best decision when facing different environmental states. The reasons and benefits of multiple learning are reflected in the following aspects:

[0364] a) Adapting to complex environments: The mobile communication network environment is complex and dynamically changing. Parameters such as crowd density, network load, and user experience are constantly changing, and a single learning cannot cover all possible states. Through multiple learning, the car can encounter and adapt to more states, thereby improving its decision-making ability in different environments.

[0365] b) Optimizing strategies: Reinforcement learning gradually optimizes strategies through trial and error. In each learning session, the car will try different actions and adjust its decision-making strategy based on the results. Multiple learning sessions ensure that the car finds the best strategy and maximizes long-term rewards (such as user experience and network performance).

[0366] c) Reduce overfitting: In machine learning, overfitting means that the model performs well on the training data but performs poorly on new data. Multiple learning and extensive environment interactions can reduce the overfitting of the car's policy, allowing it to make effective decisions in a variety of new environments.

[0367] d) Experience accumulation: Each learning adds to the car's experience. As experience accumulates, the car can more quickly and accurately identify and adapt to environmental changes. Through experience replay, the car can review past learning data and further optimize its strategy.

[0368] e) Improved robustness: Through multiple learning, the car can better handle abnormal situations and noisy data, improving the robustness and reliability of decision-making. This ability is very important in practical applications because real-world data often contains noise and outliers.

[0369] Example: Decision-making process for multiple learning

[0370] Assume that the car learns multiple times through a reinforcement learning algorithm (such as Q-learning) to optimize its decision to stay or go. The following is a simplified learning process:

[0371] Initialization: Set the initial Q value table or neural network weights. Set the learning rate α and discount factor γ;

[0372] Multiple learning iterations:

[0373] First learning: state S1 = {D = 120, U = 80%, E = 3.8}; select action A1 (continue to expand); get the new state S1′ = {D = 110, U = 70%, E = 3.2} and reward R1 = 1; update the Q value table.

[0374] Second learning: state S2 = {D = 800, U = 40%, E = 4.5, F = 4.6}; select action A2 (leave the scene); get the new state S2′ = {D = 40, U = 30%, E = 2} and reward R2 = 1; update the Q value table.

[0375] (Repeat multiple studies)

[0376] Strategy optimization: Through multiple learning, the car continuously updates and optimizes its Q value table or neural network parameters to form the optimal strategy. In each learning process, the car will try different actions and adjust the strategy based on the reward.

[0377] Final decision: After multiple learnings, the car can quickly make the best decision when encountering a specific state. For example, when the state S = {30, 40%, 2}, the car decides to leave the scene because leaving in this state can get higher long-term rewards.

[0378] It should be noted that the specific number of multiple learning is usually not fixed in advance, but is dynamically determined based on the complexity of the task, changes in the environment, and the convergence of the reinforcement learning algorithm. The following factors can help determine the number of multiple learning:

[0379] a) Task complexity: Complex tasks require more learning times to cover a variety of possible states and actions. For example, simple tasks may require hundreds to thousands of learning times, while complex tasks may require tens of thousands or even more.

[0380] b) Environmental changes: A dynamically changing environment requires continuous learning to adapt to new situations. If the environment changes frequently, more learning times are required to ensure that the robot can continuously adapt to the changes.

[0381] c) Algorithm convergence: The convergence speed of the reinforcement learning algorithm affects the number of learning times. When the strategy gradually stabilizes and the reward value no longer changes significantly, the algorithm tends to converge. You can judge whether the algorithm has converged by monitoring the cumulative reward or the strategy change rate.

[0382] Specifically, the method for determining the number of multiple learning times includes:

[0383] 1) Fixed number method: Initially set a larger number of learning times and then observe the performance of the algorithm. This method is simple but not flexible enough: Advantages: Simple to implement. Disadvantages: May lead to over-learning (waste of computing resources) or under-learning (unoptimized strategy).

[0384] 2) Monitoring convergence method: Dynamically adjust the number of learning times by monitoring the convergence of the algorithm. Stop learning when the strategy changes tend to be stable or the cumulative reward reaches the expected value: Advantages: resource efficiency, ensuring that the algorithm fully learns; Disadvantages: requires additional monitoring and adjustment mechanisms.

[0385] 3) Based on performance indicators: Set some performance indicators (such as average reward, success rate, etc.), and stop learning when these indicators reach the set values: Advantages: It can ensure that the strategy achieves the expected results; Disadvantages: It requires careful design and selection of appropriate performance indicators.

[0386] Suppose we use the Q-learning algorithm for multiple learning and determine the number of learning times by monitoring the change in cumulative rewards:

[0387] Initial settings: learning rate (α) is 0.1; discount factor (γ) is 0.9; initial Q value is 0;

[0388] Learning process: Each learning includes the following steps: the car selects an action in the environment; executes the action and receives the reward and new state; updates the Q value; and the cumulative reward changes are used as the judgment criterion.

[0389] Determine convergence: Set a threshold (such as the average cumulative reward change is less than 1%). If the average cumulative reward change is less than 1% in 100 consecutive learning times, the algorithm is considered to have converged.

[0390] In short, through the above-mentioned improvement scheme, the network reinforcement method of the signal optimization car in public places has become more intelligent and efficient, and can dynamically adapt to changes in passenger flow and improve users' network experience.

[0391] The communication operator network optimization method provided by the embodiment of the present invention first obtains at least one area that needs to be expanded in the application scenario; then calculates the priority of each area according to the current traffic density, predicted number of users, current network load, user experience score and service type score of each area in the at least one area that needs to be expanded; then uses the A-star algorithm to plan the path of the at least one area that needs to be expanded based on the order of the priorities to obtain the optimal path for the signal optimization car to move; finally, in the process of the signal optimization car driving according to the optimal path, the reinforcement learning RL method is used to decide whether to continue to expand the current area or leave the scene. The present invention uses a movable signal optimization car to intelligently identify at least one area that needs to be expanded in the application scenario, and performs priority sorting and path planning, so that the signal optimization car can move to the crowded areas in order of priority, dynamically realize network expansion, not only improve flexibility, but also ensure the user's good network experience, and reduce user complaints. The solution solves the problem that the existing communication operators' network optimization method is mainly to install fixed indoor distributed base stations in high-traffic areas to increase capacity, which is insufficient in flexibility, low in efficiency, and difficult to cope with changes in traffic within the scene, resulting in inaccurate and untimely capacity expansion, affecting users' network experience.

[0392] Embodiment 2:

[0393] like Figure 2 As shown, this embodiment provides a communication operator network optimization device, which is arranged in a signal optimization vehicle and is used to execute the above communication operator network optimization method, including:

[0394] An expansion area acquisition module 11 is used to acquire at least one area that needs to be expanded in an application scenario;

[0395] A priority calculation module 12, connected to the expansion area acquisition module 11, is used to calculate the priority of each area according to the current flow density, predicted number of users, current network load, user experience score and service type score of each area of ​​the at least one area that needs to be expanded;

[0396] A vehicle path planning module 13, connected to the priority calculation module 12, is used to perform path planning for at least one area that needs to be expanded based on the order of the priorities using the A-star algorithm to obtain an optimal path for the movement of the signal-optimized vehicle;

[0397] The car network optimization module 14 is connected to the car path planning module 13, and is used to use the reinforcement learning RL method to decide whether to continue to expand or leave the site in the current area when the signal optimization car is driving according to the optimal path.

[0398] Optionally, the expansion area acquisition module 11 includes:

[0399] A first acquisition unit is used to acquire an area in an application scenario where capacity expansion needs to be started;

[0400] A second acquisition unit is used to acquire the predicted physical resource block PRB utilization and predicted downlink traffic in the next preset time period for the area where the capacity expansion needs to be started;

[0401] The first determination unit is used to determine the area where expansion needs to be started and whose predicted PRB utilization is greater than the first expansion threshold and whose predicted downlink traffic is greater than the second expansion threshold as the area where expansion needs to be started, and obtain at least one area where expansion needs to be started.

[0402] Optionally, the first acquiring unit specifically includes:

[0403] A historical crowd density acquisition unit, used to acquire historical crowd density data of multiple areas in an application scenario from a crowd density information database of a monitoring device;

[0404] The predicted crowd density acquisition unit is used to analyze the historical crowd density data of each area through a time series model and predict the predicted crowd density data at a future moment;

[0405] An area selection unit, configured to select an area where the predicted crowd density data exceeds a crowd flow threshold from a plurality of areas;

[0406] The area determination unit is used to obtain the relevant capacity index of the selected area at the current moment, and determine the area where capacity expansion needs to be started according to the relevant capacity index.

[0407] Optionally, the second acquiring unit specifically includes:

[0408] A data collection unit, used to collect the current number of users, the current PRB utilization rate, the current downlink traffic and the current proportion of video service users for the area where the expansion needs to be started;

[0409] A predicted PRB utilization acquisition unit, configured to acquire a predicted PRB utilization for a next preset time period based on the current number of users, the current PRB utilization and the current video service user ratio;

[0410] The predicted downlink traffic acquisition unit is used to acquire the predicted downlink traffic in the next preset time period based on the current number of users and the current downlink traffic.

[0411] Optionally, the predicted PRB utilization acquisition unit specifically includes:

[0412] A first calculation unit, configured to calculate a user growth rate according to the current number of users and a predicted number of users in a next preset time period;

[0413] A second calculation unit, configured to calculate the current number of video service users and the predicted number of video service users according to the current video service user ratio;

[0414] A first definition unit is used to define a service type impact factor corresponding to a video service according to a demand of the video service for the PRB;

[0415] A third calculation unit, configured to calculate an average number of PRB requirements per user according to the current PRB utilization rate, the current number of users and the total number of PRBs;

[0416] A fourth calculation unit, configured to calculate the number of PRB requirements of a video service user according to the average number of PRB requirements per user and a service type impact factor corresponding to the video service;

[0417] A fifth calculation unit, configured to calculate the number of PRBs required by other service users according to the average number of PRBs required per user;

[0418] The sixth calculation unit is used to calculate the predicted PRB utilization rate according to the predicted number of video service users, the number of PRB requirements of video service users, the predicted number of other service users, and the number of PRB requirements of other service users.

[0419] Optionally, the predicted downlink traffic acquisition unit specifically includes:

[0420] A seventh calculation unit, configured to calculate an average traffic demand per user according to the current downlink traffic and the current number of users;

[0421] The second definition unit is used to define a flow demand influencing factor corresponding to the video service according to the flow demand of the video service;

[0422] An eighth calculation unit, configured to calculate a traffic demand number of a video service user according to the average traffic demand number per user and a traffic demand influencing factor corresponding to the video service;

[0423] A ninth calculation unit, configured to calculate the traffic demand of other service users according to the average traffic demand per user;

[0424] The tenth calculation unit is used to calculate the predicted downlink traffic according to the predicted number of video service users, the traffic demand number of video service users, the predicted number of other service users, and the traffic demand number of other service users.

[0425] Optionally, the priority calculation module 12 calculates the priority of each area according to the following formula:

[0426]

[0427] Where P is the priority, W1, W2, W3, W4, and W5 are weight coefficients, D is the current flow density in the current area, and D SUM is the current total flow density of at least one area that needs to be expanded, N is the predicted number of users in the current area, and N SUM is the predicted total number of users in at least one area that needs to be expanded, L is the current network load in the current area, E is the user experience score, and T is the service type score.

[0428] Optionally, the device further comprises:

[0429] A network parameter acquisition module, used to obtain the network delay, network packet loss rate and network jitter of each of the areas within a preset time period in the past;

[0430] A first comprehensive weighting module is used to perform comprehensive weighted calculation on the network delay, network packet loss rate and network jitter of each of the regions in the past preset time period to obtain a user experience score corresponding to each of the regions;

[0431] A service score acquisition module, used to acquire scores corresponding to different types of services in each of the areas within a preset time period in the past, wherein the different types of services include at least two of the following: video streaming, voice calls, online games, web browsing, emails, and file downloads / uploads;

[0432] The second comprehensive weighting module is used to perform comprehensive weighted calculation on the scores corresponding to different types of businesses in each of the areas within a preset time period in the past, so as to obtain a business type score corresponding to each of the areas.

[0433] Optionally, the car network optimization module 14 specifically includes:

[0434] An initialization unit, used to initialize the Q value table or the neural network;

[0435] A current state acquisition unit, used to acquire the current state S={D,U,E} in real time, where D represents the current crowd density, U represents the current PRB utilization rate, and E represents the user experience score;

[0436] An optimal action selection unit, used to select an optimal action A according to the current state S and the Q value table or the neural network, wherein the action A is to continue to expand the capacity or leave the site;

[0437] An action execution unit, used to execute the selected action A;

[0438] Feedback reward unit, used to obtain the new state S′={D′,U′,E′} and reward R of the environment feedback;

[0439] A value updating unit, used for updating the Q value table or neural network parameters according to the new state S′ and reward R;

[0440] The stay / leave judgment unit is used to judge whether to continue capacity expansion or leave the site based on the new state S′ and the preset stay / leave judgment condition.

[0441] Optionally, the stay / leave judgment condition includes:

[0442] When the current traffic density is greater than the first threshold, or the current PRB utilization is greater than the second threshold, or the user experience score is greater than the third threshold, and continues for a period of time T, take the action of continuing to expand capacity; and,

[0443] When the current crowd density is less than the fifth threshold, the current PRB utilization is less than the sixth threshold, and the user experience score is less than or equal to the third threshold, and the situation lasts for a period of time T, take the action of leaving the site;

[0444] Among them, the first threshold is greater than the fifth threshold, and the second threshold is greater than the sixth threshold.

[0445] Embodiment 3:

[0446] refer to Figure 3 This embodiment provides a communication operator network optimization device, including a memory 21 and a processor 22, the memory 21 stores a computer program, and the processor 22 is configured to run the computer program to execute the communication operator network optimization method in Example 1.

[0447] The memory 21 is connected to the processor 22. The memory 21 may be a flash memory, a read-only memory or other memory. The processor 22 may be a central processing unit or a single-chip microcomputer.

[0448] Embodiment 4:

[0449] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the communication operator network optimization method in the above-mentioned embodiment 1 is implemented.

[0450] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0451] In summary, the communication operator network optimization method, device and readable storage medium provided by the embodiment of the present invention first obtain at least one area that needs to be expanded in the application scenario; then calculate the priority of each area according to the current flow density, predicted number of users, current network load, user experience score and service type score of each area in the at least one area that needs to be expanded; then use the A star algorithm to plan the path of at least one area that needs to be expanded based on the order of the priorities to obtain the optimal path for the signal optimization car to move; finally, in the process of the signal optimization car driving according to the optimal path, the reinforcement learning RL method is used to decide whether to continue to expand or leave the scene in the current area. The present invention uses a movable signal optimization car to intelligently identify at least one area that needs to be expanded in the application scenario, and performs priority sorting and path planning, so that the signal optimization car can move to the densely populated areas in order of priority, dynamically realize network expansion, not only improve flexibility, but also ensure the user's good network experience, and reduce user complaints. The solution solves the problem that the existing communication operators' network optimization method is mainly to install fixed indoor distributed base stations in high-traffic areas to increase capacity, which is insufficient in flexibility, low in efficiency, and difficult to cope with changes in traffic within the scene, resulting in inaccurate and untimely capacity expansion, affecting users' network experience.

[0452] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A communication operator network optimization method, characterized in that: Applied to a signal optimization vehicle, the method comprises: Obtain at least one area that needs to be expanded in the application scenario; Calculate the priority of each area according to the current traffic density, predicted number of users, current network load, user experience score and service type score of each area of ​​at least one area requiring capacity expansion; Based on the order of the priorities, the A-star algorithm is used to perform path planning for at least one area that needs to be expanded, so as to obtain an optimal path for the signal optimization vehicle to move; When the signal optimization vehicle is traveling along the optimal path, a reinforcement learning (RL) method is used to decide whether to continue capacity expansion or leave the current area.

2. The method according to claim 1, characterized in that The obtaining of at least one area that needs to be expanded in the application scenario specifically includes: Get the area where expansion needs to be started in the application scenario; Obtaining predicted physical resource block PRB utilization and predicted downlink traffic in the next preset time period for the area where capacity expansion needs to be started; The area where capacity expansion needs to be started and whose predicted PRB utilization is greater than the first capacity expansion threshold and whose predicted downlink traffic is greater than the second capacity expansion threshold is determined as the area where capacity expansion needs to be started, and at least one area where capacity expansion needs to be started is obtained.

3. The method according to claim 2, characterized in that The obtaining of the area where expansion needs to be started in the application scenario specifically includes: Obtain historical crowd density data of multiple areas in the application scenario from the crowd density information database of the monitoring device; Analyze the historical crowd density data of each area through the time series model to predict the crowd density data at future moments; Selecting an area where the predicted crowd density data exceeds a crowd flow threshold from a plurality of areas; The relevant capacity index of the selected area at the current moment is obtained, and the area where capacity expansion needs to be started is determined according to the relevant capacity index.

4. The method according to claim 2, characterized in that: The obtaining, for the area where the capacity expansion needs to be started, a predicted physical resource block PRB utilization rate and a predicted downlink traffic volume in the next preset time period specifically includes: For the area where capacity expansion needs to be started, the current number of users, current PRB utilization, current downlink traffic, and current proportion of video service users are collected; Obtaining a predicted PRB utilization rate for the next preset time period based on the current number of users, the current PRB utilization rate, and the current video service user ratio; The predicted downlink traffic in the next preset time period is obtained based on the current number of users and the current downlink traffic.

5. The method according to claim 4, characterized in that The obtaining the predicted PRB utilization rate for the next preset time period based on the current number of users, the current PRB utilization rate and the current video service user ratio specifically includes: Calculating a user growth rate based on the current number of users and the predicted number of users in the next preset time period; Calculating the current number of video service users and the predicted number of video service users according to the current video service user ratio; Define the service type influencing factor corresponding to the video service according to the demand of the video service for PRB; Calculate the average number of PRBs required per user according to the current PRB utilization, the current number of users and the total number of PRBs; Calculate the number of PRBs required by video service users according to the average number of PRBs required per user and the service type impact factor corresponding to the video service; Calculate the number of PRBs required by other service users according to the average number of PRBs required by each user; The predicted PRB utilization is calculated according to the predicted number of video service users, the number of PRBs required by video service users, the predicted number of other service users, and the number of PRBs required by other service users.

6. The method according to claim 5, characterized in that The obtaining the predicted downlink traffic in the next preset time period based on the current number of users and the current downlink traffic specifically includes: Calculate the average traffic demand per user according to the current downlink traffic and the current number of users; Define the traffic demand influencing factor corresponding to the video service according to the traffic demand of the video service; Calculate the traffic demand of video service users according to the average traffic demand of each user and the traffic demand influencing factor corresponding to the video service; Calculate the traffic demand of other service users according to the average traffic demand of each user; The predicted downlink traffic is calculated according to the predicted number of video service users, the traffic demand number of video service users, the predicted number of other service users, and the traffic demand number of other service users.

7. The method according to claim 1, characterized in that The priority of each area is calculated according to the current flow density, predicted number of users, current network load, user experience score and service type score of each area in the at least one area that needs to be expanded, and is specifically calculated according to the following formula: Where P is the priority, W1, W2, W3, W4, and W5 are weight coefficients, D is the current flow density in the current area, and D SUM is the current total flow density of at least one area that needs to be expanded, N is the predicted number of users in the current area, and N SUM is the predicted total number of users in at least one area that needs to be expanded, L is the current network load in the current area, E is the user experience score, and T is the service type score.

8. The method according to claim 7, characterized in that Before calculating the priority of each area according to the current flow density, predicted number of users, current network load, user experience score and service type score of each area of ​​at least one area requiring capacity expansion, the method further includes: Obtaining the network delay, network packet loss rate and network jitter of each of the areas within a preset time period in the past; Perform a comprehensive weighted calculation on the network delay, network packet loss rate and network jitter in each of the areas within a preset time period in the past to obtain a user experience score corresponding to each of the areas; Obtain scores corresponding to different types of services in each of the areas in a preset time period in the past, where the different types of services include at least two of the following: video streaming, voice calls, online games, web browsing, emails, and file downloads / uploads; A comprehensive weighted calculation is performed on the scores corresponding to different types of businesses in each of the regions within a preset time period in the past to obtain a business type score corresponding to each of the regions.

9. The method according to claim 1, characterized in that: The signal optimization vehicle uses a reinforcement learning (RL) method to determine whether to continue capacity expansion or leave the current area during the driving process according to the optimal path, specifically including: Initialize the Q value table or neural network; Obtain the current status S = {D, U, E} in real time, where D represents the current traffic density, U represents the current PRB utilization rate, and E represents the user experience score; Select the optimal action A according to the current state S and the Q value table or the neural network, where action A is to continue to expand capacity or leave the site; Execute the selected action A; Get the new state S′={D′,U′,E′} and reward R of the environment feedback; Update the Q value table or neural network parameters according to the new state S′ and reward R; Based on the new state S′ and the preset stay / leave judgment condition, it is determined whether to continue capacity expansion or leave the site.

10. The method according to claim 9, characterized in that The stay / leave judgment conditions include: When the current traffic density is greater than the first threshold, or the current PRB utilization is greater than the second threshold, or the user experience score is greater than the third threshold, and continues for a period of time T, take the action of continuing to expand capacity; and, When the current crowd density is less than the fifth threshold, the current PRB utilization is less than the sixth threshold, and the user experience score is less than or equal to the third threshold, and the situation lasts for a period of time T, take the action of leaving the site; Among them, the first threshold is greater than the fifth threshold, and the second threshold is greater than the sixth threshold.

11. A communication operator network optimization device, characterized in that: The device is arranged on a signal optimization vehicle, and comprises: An expansion area acquisition module is used to acquire at least one area that needs to be expanded in an application scenario; a priority calculation module, connected to the expansion area acquisition module, for calculating the priority of each area according to the current flow density, predicted number of users, current network load, user experience score and service type score of each area of ​​the at least one area requiring expansion; A trolley path planning module, connected to the priority calculation module, is used to perform path planning for at least one area that needs to be expanded based on the order of the priorities using the A-star algorithm to obtain an optimal path for the movement of the signal optimization trolley; The car network optimization module is connected to the car path planning module and is used to use the reinforcement learning RL method to decide whether to continue to expand capacity or leave the site in the current area when the signal optimization car is driving according to the optimal path.

12. A communication operator network optimization device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the communication operator network optimization method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the communication operator network optimization method according to any one of claims 1 to 10 is implemented.

Images