Parameter adjustment method and device, computer readable storage medium and electronic equipment

Abstract

The invention discloses a parameter adjustment method and device, a computer readable storage medium and electronic equipment. The method comprises the following steps: acquiring a current network parameter and a current network performance index parameter of a target network; respectively selecting actions for the plurality of access points from the action set to obtain an action selection set corresponding to the target network; based on the action selection set, adjusting a current network parameter corresponding to the target networking to obtain an adjusted network parameter, and obtaining a network performance index parameter under the adjusted network parameter; according to the adjusted network performance index parameters of the target networking, adjusting the action selection set to obtain an adjusted action selection set; and based on the adjusted action selection set, adjusting a network parameter corresponding to the target network, iterating for multiple times until an iteration ending condition is reached, and determining a target network parameter. The technical problem that the overall network performance of networking is not high due to the fact that adjustment of current network parameters pays more attention to the network performance of the access point is solved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a parameter adjustment method, device, computer-readable storage medium and electronic device. Background Art

[0002] In a Wi-Fi network consisting of multiple APs, intertwined wireless signals may interfere with the normal operation of the APs. Interference between APs can be reduced by adjusting parameters such as power, channel, and CCA threshold. However, excessive protection may reduce the service capacity of the network.

[0003] In the parameter adjustment of strong competitive relationships, if a single node sets parameters in a self-interested direction, it will interfere with the normal operation of other nodes and cause the performance of the entire network to deteriorate.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a parameter adjustment method, device, computer-readable storage medium and electronic device to at least solve the technical problem that the current adjustment of network parameters focuses more on the network performance of access points, resulting in low overall network performance of the network.

[0006] According to one aspect of an embodiment of the present invention, a parameter adjustment method is provided, including: obtaining current network parameters and current network performance indicator parameters corresponding to a target network, wherein the target network includes multiple access points, and the current network parameters of the target network represent current network parameter sets corresponding to the multiple access points; selecting actions for the multiple access points respectively from an action set to obtain an action selection set corresponding to the target network; adjusting the current network parameters corresponding to the target network based on the action selection set to obtain adjusted network parameters corresponding to the target network, and obtaining network performance indicator parameters of the target network under the adjusted network parameters; adjusting the action selection set according to the adjusted network performance indicator parameters of the target network and the current network performance indicator parameters of the target network to obtain an adjusted action selection set; adjusting the network parameters corresponding to the target network based on the adjusted action selection set, iterating multiple times until an iteration end condition is met, and determining the target network parameters corresponding to the target network.

[0007] Optionally, the action selection set is adjusted according to the adjusted network performance indicator parameters of the target networking and the current network performance indicator parameters of the target networking to obtain an adjusted action selection set, including: determining a reward function based on the network performance indicator parameters of the target networking; obtaining a reward value corresponding to the action selection set according to the current network performance indicator parameters of the target networking, the adjusted network performance indicator parameters of the target networking and the reward function; adjusting the action selection set according to the reward value corresponding to the action selection set to obtain an adjusted action selection set.

[0008] Optionally, determining a reward function based on the network performance indicator parameters of the target networking includes: setting influence weights corresponding to the network performance indicator parameters of the target networking and the network performance indicator parameters of the access point; generating a first function item according to a change trend of the network performance indicator parameters corresponding to the target networking; generating a second function item according to a change trend of the network performance indicator parameters of the access point; and determining a reward function based on the influence weights, the first function item, and the second function item.

[0009] Optionally, according to the reward value corresponding to the action selection set, the action selection set is adjusted to obtain an adjusted action selection set, including: determining whether the reward value corresponding to the action selection set is greater than a first preset threshold; when the reward value corresponding to the action selection set is greater than the first preset threshold, keeping the action selection set unchanged, and adjusting the network parameters corresponding to each of the multiple access points based on the action selection set; and / or, when the reward value corresponding to the action selection set is not greater than the first preset threshold, adjusting the actions in the action selection set to obtain an adjusted action selection set.

[0010] Optionally, when the network performance indicator parameter is throughput, the method for obtaining the network performance indicator parameter corresponding to any one of the multiple access points is as follows: obtaining the sending waiting time and the data transmission time of the access point during the data sending process; and determining the throughput of the access point based on the sending waiting time and the data transmission time.

[0011] Optionally, determining the throughput of the access point based on the sending waiting time and the data transmission time includes: calculating the throughput of each access point in the target network by the following target formula; ,in, is the throughput of the access point in the target network, is the data transmission rate corresponding to the access point in the target network, is the packet error rate corresponding to the access point in the target network. The data transmission rate corresponding to the access point in the target network and the packet error rate corresponding to the access point in the target network are related to the link quality corresponding to the data transmission of the access point in the target network. To send the waiting time, The data transmission duration.

[0012] Optionally, when the network performance indicator parameter is throughput, the action selection set is adjusted according to the adjusted network performance indicator parameter of the target network and the current network performance indicator parameter of the target network to obtain an adjusted action selection set, including: calculating the difference between the adjusted throughput of the target network and the current throughput of the target network as a first difference; when the first difference is greater than a second preset threshold, keeping the action selection set unchanged; and / or, when the first difference is not greater than the second preset threshold, adjusting the action selection set to obtain an adjusted action selection set.

[0013] Optionally, when the network performance indicator parameter is throughput, adjusting the action selection set according to the adjusted network performance indicator parameter of the target network and the current network performance indicator parameter of the target network to obtain the adjusted action selection set includes: calculating the difference between the adjusted throughput of the target network and the current throughput of the target network as a first difference; determining the average difference according to the first difference and the number of access points of the multiple access points; calculating the difference between the adjusted throughput and the current throughput corresponding to each of the multiple access points to obtain a second difference corresponding to each of the multiple access points; when there is a second difference corresponding to a first access point among the multiple access points and the sum of the average difference is greater than a third preset threshold, keeping the action corresponding to the first access point in the action selection set unchanged; and / or, when there is a second difference corresponding to a second access point among the multiple access points and the sum of the average difference is not greater than the third preset threshold, adjusting the action corresponding to the second access point in the action selection set to obtain the adjusted action selection set.

[0014] Optionally, the action set includes: the network parameter of the access point remains unchanged, the network parameter of the access point increases by a first preset value, and the network parameter of the access point decreases by a second preset value.

[0015] Optionally, the method further includes: determining the adjusted network parameters corresponding to each of the multiple access points according to the adjusted network parameters corresponding to the target networking; and sending the adjusted network parameters corresponding to each of the multiple access points to the corresponding access points.

[0016] According to another aspect of an embodiment of the present invention, a parameter adjustment device is further provided, including: an acquisition module, configured to acquire current network parameters and current network performance indicator parameters corresponding to a target network, wherein the target network includes multiple access points, and the current network parameters of the target network represent current network parameter sets corresponding to the multiple access points; a selection module, configured to select actions for the multiple access points from an action set, respectively, to obtain an action selection set corresponding to the target network; a first adjustment module, configured to adjust the current network parameters corresponding to the target network based on the action selection set, to obtain adjusted network parameters corresponding to the target network, and to obtain network performance indicator parameters of the target network under the adjusted network parameters; a second adjustment module, configured to adjust the action selection set according to the adjusted network performance indicator parameters of the target network and the current network performance indicator parameters of the target network, to obtain an adjusted action selection set; an iteration module, configured to adjust the network parameters corresponding to the target network based on the adjusted action selection set, iterate multiple times until an iteration end condition is reached, and determine the target network parameters corresponding to the target network.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored executable program, wherein when the executable program runs, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned parameter adjustment methods.

[0018] According to another aspect of the present invention, there is provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein any one of the above parameter adjustment methods is executed when the program is running.

[0019] According to yet another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, any one of the above parameter adjustment methods is implemented.

[0020] In an embodiment of the present invention, a parameter adjustment method is adopted, by obtaining current network parameters and current network performance index parameters corresponding to a target network, wherein the target network includes multiple access points, and the current network parameters of the target network represent current network parameter sets corresponding to the multiple access points; actions are selected for the multiple access points from an action set to obtain an action selection set corresponding to the target network; based on the action selection set, the current network parameters corresponding to the target network are adjusted to obtain adjusted network parameters corresponding to the target network, and network performance index parameters of the target network under the adjusted network parameters are obtained; according to the adjusted network performance index parameters of the target network and the current network performance index parameters of the target network, the action selection set is adjusted to obtain an adjusted action selection set; based on the adjusted action selection set, the network parameters corresponding to the target network are adjusted, iterated multiple times until the iteration end condition is reached, and the target network parameters corresponding to the target network are determined, so as to achieve the purpose of adaptively adjusting the network parameters of the access points in the network taking into account the network performance of the network, thereby achieving the technical effect of improving the network performance of the network, and further solving the technical problem that the current adjustment of network parameters pays more attention to the network performance of the access points, resulting in low overall network performance of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0022] Figure 1 is a flow chart of a parameter adjustment method provided according to an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of a data sending process of an access point provided in an optional embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of a reinforcement learning state transfer process provided according to an optional embodiment of the present invention;

[0025] Figure 4 is a flow chart of a Q learning algorithm provided according to an optional embodiment of the present invention;

[0026] Figure 5 is a structural block diagram of a parameter adjustment device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0030] An access point (AP) is a key device in a wireless network. It acts as a bridge between a wireless local area network (WLAN) and a wired network, allowing wireless devices such as smartphones, laptops, and tablets to connect to a wired network. An access point typically contains a wireless transceiver and an antenna that can receive and send wireless signals. Its job is to convert wireless signals into wired network signals and vice versa, enabling communication between wireless devices and wired networks. In large wireless network deployments, multiple access points can be configured to cover a larger area, while being coordinated through a network controller to optimize the quality of wireless signals and network performance. In patent writing or technical discussions, an access point is a core component in a wireless network architecture, and its performance and configuration directly affect the stability and efficiency of the entire network.

[0031] An access point controller (AC) is a network device that is mainly used to manage multiple access points (APs) in a large-scale wireless local area network (WLAN).

[0032] Network performance indicators are any indicators that can characterize network performance, such as throughput, latency performance, retransmission rate, average backoff number, and some composite indicators (such as txrate, per, txop, etc.); among them, the throughput of an access point (AP) refers to the amount of data that the AP can successfully transmit to terminal devices within its coverage range within a specific time. It is an important indicator for measuring wireless network performance, usually expressed in bits per second (bps) or higher units such as megabits per second (Mbps) and gigabits per second (Gbps). Network performance indicator parameters refer to the specific values ​​or conditions used to calculate or evaluate the above network performance indicators. They are the basic data that constitute and quantify network performance indicators.

[0033] The Q-learning algorithm is a model-free algorithm widely used in the field of machine learning, especially in reinforcement learning. It allows the agent to learn an action-value function, namely the Q function, in the process of interacting with the environment to determine the best action to take in a given state. The Q function evaluates the expected value of the subsequent reward that can be obtained after performing a specific action in a specific state.

[0034] Link quality in wireless networks usually refers to the performance of wireless communication links. It can be measured from multiple perspectives, and common indicators include signal strength, signal-to-noise ratio (SNR), bit error rate (BER), packet loss rate, latency, etc. Link quality reflects the reliability and efficiency of data transmission on wireless channels, and directly affects the throughput, coverage, and user experience of wireless networks.

[0035] Packet Error Rate (PER) is a measure of the proportion of errors in data packets during transmission, usually defined as the ratio of the number of erroneous data packets to the total number of transmitted data packets. The packet error rate is directly related to the link quality. Link quality reflects the reliability of data transmission between two communication nodes, which is affected by many factors, including signal strength, signal-to-noise ratio, interference level, multipath effect, and physical layer coding and modulation technology. In wireless networks, poor link quality will lead to more errors in data packets during transmission, which will directly affect PER.

[0036] The network includes multiple access points, and the intertwined wireless signals will interfere with the normal operation of the access points. The interference between the access points can be reduced by adjusting the network parameters, thereby improving the network performance of the access points. Network deployment is to apply a set of theoretically analyzed and verified effective rules to the network during networking, so that the access points in the network can work more efficiently. However, in networking, there are many parameters involved, which are interdependent and influential, and the wireless environment has large external interference. Therefore, it is difficult to derive a set of rules from the communication network knowledge, and the complexity of the solution is superlinear. As the network scale increases, the difficulty increases rapidly. Moreover, the current access point network parameter adjustment depends on the periodic or triggered execution of the rules set in advance, so it is difficult to face the changes in network conditions, and it is impossible to adjust the network parameters adaptively. In addition, it only takes into account the network performance of a single access point and ignores the network performance of the network, which is prone to poor network performance of the network. In view of the above technical problems, an embodiment of the present invention proposes a parameter adjustment method. According to an embodiment of the present invention, a method embodiment of a parameter adjustment method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Figure 1 is a parameter adjustment method according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0038] Step S102, obtaining current network parameters and current network performance indicator parameters corresponding to the target network, wherein the target network includes multiple access points, and the current network parameters of the target network represent current network parameter sets corresponding to the multiple access points.

[0039] Because when adjusting the network parameters of the access point, the performance of the entire network needs to be considered, so the network parameters of the access point need to be adjusted with the help of the network access point controller (AC). The implementation method can be to maintain a global reinforcement learning process on the AC and then issue the actions to be executed, or the AC can collect the network performance indicator parameter information of the access point (AP) in the network and then share the network performance indicator parameter information of the network to the access point.

[0040] In this step, the execution subject of the method of this embodiment may be an AC. The AC may be used to collect information of all access points in the network and adjust the network parameters of the access points. In the wireless network architecture, the AC may be in the following device forms:

[0041] Hardware device: AC can be an independent physical device used to centrally manage large-scale wireless access points (APs). This device usually has a high-performance processor, large memory, and sufficient storage space to support the configuration, monitoring, and troubleshooting of many APs.

[0042] Software application: AC can also be a software application running on a standard server. This application can be a dedicated software provided by the manufacturer or a solution from the open source community. Software AC can be flexibly deployed on existing servers without the need for additional hardware investment.

[0043] Virtualized devices: With the development of virtualization technology, AC can be implemented as a virtual machine (VM) or container and run in a cloud platform or local virtualized environment. This implementation method can make full use of the resource management and elastic scaling capabilities of the virtualized environment and dynamically adjust the resource allocation of AC according to network requirements.

[0044] Integrated in routers or switches: Some high-end network devices, such as routers or switches, may have built-in AC functions. This integrated design can simplify the network architecture, reduce communication delays between devices, and improve management efficiency.

[0045] The main responsibilities of AC include but are not limited to configuring and managing APs, optimizing wireless network performance, monitoring network status, providing security policies, and supporting roaming and load balancing. Through centralized control and management, AC can ensure the stability and efficiency of the wireless network and improve the automation level of network maintenance.

[0046] The AC can obtain the current network parameters and current network performance index parameters in the target network, wherein the current network parameters of the target network can be regarded as a collection of current network parameters of multiple access points in the target network. The current network performance index parameters of the target network are also determined according to the current network performance index parameters corresponding to the multiple access points. Among them, the network performance index parameters can reflect the network performance of the access point and the network. When the network performance index parameter is throughput, the greater the throughput, the better the network performance. When the network performance index parameter is retransmission rate, the smaller the retransmission rate, the better the network performance.

[0047] Step S104: Select actions for the multiple access points from the action set to obtain an action selection set corresponding to the target network.

[0048] In this step, the action set may include multiple actions for adjusting network parameters. An action may be randomly selected to adjust the current network parameters of the access point, and the adjusted network parameters may be obtained. Therefore, when the target network includes multiple access points, an action may be selected from the action set for each access point, and an action selection set corresponding to the target network may be obtained. The action selection may be based on the ε-greedy (epsilon-greedy) strategy, where ε-greedy is a method for making a compromise between exploration and exploitation in reinforcement learning. In this strategy, the agent acts most of the time according to the currently known best strategy (to take advantage of the knowledge it has learned; but in a small part of the time, the agent randomly chooses an action to explore other possibilities in the environment, thereby updating its understanding and strategy of the environment. Specifically, the ε-greedy strategy works as follows: a probability value ε (epsilon) between 0 and 1 is selected. Usually this value is small, such as 0.1, indicating that 10% of the time is spent on exploration. At each decision point, the agent randomly chooses an action with a probability of ε, even if this action may not be optimal at the moment. Through continuous trial and learning, the agent gradually reduces the number of random explorations (i.e., reduces the ε value) and relies more on the learned strategy, thereby finding a balance between exploration and utilization.

[0049] The action set may include: keeping the network parameter of the access point unchanged, increasing the network parameter of the access point by a first preset value, and decreasing the network parameter of the access point by a second preset value.

[0050] The action set includes an adjustment action for the network parameters of the access point, which may specifically include keeping the network parameters of the access point unchanged, increasing the network parameters and decreasing the network parameters, wherein the specific values ​​of increasing the network parameters and decreasing the network parameters can be set according to actual conditions. The smaller the first preset value and the second preset value are set, the more accurate the adjustment of the network parameters will be, but the corresponding adjustment speed will be slower.

[0051] Step S106: Based on the action selection set, adjust the current network parameters corresponding to the target network, obtain the adjusted network parameters corresponding to the target network, and obtain the network performance indicator parameters of the target network under the adjusted network parameters.

[0052] In this step, the current network parameters corresponding to the target network, that is, the current network parameters corresponding to each of the multiple access points are adjusted based on the action selection set, so that the adjusted network parameters corresponding to the target network can be obtained, and the network performance indicator parameters of the target network under the adjusted network parameters can be obtained.

[0053] Step S108, adjusting the action selection set according to the adjusted network performance indicator parameters of the target network and the current network performance indicator parameters of the target network to obtain an adjusted action selection set.

[0054] In this step, the action selection set can be adjusted according to the adjusted network performance indicator parameters of the target network and the current network performance indicator parameters of the target network. Because the change of the network performance indicator parameters corresponding to the target network can reflect the effect of the adjustment action in the action selection set. When the network performance indicator parameter is throughput, if the adjusted throughput of the target network is greater than the current throughput of the target network, it can be considered that the action effect in the previous action selection set is better, which is the correct adjustment direction. If the adjusted throughput of the target network is not greater than the current throughput of the target network, it is considered that the action adjustment effect in the previous action selection set is not good, and the adjustment direction is wrong. Therefore, the actions in the action selection set can be adjusted to obtain the adjusted action selection set. When the network performance indicator parameter is retransmission rate, if the adjusted retransmission rate of the target network is less than the current retransmission rate of the target network, it can be considered that the action effect in the previous action selection set is better, which is the correct adjustment direction. If the adjusted retransmission rate of the target network is not less than the current retransmission rate of the target network, it is considered that the action adjustment effect in the previous action selection set is not good, and the adjustment direction is wrong.

[0055] As an optional embodiment, when the network performance indicator parameter is throughput, the action selection set is adjusted according to the adjusted network performance indicator parameter of the target network and the current network performance indicator parameter of the target network to obtain an adjusted action selection set, including: calculating the difference between the adjusted throughput of the target network and the current throughput of the target network as a first difference; when the first difference is greater than a second preset threshold, keeping the action selection set unchanged; and / or, when the first difference is not greater than the second preset threshold, adjusting the action selection set to obtain an adjusted action selection set.

[0056] Optionally, the difference between the adjusted throughput of the target network and the current throughput can be calculated as the first difference. When the first difference is greater than the second preset threshold, it can be considered that the action adjustment effect in the action selection set is good and the action selection is reasonable, so the action selection set can be kept unchanged, and the previous action selection set is still used in the next adjustment. When the first difference is not greater than the second preset threshold, it can be considered that the action adjustment effect in the action selection set is not good and the action selection is not reasonable, so the action selection set can be adjusted, and all actions in the action selection set can be adjusted, or a part of the actions can be adjusted to obtain an adjusted action selection set.

[0057] Optionally, when the network performance indicator parameter is the retransmission rate, the difference between the adjusted retransmission rate of the target network and the current retransmission rate of the target network can be calculated, and the difference can be compared with a threshold value corresponding to a preset retransmission rate. When the difference is less than the threshold value, the action selection set is kept unchanged; and / or, when the difference is not less than the threshold value, the action selection set is adjusted to obtain an adjusted action selection set.

[0058] As an optional embodiment, in the case where the network performance indicator parameter is throughput, adjusting the action selection set according to the adjusted network performance indicator parameter of the target network and the current network performance indicator parameter of the target network to obtain the adjusted action selection set includes: calculating the difference between the adjusted throughput of the target network and the current throughput of the target network as a first difference; determining the average difference according to the first difference and the number of access points of the multiple access points; calculating the difference between the adjusted throughput and the current throughput corresponding to each of the multiple access points to obtain the second difference corresponding to each of the multiple access points; in the case where the sum of the second difference corresponding to the first access point and the average difference is greater than a third preset threshold value, keeping the action corresponding to the first access point in the action selection set unchanged; and / or, in the case where the sum of the second difference corresponding to the second access point and the average difference is not greater than the third preset threshold value, adjusting the action corresponding to the second access point in the action selection set to obtain the adjusted action selection set.

[0059] Optionally, in addition to considering the network performance of the target network, the network performance of the access point can also be comprehensively considered. The average difference can be determined based on the first difference and the number of access points. Then, based on the average difference and the second difference between the adjusted network parameter corresponding to the access point and the current network parameter, the action selection set is adjusted. If the sum of the second difference corresponding to the first access point and the average difference is greater than the third preset threshold, it is considered that the action adjustment for the first access point in the action selection set is more reasonable, so the adjustment action of the first access point can be kept unchanged in the next adjustment of the action selection set. If the sum of the second difference corresponding to the second access point and the average difference is not greater than the third preset threshold, it is considered that the action adjustment for the second access point in the action selection set is not reasonable enough, so in the next adjustment of the action selection set, the adjustment action of the second access point can be selected to adjust, and an action different from the previous one is selected to adjust the network parameters of the access point to obtain the adjusted action selection set. By adjusting the action selection according to the throughput changes of the access point and the throughput changes of the target network, the performance of the entire network and the network performance of the access point are comprehensively considered, avoiding the situation in which a single node sets parameters in a self-interested direction during parameter adjustment with strong competitive relationships, which will interfere with the normal operation of other nodes and cause the performance of the entire network to deteriorate.

[0060] Step S110, based on the adjusted action selection set, adjusting the network parameters corresponding to the target networking, iterating multiple times until the iteration end condition is met, and determining the target network parameters corresponding to the target networking.

[0061] In this step, the network parameters corresponding to the target network can be adjusted based on the adjusted action selection set, and multiple iterations can be performed until the iteration end condition is reached, and the target network parameters at this time can be determined as the target network parameters corresponding to the target network. At this time, it can be considered that the access point has achieved good performance, and if the action selection continues in the future, the network parameters can be kept unchanged.

[0062] There are many conditions for the end of iteration. For example, when the reinforcement learning algorithm is used for iteration, the end condition of iteration can be related to the convergence standard of the reinforcement learning algorithm. Specifically, a preset number of iterations can be set. When the preset number of iterations is reached, the condition for the end of iteration is considered to be reached. When the learning process starts, a maximum number of iterations can be set. When the algorithm iteration reaches this number, the iteration stops and the target network parameters corresponding to the target network are determined. The end condition of iteration can also be determined based on the convergence of reward changes. If the reward obtained after adjusting the parameters according to the learning algorithm changes very little in multiple consecutive iterations, or the absolute value of the reward change is lower than a preset threshold, this usually indicates that the learning process has converged and the parameter adjustment has stabilized, and the iteration can be ended. The key performance indicators such as the network performance indicator parameters of the access point and the target network can also be monitored through the network performance evaluation algorithm. If these performance indicators have not changed significantly over a continuous period of time, it indicates that the network is already in a relatively stable and optimized state, and the iteration can also be considered to end.

[0063] Through the above steps, the purpose of adaptively adjusting the network parameters of the access points in the networking taking into account the network performance of the networking is achieved, thereby achieving the technical effect of improving the network performance of the networking, and further solving the technical problem that the current adjustment of network parameters pays more attention to the network performance of the access points, resulting in low overall network performance of the networking.

[0064] As an optional embodiment, the action selection set is adjusted according to the adjusted network performance indicator parameters of the target networking and the current network performance indicator parameters of the target networking to obtain the adjusted action selection set, including: determining a reward function based on the network performance indicator parameters of the target networking; obtaining a reward value corresponding to the action selection set according to the current network performance indicator parameters of the target networking, the adjusted network performance indicator parameters of the target networking and the reward function; adjusting the action selection set according to the reward value corresponding to the action selection set to obtain the adjusted action selection set.

[0065] Optionally, a reinforcement learning algorithm can be used to adjust the network parameters. Among them, the reinforcement learning algorithm usually sets a reward function, and uses the reward function to judge whether the action selection is reasonable. Reinforcement learning mainly focuses on how to enable the agent to learn the optimal behavior strategy in the interaction with the environment. The agent here is the target network, where the target network includes multiple access points. After the AC obtains the current network parameters and current network performance index parameters of the target network, it can select multiple actions to act on each access point, and the network parameters of each access point will be adjusted according to the corresponding actions. The core idea of ​​reinforcement learning is that the agent observes the feedback of the environment (usually in the form of rewards or penalties) by performing a series of actions in the environment, and adjusts its behavior strategy according to these feedback signals to maximize the cumulative reward. Specifically, the Q learning algorithm in reinforcement learning can be used, where the Q learning algorithm contains three elements, namely state, action and reward. When the agent is in a certain state, it faces multiple actions. Executing each action will transfer to another corresponding state, and will also receive corresponding rewards. The ultimate goal of the agent is to maximize the reward. Therefore, after continuous iterative learning, the agent will eventually learn an optimal path or an optimal state. Here, the state can be the current network parameters corresponding to the target network, the action set is the action space, and the reward is obtained based on the changes in the network performance indicator parameters of the target network after executing the action.

[0066] Therefore, a reward function related to the network performance index parameters can be set, and then the current network performance index parameters of the target network and the adjusted network performance index parameters can be substituted into the reward function to obtain the reward value of the action selection set corresponding to the last adjustment of the network parameters. The action selection set can be adjusted according to the reward value.

[0067] For example, substitute the current network performance index parameters of the target network and the adjusted network performance index parameters of the target network into the reward function to obtain the reward value corresponding to the action selection set. If the corresponding reward value is greater than 0, the current action selection set can be considered a reasonable choice, so the actions in the action selection set can still be retained when the action selection is performed next time. If the corresponding reward value is not greater than 0, the current action selection set can be considered unreasonable, so the next time the action selection is performed, you can try to select an action selection set composed of different actions from the previous one.

[0068] For another example, the reward value of the adjustment action corresponding to each access point in the target network can be determined based on the current network performance index parameter of the target network and the adjusted network performance index parameter of the target network and the current network performance index parameter and the adjusted network performance index parameter corresponding to each access point. When the reward value corresponding to the adjustment action is greater than 0, it can be considered that the selection of the adjustment action is more reasonable, and the action can still be selected to adjust the network parameters of the access point next time. When the reward value corresponding to the adjustment action is not greater than 0, it can be considered that the adjustment action is not reasonable, and actions other than the action can be selected for adjustment next time to obtain a collection of adjusted action selections.

[0069] Figure 3 is a schematic diagram of a reinforcement learning state transfer process according to an optional embodiment of the present invention. Figure 3 As shown, the target network is in a state S, and it can choose to perform a series of possible actions A1, A2, ..., A n When action A is executed i (A i Can be A1 to A n After performing any action in the target network), the target network will transfer to a new state S', and according to the current state transfer and action, the environment will give a reward R, where the environment can be the external system where the access point in the target network is located, and the environment corresponding to each of the multiple access points can be regarded as the environment corresponding to the target network. When the state of the access point changes, causing the state of the target network to change, the environment will change accordingly, and then the reward can be determined. For example, the environment here can be the network environment where the access point is located. After the state of the access point changes, the network environment where it is located will also change, and the corresponding state of the target network will also be affected. Through the interaction between the multiple access points included in the target network and the environment, that is, performing different actions, observing the changes in the environmental states corresponding to the multiple access points and the feedback of the rewards, and then determining the overall environmental state changes and reward feedback corresponding to the target network, the target network can learn how to act in the environment to achieve optimal or near-optimal performance. The reward given can be a positive number, indicating that it is beneficial to the target network, or a negative number, indicating that it is not beneficial to the target network. The goal of the target network is to learn a strategy π by constantly trying different actions, so that starting from the initial state, after a series of state transfers, the cumulative reward is maximized. Specifically, Figure 3The target network will be in state S0. At this time, Action1 can be selected from the action set for execution. Here, Action1 can be the action selection set, and the adjusted network parameters, that is, state S1, can be obtained from state S0 and state S1. The reward value Reward1 can be obtained. When Action2 is selected from the action set for execution, here Action2 can be the action selection set, and the adjusted network parameters, that is, state S2, can be obtained from state S2 and state S0. The reward value Reward2 can be obtained. After continuous iteration, an optimal path can be learned. The optimal state of the network parameters obtained at the end is the target network parameters corresponding to the target network.

[0070] The reward function may have a dependent variable that is a reward value and an independent variable that is a network performance indicator parameter of the target network and the number of access points of the multiple access points in the target network.

[0071] Optionally, the reward function can be set to use the value of the change in the network performance indicator parameter of the access point and the target network as the reward for the current action. Specifically, when the network performance indicator parameter is throughput, the reward function is as follows:

[0072]

[0073] in, is the reward value, is the value of the access point performance change, that is, the access point throughput change value, It is the value of the performance change of the entire network, that is, the throughput change value corresponding to the target network. is the number of access points in the target network, is the step size of adjusting the parameters, that is, the value of each adjustment. When the network parameters of the access point are adjusted, the reward for this action is the performance change of the access point itself plus the average performance change of the access points in the target network. When the access point does not adjust the parameters, the reward obtained is the average performance change of the access points in the target network.

[0074] As an optional embodiment, a reward function is determined based on the network performance indicator parameters of the target network, including: setting influence weights corresponding to the network performance indicator parameters of the target network and the network performance indicator parameters of the access point; generating a first function item according to a change trend of the network performance indicator parameters corresponding to the target network; generating a second function item according to a change trend of the network performance indicator parameters of the access point; and determining a reward function based on the influence weights, the first function item, and the second function item.

[0075] Optionally, the reward function can also set different influence weights for the network performance indicator parameters of the target network and the network performance indicator parameters of the access point. When more attention is paid to the overall network performance of the target network, the influence weight of the network performance indicator parameters corresponding to the target network can be set higher, while the influence weight of the network performance indicator parameters of the access point can be set lower. When the overall network performance change accounts for a larger proportion in the reward calculation, the access points in the target network can be driven to adjust parameters in the direction of higher overall network performance. The first function item can be set according to the change trend of the network performance indicator parameters corresponding to the target network, and the second function item can be set according to the change trend of the network performance indicator parameters of the access point. The reward function can be determined based on the influence weight, the first function item, and the second function item. For example, in the case where the network performance indicator parameter is throughput, the first function item can be set to:

[0076]

[0077] in, is the throughput variation of the access point, It is assumed that the throughput trend of the access point is increasing, and rewards can be given at this time, so the reward value can be set to a positive number. The throughput trend of the access point is decreasing, and a penalty can be imposed at this time, so the reward value can be set to a negative number.

[0078] When the network performance indicator parameter is throughput, the second function item can be set to:

[0079]

[0080] in, is the throughput variation of the access point, It is assumed that the throughput trend of the access point is increasing, and rewards can be given at this time, so the reward value can be set to a positive number. The throughput trend of the access point is decreasing, and a penalty can be imposed at this time, so the reward value can be set to a negative number.

[0081] Based on the influence weight, the first function term and the second function term, the reward function can be obtained , where m and n are the impact weights of the target network throughput and access point throughput, respectively.

[0082] As an optional embodiment, adjusting the action selection set according to the reward value corresponding to the action selection set to obtain the adjusted action selection set includes: determining whether the reward value corresponding to the action selection set is greater than a first preset threshold; when the reward value corresponding to the action selection set is greater than the first preset threshold, keeping the action selection set unchanged, and adjusting the network parameters corresponding to each of the multiple access points based on the action selection set; and / or, when the reward value corresponding to the action selection set is not greater than the first preset threshold, adjusting the actions in the action selection set to obtain the adjusted action selection set.

[0083] Optionally, the action selection set is further adjusted according to the reward value, thereby adjusting the network parameters of the target network. The action selection set can be adjusted according to the size of the reward value. When the reward value is greater than the first preset threshold, it can be considered that the adjustment action selected last time is correct, so the last action can still be selected this time to adjust the network parameters of the access point. When the reward value is not greater than the first preset threshold, it is considered that the action selected last time is unreasonable, so an action other than the last action can be selected to continue adjusting the network parameters of the access point. The first preset threshold can be set to 0. When the reward value is greater than 0, the adjustment direction of the action is considered to be correct. When the reward value is not greater than 0, the adjustment direction of the action is considered to be wrong. Through continuous iterative adjustment, an optimal path will eventually be learned. At this time, the target network parameters of the access point can be considered to be an optimal state. If learning continues at this time, the action selection is to keep the current network parameters unchanged.

[0084] As an optional embodiment, when the network performance indicator parameter is throughput, the method for obtaining the network performance indicator parameter of the target network is as follows: obtain the throughput corresponding to each of multiple access points in the target network; calculate the sum of the throughput corresponding to each of the multiple access points to determine the throughput of the target network.

[0085] Optionally, the network throughput refers to the amount of data that can be transmitted by the entire network per unit time, and the access point throughput refers to the amount of data that can be transmitted by a single access point per unit time. In a wireless network consisting of multiple access points, the throughput of each access point will affect the throughput performance of the entire network. Among them, the throughput of the target network is equal to the sum of the throughputs of the access points in the target network. Therefore, by obtaining the throughputs corresponding to each of the multiple access points, the throughput of the target network can be determined. In an environment with multiple access points, the allocation and competition of resources (such as channels and power) will affect the throughput of each access point. If the resource allocation is unreasonable or the competition is fierce, the throughput of some access points may decrease, thereby affecting the throughput of the overall network. Therefore, the throughput of the access point is closely related to the throughput of the target network, and the throughputs of multiple access points will also affect each other.

[0086] As an optional embodiment, a method for obtaining the throughput corresponding to any one of the multiple access points is as follows: obtaining the sending waiting time and the data transmission time of the access point during the data sending process; and determining the throughput of the access point based on the sending waiting time and the data transmission time.

[0087] Optionally, the throughput is the maximum transmission capacity that the access point can provide in the current network state. However, in actual scenarios, since the access point is not always in a full-load working state, the throughput of the access point is difficult to obtain directly. The states of the access point usually include three types: sending waiting state, data transmission state, and idle state. Among them, the sending waiting state means that when the access point has a sending demand, it needs to back off and wait because the current channel is busy. When the access point obtains the right to use the channel, it is in the data transmission state and the access point can transmit data. The idle state means that the current access point has no data packets to send. When there is a sending demand, the proportion of the transmission time per unit time determines the performance of the access point, that is, the throughput, so the throughput can be determined based on the sending waiting time and the data transmission time. The sending waiting time and the data transmission time of the access point sending data in the historical process can be obtained. Among them, if you want to obtain the throughput under the current network parameters, you need to obtain the data sending waiting time and the data transmission time of the access point under the current network parameters. If you want to obtain the throughput under the adjusted network parameters, you need to obtain the sending waiting time and the data transmission time under the adjusted network parameters. Then the throughput of the access point can be determined based on the sending waiting time and the data transmission time.

[0088] Figure 2 is a schematic diagram of a data transmission process of an access point provided according to an optional embodiment of the present invention, such as Figure 2 As shown, IDLE is an idle state, that is, the access point has no sending requirements at this time. It is the length of time that the access point is in the sending waiting state, that is, the sending waiting time. At this time, the access point has a sending demand, but the channel is busy, so it needs to wait. It is the data transmission duration. At this time, the access point is in the data transmission state. The access point obtains the right to use the channel and can send data.

[0089] As an optional embodiment, determining the throughput of the access point based on the sending waiting time and the data transmission time includes: calculating the throughput of each access point in the target network by the following target formula; ,in, is the throughput of the access point in the target network, is the data transmission rate corresponding to the access point in the target network, is the packet error rate corresponding to the access point in the target network. The data transmission rate corresponding to the access point in the target network and the packet error rate corresponding to the access point in the target network are related to the link quality corresponding to the data transmission of the access point in the target network. To send the waiting time, The data transmission duration.

[0090] Optionally, a target formula may be set, which is related to the sending waiting time, the data transmission time, and the link quality corresponding to the access point transmitting data. Then, the sending waiting time and the transmission time may be substituted into the target formula to obtain the throughput of the access point. The target formula may be expressed as:

[0091]

[0092] in, Indicates link quality. and They are the waiting time for sending and the data transmission time. In wireless networks, link quality usually refers to the stability and efficiency of wireless connections. It can be evaluated from multiple dimensions, including but not limited to signal strength, signal quality, data transmission rate, packet error rate, etc. In wireless network parameter optimization, link quality is a very critical indicator because it directly affects the efficiency and stability of data transmission. By optimizing network parameters such as power, channel, CCA threshold, etc., link quality can be improved, thereby improving the overall performance of the network.

[0093] Specifically, the target formula is: ,in, is the throughput of the access point in the target network, is the data transmission rate corresponding to the access point in the target network, is the packet error rate corresponding to the access point in the target network. The data transmission rate corresponding to the access point in the target network and the packet error rate corresponding to the access point in the target network are related to the link quality corresponding to the data transmission of the access point in the target network. To send the waiting time, The data transmission duration.

[0094] Optionally, the packet error rate and the data transmission rate corresponding to the access point are both related to the link quality, wherein the higher the link quality, the lower the packet error rate. In a wireless network, link quality is usually measured by indicators such as signal strength, signal quality (such as signal-to-noise ratio SNR), and packet loss rate. These factors directly affect whether a data packet can be successfully transmitted and the transmission rate. Therefore, based on the above parameters, the throughput of the access point in actual situations can be calculated, thereby making the adjustment process of network parameters more accurate.

[0095] As an optional embodiment, the adjusted network parameters corresponding to the multiple access points are determined according to the adjusted network parameters corresponding to the target networking; and the adjusted network parameters corresponding to the multiple access points are sent to the corresponding access points.

[0096] Optionally, when maintaining a global reinforcement learning process on the AC, the adjusted network parameters corresponding to each of the multiple access points can be sent to the corresponding access points, and the multiple access points can determine the adjusted network performance index parameters corresponding to each of the access points based on the adjusted network parameters. The AC collects information again to obtain the adjusted network performance index parameters of the target network. Alternatively, the AC collects the network performance index parameter information of the APs in the network, determines the network performance index parameters corresponding to the target network, and then sends them to each access point, so as to implement the reinforcement learning process within each access point.

[0097] The following is an optional embodiment. The present invention can use the Q learning algorithm to adjust the network parameters, wherein three factors need to be set in the Q learning algorithm: state, action, and reward. The current network parameters corresponding to each of the multiple access points in the target network can be set to the state of each access point. The action space is composed of increasing the network parameter value of the access point, decreasing the network parameter value of the access point, and keeping the network parameter value of the access point unchanged. The reward function is determined according to the performance change of the target network and the performance change trend of the access point. At the same time, the central learning strategy is adopted in the execution process of the algorithm, and the learning and execution of the strategy are completed with the help of AC. The implementation method can be to maintain a global reinforcement learning process on the AC, and then send the action to be executed, or the AC can collect the network performance indicator parameter information of the access points in the network and share it.

[0098] Figure 4 is a flow chart of a Q learning algorithm provided according to an optional embodiment of the present invention, such as Figure 4 As shown, the specific Q learning process is as follows:

[0099] Initialize Q-table: First create a Q-table, which is the core data structure of Q learning, used to store the expected reward value of each action taken in each state. Usually, all Q values ​​are initialized to 0 or a small positive number at the beginning.

[0100] Selecting an action: At each time step, the agent (i.e., the access point in the present invention) selects an action based on the current state. This is usually achieved through the ε-greedy strategy, which is a method of compromise between exploration and exploitation in reinforcement learning. In this strategy, the agent acts according to the best strategy currently known most of the time to exploit the learned knowledge; but in a small part of the time, the agent will randomly select an action to explore other possibilities in the environment, thereby updating its understanding and strategy of the environment.

[0101] Execute action and receive feedback: The agent executes the selected action and receives feedback from the environment, including the new state and reward value. In the present invention, the reward value is calculated based on the change in the network performance index parameter of the access point after the access point executes the action (such as increasing the network parameter or decreasing the network parameter) and the change in the network performance index parameter of the target network.

[0102] Update Q-value: Based on the immediate reward and the Q-value of the next state, the agent updates the Q-value of the current action.

[0103] Repeated process: The agent continues to select actions, execute actions, receive feedback and update Q values ​​until the stopping condition is reached (such as reaching the maximum number of iterations, or the performance indicator meets a certain threshold), and the target network parameters of each access point in the target network can be determined.

[0104] After multiple iterations, the Q-table will converge to a stable state. At this time, each access point in the target network can extract the strategy for selecting the best action in each state based on the value in the Q-table. That is, when encountering similar network conditions in the future, the previous strategy can be directly adopted to select actions, which can reduce learning time and improve the efficiency of network parameter optimization.

[0105] The above method can take into account the overall network performance of the network, automatically adjust the network parameters of the access point according to the network environment, and has learning capabilities, and can be adaptively adjusted in different network topologies or interference environments. This method uses the performance change feedback of the action execution as input for re-learning, so it can be used to adjust network parameters and improve the overall network performance of the network.

[0106] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0107] Through the description of the above implementation methods, those skilled in the art can clearly understand that the parameter adjustment method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0108] According to an embodiment of the present invention, a parameter adjustment device for implementing the above parameter adjustment method is also provided. Figure 5 is a structural block diagram of a parameter adjustment device provided according to an embodiment of the present invention. Figure 5 As shown, the parameter adjustment device includes: an acquisition module 502, a selection module 504, a first adjustment module 506, a second adjustment module 508 and an iteration module 510. The parameter adjustment device is described below.

[0109] The acquisition module 502 is used to acquire current network parameters and current network performance indicator parameters corresponding to the target network, wherein the target network includes multiple access points, and the current network parameters of the target network represent the current network parameter sets corresponding to the multiple access points.

[0110] The selection module 504 is connected to the acquisition module 502 and is used to select actions for multiple access points from the action set to obtain an action selection set corresponding to the target network.

[0111] The first adjustment module 506 is connected to the selection module 504, and is used to adjust the current network parameters corresponding to the target network based on the action selection set, obtain the adjusted network parameters corresponding to the target network, and obtain the network performance indicator parameters of the target network under the adjusted network parameters.

[0112] The second adjustment module 508 is connected to the first adjustment module 506, and is used to adjust the action selection set according to the adjusted network performance indicator parameters of the target network and the current network performance indicator parameters of the target network to obtain the adjusted action selection set.

[0113] The iteration module 510 is connected to the second adjustment module 508 and is used to adjust the network parameters corresponding to the target networking based on the adjusted action selection set, iterate multiple times until the iteration end condition is met, and determine the target network parameters corresponding to the target networking.

[0114] It should be noted here that the above-mentioned acquisition module 502, selection module 504, first adjustment module 506, second adjustment module 508 and iteration module 510 correspond to steps S102 to S110 in the embodiment, and the instances and application scenarios implemented by the multiple modules are the same as the corresponding steps, but are not limited to the contents disclosed in the above-mentioned embodiments.

[0115] According to an embodiment of the present invention, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned parameter adjustment methods.

[0116] According to an embodiment of the present invention, there is provided an electronic device, characterized in that it includes:

[0117] A memory storing an executable program;

[0118] A processor is used to run the program, wherein any one of the above parameter adjustment methods is executed when the program is running.

[0119] According to an embodiment of the present invention, a computer program product is provided, including a computer program, characterized in that when the computer program is executed by a processor, the steps of any one of the above parameter adjustment methods are implemented.

[0120] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0121] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0123] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0124] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0125] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0126] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A parameter adjustment method, characterized in that: include: Acquire current network parameters and current network performance indicator parameters corresponding to the target network, wherein the target network includes multiple access points, and the current network parameters of the target network represent current network parameter sets corresponding to each of the multiple access points; Selecting actions for the multiple access points respectively from the action set to obtain an action selection set corresponding to the target network; Based on the action selection set, adjust the current network parameters corresponding to the target network, obtain the adjusted network parameters corresponding to the target network, and obtain the network performance indicator parameters of the target network under the adjusted network parameters; Adjusting the action selection set according to the adjusted network performance indicator parameter of the target network and the current network performance indicator parameter of the target network to obtain an adjusted action selection set; Based on the adjusted action selection set, the network parameters corresponding to the target networking are adjusted, and the iteration is repeated multiple times until an iteration end condition is reached to determine the target network parameters corresponding to the target networking.

2. The method according to claim 1, characterized in that The step of adjusting the action selection set according to the adjusted network performance indicator parameter of the target networking and the current network performance indicator parameter of the target networking to obtain the adjusted action selection set includes: Determining a reward function based on the target networking network performance indicator parameters; Obtaining a reward value corresponding to the action selection set according to the current network performance indicator parameter of the target network, the adjusted network performance indicator parameter of the target network, and the reward function; The action selection set is adjusted according to the reward value corresponding to the action selection set to obtain the adjusted action selection set.

3. The method according to claim 2, characterized in that The determining of a reward function based on the target networking network performance indicator parameter includes: Setting the impact weights corresponding to the network performance index parameters of the target network and the network performance index parameters of the access point; Generate a first function item according to a change trend of a network performance indicator parameter corresponding to the target network; generating a second function item according to a change trend of a network performance indicator parameter of the access point; The reward function is determined based on the influence weight, the first function term, and the second function term.

4. The method according to claim 2, characterized in that: The step of adjusting the action selection set according to the reward value corresponding to the action selection set to obtain the adjusted action selection set includes: Determining whether the reward value corresponding to the action selection set is greater than a first preset threshold; When the reward value corresponding to the action selection set is greater than the first preset threshold, keeping the action selection set unchanged; And / or, when the reward value corresponding to the action selection set is not greater than the first preset threshold, adjusting the actions in the action selection set to obtain the adjusted action selection set.

5. The method according to claim 1, characterized in that When the network performance indicator parameter is throughput, the method for obtaining the network performance indicator parameter of the target network includes: Obtain the sending waiting time and data transmission time of the access point during data sending; Determining the throughput of the access point based on the sending waiting time and the data transmission time; The sum of the throughputs corresponding to the multiple access points in the target network is calculated to determine the throughput of the target network.

6. The method according to claim 5, characterized in that The determining the throughput of the access point based on the sending waiting time and the data transmission time includes: The throughput of each access point in the target network is calculated by the following target formula; 7. Among them, is the throughput of the access point in the target network, is the data transmission rate corresponding to the access point in the target network, is a packet error rate corresponding to the access point in the target network, the data transmission rate corresponding to the access point in the target network and the packet error rate corresponding to the access point in the target network are related to the link quality corresponding to the access point transmitting data in the target network, To send the waiting time, The data transmission duration.

8. The method according to claim 1, characterized in that In the case where the network performance indicator parameter is throughput, adjusting the action selection set according to the adjusted network performance indicator parameter of the target networking and the current network performance indicator parameter of the target networking to obtain the adjusted action selection set includes: Calculating a difference between the adjusted throughput of the target network and the current throughput of the target network as a first difference; When the first difference is greater than a second preset threshold, keeping the action selection set unchanged; And / or, when the first difference is not greater than the second preset threshold, the action selection set is adjusted to obtain an adjusted action selection set.

9. The method according to claim 1, characterized in that: In the case where the network performance indicator parameter is throughput, adjusting the action selection set according to the adjusted network performance indicator parameter of the target networking and the current network performance indicator parameter of the target networking to obtain the adjusted action selection set includes: Calculating a difference between the adjusted throughput of the target network and the current throughput of the target network as a first difference; determining an average difference value according to the first difference value and the number of access points of the plurality of access points; Calculate the difference between the adjusted throughput and the current throughput corresponding to each of the multiple access points to obtain a second difference corresponding to each of the multiple access points; When there is a first access point among the multiple access points and the sum of the second difference value and the average difference value is greater than a third preset threshold, keep the action corresponding to the first access point in the action selection set unchanged; and / or when there is a second access point among the multiple access points and the sum of the second difference value and the average difference value is not greater than the third preset threshold value, adjust the action corresponding to the second access point in the action selection set to obtain the adjusted action selection set.

10. The method according to any one of claims 1 to 8, characterized in that: Also includes: Determining the adjusted network parameters corresponding to each of the plurality of access points according to the adjusted network parameters corresponding to the target networking; The adjusted network parameters corresponding to each of the plurality of access points are sent to the corresponding access points.

11. A parameter adjustment device, characterized in that: An acquisition module, configured to acquire current network parameters and current network performance indicator parameters corresponding to a target network, wherein the target network includes a plurality of access points, and the current network parameters of the target network represent a set of current network parameters corresponding to each of the plurality of access points; A selection module, configured to select actions for the plurality of access points from the action set, respectively, to obtain an action selection set corresponding to the target network; A first adjustment module, configured to adjust the current network parameters corresponding to the target network based on the action selection set, obtain the adjusted network parameters corresponding to the target network, and obtain the network performance indicator parameters of the target network under the adjusted network parameters; A second adjustment module, configured to adjust the action selection set according to the adjusted network performance indicator parameters of the target network and the current network performance indicator parameters of the target network to obtain an adjusted action selection set; The iteration module is used to adjust the network parameters corresponding to the target networking based on the adjusted action selection set, iterate multiple times until the iteration end condition is reached, and determine the target network parameters corresponding to the target networking.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the parameter adjustment method according to any one of claims 1 to 9.

13. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the parameter adjustment method described in any one of claims 1 to 9 is executed when the program is run.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the parameter adjustment method according to any one of claims 1 to 9 are implemented.

Images