A polling strategy determination method, device, equipment, storage medium and product

Abstract

The application discloses a polling strategy determination method and device, equipment, a storage medium and a product. The polling strategy determination method comprises the following steps: obtaining a plurality of network state parameters; based on the plurality of network state parameters, performing parameter optimization on strategy parameters of a plurality of polling strategies by using a reinforcement learning algorithm to obtain optimized strategy parameter values; and based on the plurality of network state parameters and the optimized strategy parameter values, determining an optimal polling strategy and a corresponding polling interval time.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a polling strategy determination method, apparatus, device, storage medium, and product. Background Technology

[0002] Currently, to optimize the Quality of Service (QoS) between servers and clients, communication between them often requires a polling mechanism to adjust request frequency and address issues such as network latency and congestion. The polling strategy directly impacts network response speed, resource utilization, and network load balancing.

[0003] Common polling strategies in related technologies mainly include the following three types: fixed-interval polling, simple exponential backoff, and dynamic adjustment methods based on Round Trip Time (RTT). Fixed-interval polling initiates requests at constant time intervals, lacking awareness of network status; simple exponential backoff gradually increases the interval based on the number of failures, but does not fully consider comprehensive network quality indicators; while the RTT-based adjustment method adjusts the polling frequency based on the RTT of the most recent requests. However, all of these methods have certain limitations, exhibiting poor adaptability to complex scenarios and generally poor overall network quality optimization. Summary of the Invention

[0004] This application provides a method, apparatus, device, storage medium, and product for determining a polling strategy.

[0005] The technical solution of this application is implemented as follows:

[0006] A method for determining a polling strategy, comprising:

[0007] Obtain various network state parameters;

[0008] Based on the aforementioned network state parameters, reinforcement learning algorithms are used to fine-tune the policy parameters of various polling strategies to obtain optimized policy parameter values.

[0009] Based on the various network state parameters and the optimized strategy parameter values, the optimal polling strategy and its corresponding polling interval are determined.

[0010] A polling strategy determination device, comprising:

[0011] The network monitoring unit is configured to obtain various network status parameters;

[0012] The parameter adjustment unit is configured to fine-tune the policy parameters of various polling strategies based on the multiple network state parameters and using a reinforcement learning algorithm to obtain optimized policy parameter values.

[0013] The dynamic strategy unit is configured to determine the optimal polling strategy and its corresponding polling interval based on the various network state parameters and the optimized strategy parameter values.

[0014] An electronic device includes: a processor and a memory for storing a computer program capable of running on the processor, wherein, when the processor runs the computer program, it performs the steps of any of the methods described above.

[0015] A storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above methods.

[0016] A computer program product includes a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0017] This application provides a polling strategy determination method that dynamically optimizes the strategy parameters of various polling strategies by acquiring network state parameters in real time and using a reinforcement learning algorithm. Based on the multiple network state parameters and the optimized strategy parameter values, the optimal polling strategy and its corresponding polling interval are determined. This method maintains high adaptability in complex network scenarios and effectively improves network quality optimization. Attached Figure Description

[0018] Figure 1 A flowchart illustrating a polling strategy determination method provided in an embodiment of this application;

[0019] Figure 2 A schematic diagram of the system architecture for the polling strategy determination method provided in the embodiments of this application;

[0020] Figure 3 A schematic diagram of the interaction scenario for the polling strategy determination method provided in the embodiments of this application;

[0021] Figure 4 A schematic diagram of a polling strategy determination device provided in an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0024] It should be understood that the phrases "embodiments of this application" or "foreign embodiments" throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "embodiments of this application" or "in the foreign embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0025] It should be noted that the terms "first, second, and third" used in the embodiments of this application are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0026] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0027] The technical terms used in the embodiments of this application will be explained below.

[0028] Cloud servers (such as virtual machines): These run on physical hardware using virtualization technology. Hardware resources can be divided into multiple virtual machines, each with its own operating system and applications, and resources are allocated on demand. Virtualization technology can improve the utilization of hardware resources, but due to the overhead of virtualization, the performance of virtual machines is somewhat limited.

[0029] Bare Metal Server (BMS): This refers to a computing service that combines the elasticity of cloud servers with the performance of physical machines. Bare Metal Servers integrate the advantages of traditional physical servers and modern cloud computing, providing a high-performance, secure, and flexible computing resource. They support running code directly on the hardware, avoiding the performance degradation associated with virtualization and ensuring resource exclusivity and isolation.

[0030] Elastic bare metal servers: Compared to traditional bare metal servers, elastic bare metal servers offer the flexibility of cloud computing. Users not only have complete control over hardware resources but can also dynamically adjust resource scale according to demand. Elastic bare metal servers can respond quickly to load changes, offering the flexibility of virtual machines while maintaining the high performance of physical servers.

[0031] Quality of Service (QoS) is a network technology that aims to provide better service capabilities for specified network communications through various underlying technologies. It is a network security mechanism and a technology used to solve problems such as network latency and congestion.

[0032] Reinforcement learning (RL) algorithms are a type of machine learning method that uses an agent to interact with the environment in order to learn the optimal policy with the goal of maximizing cumulative rewards.

[0033] Q-learning is a value-iterative reinforcement learning algorithm primarily used to learn a policy in a given environment, enabling the agent to maximize cumulative rewards through interaction with the environment. It guides the agent's behavioral choices by learning a state-action value function (Q-function), and is applicable to various discrete states and actions. The state-action value table (Q-Table) is the core data structure of the Q-learning algorithm, used to record the value (Q-Value) obtained by taking different actions in each state, thereby helping the agent select the optimal policy.

[0034] Currently, to optimize network service quality between the server and client, communication between them often requires polling strategies to adjust request frequency, task processing progress, and other aspects to address network latency and congestion. Common polling strategies include three main types: Fixed Interval Polling, Basic Exponential Backoff, and RTT-based dynamic adjustment methods. In the Fixed Interval Polling strategy, the console polls the underlying status at fixed time intervals (e.g., every 5 seconds). This approach cannot adapt to network fluctuations or task changes, leading to frequent timeouts when network quality is poor, resulting in high latency. Conversely, when network quality is good, it wastes resources, leading to high resource consumption. In the simple exponential backoff approach, the polling interval is gradually increased after each failed request (e.g., doubling the interval after each failure). This method relies solely on the number of failed requests as an indicator, which can lead to a lack of comprehensive awareness of network service quality and may result in over-backoff or insufficient recovery, such as prolonged backoff or ineffective retries even after network recovery. In the RTT-based dynamic adjustment approach, the polling interval is adjusted based on the RTT of the most recent requests. This method also relies solely on RTT as an indicator, easily ignoring task characteristics and other network metrics, making it less adaptable to complex network scenarios. Therefore, the polling strategies in these technologies tend to lack a comprehensive understanding of network conditions, resulting in poor adaptability to complex network scenarios and ultimately poor network service quality optimization.

[0035] Furthermore, since polling strategies in related technologies often use fixed parameters, manual parameter tuning is frequently required to obtain a more suitable network service quality for different scenarios, resulting in high operation and maintenance costs. Therefore, current polling strategies suffer from inflexibility, poor adaptability to complex scenarios, and poor network quality optimization effects.

[0036] The polling strategy determination method provided in this application dynamically optimizes the parameters of various polling strategies by acquiring multiple network state parameters in real time and using reinforcement learning algorithms. Based on these network state parameters and the optimized strategy parameter values, the optimal polling strategy and its corresponding polling interval are determined. This overcomes the limitations of fixed parameters and enables automatic, timely, and intelligent adjustment of the polling strategy under different network scenarios (such as stable network quality, unstable network quality, high network load, and stable network processing speed). The method polls requests using the corresponding polling interval, maintaining high adaptability in complex network scenarios, improving network quality optimization, increasing polling efficiency, reducing invalid requests, and minimizing resource waste. It can be widely applied to scenarios requiring network service quality (QoS) optimization, such as IoT monitoring, cloud service status synchronization, and bare metal servers. Compared to traditional polling mechanisms based on a single metric, this application achieves better network service quality optimization and demonstrates better network adaptability in complex network scenarios. This solves the drawbacks of some technologies that can only use a single polling strategy, use fixed polling parameters, and require manual intervention to change the polling strategy parameters.

[0037] In some exemplary embodiments, the polling strategy determination method provided in this application can be executed by a network service quality optimization system. This system can be applied to or operated as an electronic device. The electronic device can be a client device, a server device, or an intermediate device located between the client and the server. The electronic device here can include terminal devices such as smartphones, tablets, laptops, desktop computers, smart voice interaction devices, smart home appliances, smartwatches, vehicle terminals, and aircraft; or, the electronic device can also include a server (such as a standalone physical server), a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0038] For example, a network service quality optimization system can run as a bare metal server (e.g., a non-virtualized host computing node) in a cloud computing environment (also known as a cloud service platform or cloud platform), capable of communicating with a console. The console can refer to the service management system provided by the cloud service platform. For instance, the bare metal server can be a high-performance network-based bare metal server or an elastic bare metal server.

[0039] For example, the network service quality optimization system can be deployed on either the server or the client. For instance, it can be an application module running on the server, a local scheduler on the client, or an intelligent decision engine deployed on an edge node or cloud platform.

[0040] Figure 1 This is a flowchart illustrating a polling strategy determination method provided in an embodiment of this application. The following will be combined with... Figure 1 Please provide an explanation. For example... Figure 1 As shown, the polling strategy determination method may include the following steps S101 to S103:

[0041] Step S101: Obtain various network state parameters;

[0042] Step S102: Based on multiple network state parameters, use reinforcement learning algorithms to fine-tune the policy parameters of multiple polling strategies to obtain optimized policy parameter values;

[0043] Step S103: Based on multiple network state parameters and optimized policy parameter values, determine the optimal polling policy and its corresponding polling interval.

[0044] Here, multiple network state parameters refer to a set of quantitative indicators used to describe the current quality of the communication network. These parameters reflect information such as network environment stability, transmission capacity, request response, and task execution progress. Thus, by using multiple network state parameters, a multi-dimensional network state can be constructed, enabling a more comprehensive understanding of the network environment.

[0045] In some exemplary embodiments, the various network state parameters may include multiple parameters such as round-trip time, packet loss rate, bandwidth, and task progress.

[0046] Round-trip time (RTT) refers to the total time delay it takes for a data packet to travel from the sender to the receiver and receive an acknowledgment. This metric measures the response speed of network communication and is a crucial parameter for evaluating network quality. RTT reflects whether the network is stable or congested, providing a basis for adjusting the polling interval. Generally, a shorter RTT indicates faster network transmission, and vice versa. Packet loss rate refers to the proportion of failed requests to total requests within a given time period due to network failures or insufficient resources. Bandwidth (also known as throughput) refers to the amount of data transmitted per unit time (e.g., 1 second). It can also be understood as the average rate of a specific data stream between two nodes in a network, typically used to describe the data transmission speed of a network connection. Common units of bandwidth include bps (bits per second) or Mbps (megabits per second). Task progress indicates the degree of completion of the current processing task, usually measured as the percentage of completed work relative to the total task, typically expressed as a percentage.

[0047] For example, packet loss rate can be the request failure rate within a sliding window with a preset window length. For example, bandwidth can be the average effective bandwidth, which can be the total number of bytes transmitted divided by the total transmission time. The total number of bytes transmitted can be the sum of the number of bytes transmitted in the most recent request and the historical cumulative total number of bytes transmitted within a preset time period (such as the polling interval). The total transmission time can be the sum of the RTT of the most recent request and the RTT of all requests within a preset time period (such as the polling interval).

[0048] In practical applications, network communication between the client and server can be monitored in real time. This includes monitoring client requests and server responses, and collecting network status parameters based on request and response messages. For example, RTT (Round-Trip Time) can be calculated from the response time of each request. Packet loss rate can be calculated by using a sliding window to determine the ratio of failed requests to total requests, or vice versa. Bandwidth can be estimated by the relationship between cumulative data transmission volume and time. Task progress can be monitored by tracking the amount of data generated during task execution (e.g., file upload).

[0049] In some exemplary embodiments, the strategy parameters of multiple polling strategies may include: a backoff coefficient for a preset first polling strategy to control the aggressiveness of backoff, a prediction coefficient for a preset second polling strategy to correct the calculation of the remaining task volume, and a smoothing coefficient for a preset third polling strategy to adjust the smoothing sensitivity of the round-trip delay time. Thus, in the selection of strategy parameters for optimization, reinforcement learning algorithms are used to dynamically optimize the strategy parameters of multiple polling strategies, enabling them to cope with more complex network environments while significantly reducing operational costs.

[0050] In some exemplary embodiments, the optimal polling strategy refers to the polling strategy that is most suitable for the current network state among multiple polling strategies. Thus, by selecting the optimal polling strategy from among multiple options, a wider range of scenarios are covered, avoiding the poor adaptability of a single strategy.

[0051] For example, based on the current network state, the optimal polling strategy can be any one of the preset first polling strategy, preset second polling strategy, and preset third polling strategy. For example, the preset first polling strategy can be enhanced exponential backoff. For example, the preset second polling strategy can be predictive polling, which adjusts the polling frequency based on the remaining task load. For example, the preset third polling strategy can be exponential moving average (EMA), a time-series weighted averaging method that assigns higher weight to recent observations, also known as smoothing adjustment.

[0052] Thus, in the polling strategy determination method provided in this application embodiment, by acquiring various network state parameters in real time (such as round-trip latency, packet loss rate, bandwidth, task progress, etc.), and using reinforcement learning algorithms to dynamically optimize the strategy parameters (such as backoff coefficient, smoothing coefficient, prediction coefficient, etc.) of various polling strategies, it is possible to intelligently adjust the strategy parameters according to the current network state, obtaining optimized strategy parameter values ​​with higher adaptability to the current network environment. Based on the various network state parameters and the optimized strategy parameter values, the optimal polling strategy and its corresponding polling interval are determined. This enables dynamic adaptive selection of the optimal polling strategy that is more suitable for the current network environment from among various polling strategies, so that requests are polled using the polling interval of the optimal polling strategy corresponding to the current network environment. Therefore, it overcomes the limitations of fixed parameters and realizes automatic, timely, and intelligent adjustment of the polling strategy under different network scenarios (such as stable network quality, unstable network quality, network under high load, stable network processing speed, etc.). It can maintain high adaptability in complex network scenarios, effectively improve network quality optimization, increase polling efficiency, reduce the number of invalid requests, and reduce resource waste. This can reduce cloud computing call costs and bandwidth overhead. Furthermore, with fewer invalid requests, the pressure on the server to process requests is reduced, allowing for a corresponding reduction in the number of server clusters.

[0053] In some exemplary embodiments, step S102 may include the following step S20:

[0054] Step S20: Based on round-trip delay, packet loss rate and task progress among various network state parameters, use reinforcement learning algorithm to fine-tune the policy parameters of various polling strategies to obtain optimized policy parameter values.

[0055] In this way, multiple network state parameters such as round-trip latency, packet loss rate, and task progress together constitute a multi-dimensional network quality profile. Based on the multi-dimensional network state parameters, reinforcement learning algorithms are used to fine-tune the policy parameters of various polling strategies to obtain optimized policy parameter values. This makes it easier to dynamically select the most suitable polling strategy according to different network scenarios, thereby achieving more efficient resource utilization and better service quality.

[0056] In some exemplary embodiments, step S20 may include the following steps S201 to S202:

[0057] Step S201: Discretize the current values ​​of round-trip delay, packet loss rate and task progress among various network state parameters to obtain the current state vector;

[0058] Step S202: Based on the current state vector, the strategy parameters of various polling strategies are tuned using a pre-built state-action value table to obtain optimized strategy parameter values.

[0059] Thus, by discretizing, the state parameters of continuous networks can be transformed into a state-space representation suitable for reinforcement learning algorithms, improving the algorithm's operability and convergence speed. Furthermore, by using a pre-constructed state-action value table to fine-tune the policy parameters of various polling strategies, the adjustment process becomes more directional and goal-oriented, improving both tuning efficiency and the accuracy of policy decisions.

[0060] In some exemplary embodiments, before discretizing the round-trip delay, packet loss rate, and task progress among various network state parameters in step S201 to obtain the current state vector, the polling strategy determination method may further include:

[0061] Step S203: Predefine the elements of the reinforcement learning algorithm and obtain the state-action value table. The elements include: state space, action space, and reward function. The state space is obtained by discretizing the round-trip delay time, packet loss rate, and task progress. Each state in the state space corresponds to a combination of network state values. The action space is obtained by discretizing the policy parameters of various polling strategies. Each action in the action space corresponds to a combination of policy parameter values. The reward function is used to update the state-action value table.

[0062] In this way, the construction of the state space ensures the complete expression of the network state, the design of the action space makes the adjustment of policy parameters feasible, and the setting of the reward function guides the system to optimize towards the goal of improving service quality. This yields the elements of a reasonable reinforcement learning algorithm, thereby ensuring the structured and efficient nature of the reinforcement learning process and improving the intelligence level of policy parameter tuning.

[0063] In some exemplary embodiments, step S103, based on multiple network state parameters and optimized policy parameter values, determines the optimal polling policy and its corresponding polling interval. This may include: determining whether the network state is a target state based on multiple network state parameters; when the network state is determined to be a target state, determining the polling policy corresponding to the target state among the multiple polling policies as the optimal polling policy. For example, the target state may include: unstable state, high load state, stable state, small RTT fluctuation, stable task processing speed, etc.

[0064] In some exemplary embodiments, step S103, based on multiple network state parameters and optimized policy parameter values, determines the optimal polling policy and its corresponding polling interval, and may include the following steps S301 to S303:

[0065] Step S301: Determine whether multiple network state parameters meet the preset first condition;

[0066] Step S302: When it is determined that multiple network state parameters meet the preset first condition, the preset first polling strategy corresponding to the preset first condition is determined as the optimal polling strategy.

[0067] Step S303: Based on the current value of packet loss rate among various network state parameters and the backoff coefficient value among optimized strategy parameter values, determine the first polling interval time of the preset first polling strategy, which is used as the polling interval time corresponding to the optimal polling strategy.

[0068] Thus, when the network is typically unstable or under high load, adjusting the backoff coefficient can prevent frequent polling from increasing the network burden, significantly reducing the request frequency and improving network stability. Furthermore, when the network recovers, successful requests immediately reset the polling interval, avoiding the recovery delay of traditional exponential backoff.

[0069] In some exemplary embodiments, the preset first condition may refer to the current value of the packet loss rate in the network state parameters being greater than a preset threshold. For example, the preset threshold may be 0.3. When the current value of the packet loss rate in the network state parameters is greater than the preset threshold, the network is typically in an unstable or high-load state.

[0070] In some exemplary embodiments, step S301, determining that multiple network state parameters satisfy a preset first condition, may include: determining whether the current value of the packet loss rate among the multiple network state parameters is greater than a preset threshold. Thus, when the current value of the packet loss rate among the network state parameters is greater than the preset threshold, it can be determined that multiple network state parameters satisfy the preset first condition; at this time, the network state is an unstable state or a high-load state.

[0071] In some exemplary embodiments, step S303 determines the first polling interval of the preset first polling strategy based on the current value of the packet loss rate among various network state parameters and the backoff coefficient value among the optimized strategy parameter values. This may include: determining the first polling interval of the preset first polling strategy based on the current value of the packet loss rate among various network state parameters, the preset base interval time, and the backoff coefficient value among the optimized strategy parameter values, according to the following formula (1).

[0072] interval=base_interval*(1+loss_rate)^backoff_base(1);

[0073] Where interval represents the polling interval, base_interval represents the preset base interval, loss_rate represents the packet loss rate in the network state parameters, and backoff_base represents the backoff coefficient.

[0074] Therefore, the higher the packet loss rate, the more aggressive the backoff strategy becomes.

[0075] In some other exemplary embodiments, step S103, based on multiple network state parameters and optimized policy parameter values, determines the optimal polling policy and its corresponding polling interval, and may include the following steps S401 to S403:

[0076] Step S401: Determine whether multiple network state parameters meet the preset second condition;

[0077] Step S402: When it is determined that multiple network state parameters meet the preset second condition, the preset second polling strategy corresponding to the preset second condition is determined as the optimal polling strategy.

[0078] Step S403: Based on the current values ​​of task progress and bandwidth among various network state parameters, as well as the prediction coefficient values ​​among the optimized strategy parameter values, determine the second polling interval time of the preset second polling strategy, which is used as the polling interval time corresponding to the optimal polling strategy.

[0079] Thus, in the polling strategy determination method provided in this application embodiment, when the network is in a stable state and the historical processing speed tends to stabilize after half of the tasks have been completed, the second polling interval of the preset second polling strategy is determined by using the current value of the task progress and the current value of the bandwidth among various network state parameters, as well as the prediction coefficient value among the optimized strategy parameter values. The polling interval can be gradually increased as the task nears completion. Compared to fixed-frequency polling, the preset second polling strategy can reduce a large number of invalid requests. Furthermore, if the processing speed (i.e., the current value of the bandwidth) suddenly increases, the polling interval can be automatically shortened to maintain real-time synchronization.

[0080] In some exemplary embodiments, the preset second condition may refer to the current value of the packet loss rate in the network status parameters not being greater than a preset threshold, and the current value of the task progress in the network status parameters being greater than half of the total task volume. For example, the preset threshold may be 0.3. Thus, when the current value of the packet loss rate in the network status parameters is not greater than the preset threshold, the network is usually in a stable state, and when the current value of the task progress is greater than half of the total task volume, the processing speed tends to stabilize.

[0081] In some exemplary embodiments, step S401, determining whether multiple network state parameters satisfy a preset second condition, may include: determining whether the current value of the packet loss rate in the network state parameters is not greater than a preset threshold, and whether the current value of the task progress in the network state parameters is greater than half of the total task volume. Thus, when it is determined that the current value of the packet loss rate in the network state parameters is not greater than the preset threshold, and the current value of the task progress in the network state parameters is greater than half of the total task volume, it can be determined that the network state parameters satisfy the preset second condition.

[0082] In some exemplary embodiments, step S403 determines the second polling interval of the preset second polling strategy based on the current value of the task progress and the current value of the bandwidth among various network state parameters and the prediction coefficient value among the optimized strategy parameter values. This may include the following steps: Based on the current value of the task progress and the current value of the bandwidth among various network state parameters, the total task volume and the prediction coefficient value among the optimized strategy parameter values, the second polling interval of the preset second polling strategy is determined according to the following formulas (2) to (3).

[0083] remaining=total-progress(2);

[0084] interval=remaining / speed*β(3);

[0085] Wherein, remaininging represents the remaining task quantity, total represents the total task quantity, progress represents the current value of task progress among various network state parameters, interval represents the polling interval, speed represents the current value of bandwidth (such as effective bandwidth) among various network state parameters, and β represents the prediction coefficient among the optimized policy parameter values.

[0086] In addition, since the unit of the current value of bandwidth (speed) is usually bps or Mbps, and the unit of the polling interval (interval) is usually milliseconds (ms), in order to unify the units, formula (3) can be transformed into: interval = remaining / speed * 1000 * β.

[0087] For example, the current value of the bandwidth (i.e., the effective bandwidth) can be determined according to the following formula (4):

[0088] speed=(progress-last_progress) / (current_time-last_time) (4);

[0089] Among them, speed represents the current value of bandwidth (such as effective bandwidth) among various network status parameters, progress represents the current value of task progress, last_progress represents the historical value of the previous task progress, current_time represents the time of task progress, and last_time represents the time of the previous task progress.

[0090] In some other exemplary embodiments, step S103, based on multiple network state parameters and optimized policy parameter values, determines the optimal polling policy and its corresponding polling interval, and may include the following steps S501 to S503:

[0091] Step S501: Determine whether multiple network state parameters meet the preset third condition;

[0092] Step S502: When it is determined that multiple network state parameters meet the preset third condition, the preset third polling strategy corresponding to the preset third condition is determined as the optimal polling strategy.

[0093] Step S503: Based on the current value of the round-trip delay time among various network state parameters and the smoothing coefficient value among the optimized strategy parameter values, determine the third polling interval time of the preset third polling strategy as the polling interval time corresponding to the optimal polling strategy.

[0094] Thus, when the network is in a stable state but the task processing speed has not yet stabilized, the third polling interval of the preset third polling strategy can be determined by using the current value of the round-trip time (RTT) among various network state parameters and the smoothing coefficient value among the optimized strategy parameters. This allows for smooth adjustment, reduces the impact of instantaneous RTT jitter, and avoids frequent interval adjustments.

[0095] In some exemplary embodiments, the preset third condition may refer to the current value of the packet loss rate in the network state parameters not exceeding a preset threshold (e.g., 0.3), and the current value of the task progress in the network state parameters not exceeding half of the total task volume. In this way, the network is in a stable state, RTT jitter is small, and the task processing speed has not yet reached a stable state.

[0096] In some exemplary embodiments, step S501, determining whether multiple network state parameters satisfy a preset third condition, may include: determining whether the current value of the packet loss rate in the network state parameters is not greater than a preset threshold, and whether the current value of the task progress in the network state parameters is not greater than half of the total task volume. Thus, when it is determined that the current value of the packet loss rate in the network state parameters is not greater than the preset threshold, and the current value of the task progress in the network state parameters is not greater than half of the total task volume, it can be determined that the network state parameters satisfy the preset third condition.

[0097] In some exemplary embodiments, step S503, determining the third polling interval of the preset third polling strategy based on the current value of the round-trip delay time among various network state parameters and the smoothing coefficient value among the optimized strategy parameter values, may include the following steps: determining the third polling interval of the preset third polling strategy based on the current value of the round-trip delay time among various network state parameters and the smoothing coefficient value among the optimized strategy parameter values, according to the following formulas (5) to (6).

[0098] ema_rtt=α*ema_rtt+(1-α)*last_rtt(5);

[0099] interval=ema_rtt*1.2(6);

[0100] Where interval represents the polling interval, α represents the smoothing coefficient in the optimized strategy parameter values, ema_rtt represents the current value of the round-trip delay, and last_rtt represents the value of the previous round-trip delay.

[0101] Thus, in the polling strategy determination method provided in this application embodiment, when the network conditions are relatively stable (such as stable network quality, low packet loss rate, small RTT fluctuation and task progress not yet completed), the preset third polling strategy corresponding to the preset third condition is determined as the optimal polling strategy, and a smoothing coefficient is used for adjustment, which can reduce the impact of instantaneous RTT jitter and avoid frequent adjustments to the polling interval.

[0102] In some exemplary embodiments, after determining the optimal polling strategy and its corresponding polling interval time based on multiple network state parameters and optimized strategy parameter values ​​in step S103, the polling strategy determination method may further include the following step S60:

[0103] Step S60: Schedule the client's pending requests according to the polling interval time corresponding to the optimal polling strategy.

[0104] In some exemplary embodiments, step S60, which schedules the client's pending requests according to the polling interval time corresponding to the optimal polling strategy, may include the following steps S601 to S603:

[0105] Step S601: Obtain the connection status of either the server or the client;

[0106] Step S602: Determine whether a network anomaly has occurred based on whether the connection status is online;

[0107] Step S603: When it is determined that no network anomaly has occurred, the client's pending requests are scheduled according to the polling interval time corresponding to the optimal polling strategy.

[0108] For example, if either the server or the client is online, it indicates that there is no network problem. If either the server or the client is offline, it indicates that there is a network problem.

[0109] In some exemplary embodiments, after determining whether a network anomaly has occurred in step S602 based on whether the connection status is online, the polling strategy determination method may further include the following steps S604 to S605:

[0110] Step S604: When a network anomaly is detected, add the client's pending requests to the cache queue;

[0111] Step S605: At a preset fixed time (e.g., every 5 seconds), reacquire the connection status, and based on whether the reacquired connection status is online, re-determine whether a network anomaly has occurred, until it is determined that no network anomaly has occurred, and then execute the pending requests in the cache queue according to the polling interval time corresponding to the optimal polling strategy.

[0112] Thus, in the polling strategy determination method provided in this application embodiment, network status is monitored at preset fixed intervals to maintain connection activity during the polling interval. When the network status is normal, the polling task can be executed normally. When a network anomaly occurs (such as a network interruption), requests during the anomaly period are added to a cache queue to await network recovery, avoiding task loss due to network issues. After the network recovers, a scheduled task restarts the tasks in the cache queue. This improves the reliability of network service quality.

[0113] In some exemplary embodiments, in a scenario where an implementable polling strategy is determined, Figure 2 A schematic diagram of the system architecture for the polling strategy determination method provided in the embodiments of this application is shown below. Figure 2 As shown, the system architecture may include: a network quality awareness module 201, a reinforcement learning (RL) dynamic parameter tuning module 202, a dynamic policy module 203, and a state compensation module 204. The solution of this application can be divided into the following four parts: The first part is that the network quality awareness module monitors the network state in real time, providing data support for policy selection. The second part is that the RL dynamic parameter tuning module uses reinforcement learning algorithms to dynamically adjust policy parameters, including: backoff coefficient, prediction coefficient, and smoothing coefficient. The third part is that the dynamic policy module selects the optimal polling policy based on the network state parameters and calculates the corresponding polling interval time. The fourth part is that the state compensation module handles network anomaly scenarios.

[0114] The following section will explain how to obtain various network status parameters, including round-trip time, packet loss rate, and bandwidth, as an example. The first part is that the network quality awareness module monitors network status parameters in real time to provide data support for policy selection, so that the dynamic policy module can better select the appropriate polling policy when facing complex environments.

[0115] Step 2-1: Obtain the round-trip time (RTT) in real time by recording the response time of the most recent request.

[0116] Step 2-2: Based on the sliding window, calculate the failure rate of the most recent N requests and calculate the packet loss rate according to the following formula (7). Where N is a positive integer greater than 1 (e.g., 10).

[0117] (7);

[0118] in, This represents the packet loss rate, and N represents the total number of requests within the sliding window. This represents the number of failed requests in the most recent N requests; This indicates the size of the sliding window.

[0119] Steps 2-3: Calculate the average effective bandwidth by accumulating the total amount of transmitted data and its corresponding cumulative delay time, and use the average effective bandwidth as the bandwidth among various network state parameters.

[0120] For example, the implementation of the first part is shown in Table 1:

[0121] Table 1. Algorithms for obtaining network state parameters

[0122]

[0123] The ternary operator has the syntax: condition ? value1 : value2. If the condition is true, it returns value 1; otherwise, it returns value 2. That is, if the network request result `success` (a boolean value) is true, it returns the RTT value of that request as the current RTT value; that is, when `success=true`, `last_rtt=rtt`. If the request result `success` (a boolean value) is false, it returns infinity as the current RTT value and marks an exception; that is, when `success=false`, `last_rtt=INFINITE`.

[0124] The following section uses the Q-Learning algorithm in reinforcement learning as an example to explain the second part. This second part describes the dynamic parameter tuning module of RL, which uses reinforcement learning to dynamically adjust policy parameters, including backoff coefficients, prediction coefficients, and smoothing coefficients. Reinforcement learning algorithms learn the mapping from state to optimal action by constructing a state-action value table. The elements of the reinforcement learning algorithm are predefined to obtain the state-action value table. These elements include a state space, an action space, and a reward function. The state space is discretized based on round-trip latency, packet loss rate, and task progress. Each state in the state space corresponds to a combination of network state values. The action space is discretized based on policy parameters from various polling strategies. Each action in the action space corresponds to a combination of policy parameter values. The reward function is used to update the state-action value table.

[0125] Steps 2-4: Define the state space: This state space consists of the monitoring indicators and task features output by the network quality perception module. The state space is then discretized to adapt to the discrete space characteristics of reinforcement learning algorithms.

[0126] For example, packet loss rate can be divided into three levels: packet loss rate of 0-10%, packet loss rate of 10% to 30%, and packet loss rate of >30%.

[0127] For example, RTT can be divided into three levels: RTT < 100ms, RTT between 100ms and 500ms, and RTT > 500ms.

[0128] For example, the task progress can be divided into two levels, such as task progress < 50% and task progress ≥ 50%.

[0129] Steps 2-5 define the action space and discretize and adjust the three parameters of this module: the backoff coefficient for controlling the aggressiveness of the backoff in the first polling strategy, the prediction coefficient for correcting the calculation of the remaining task amount in the second polling strategy, and the smoothing coefficient for adjusting the smoothing sensitivity of the round-trip delay time in the third polling strategy.

[0130] For example, the backoff coefficient (backoff_base) can take values ​​in the range of {1.5, 2.0, 2.5}.

[0131] For example, the prediction coefficient (β) can take values ​​in the range of {1.0, 1.2, 1.5}.

[0132] For example, the smoothing coefficient (α) can take values ​​in the range of {0.7, 0.8, 0.9}.

[0133] Each action corresponds to a combination of policy parameter values, for example, Action = (backoff_base = 2.0, α = 0.8, β = 1.2).

[0134] Steps 2-6: Design the reward function as follows: Reward = (Request Success Rate × a) - (Average Round-Trip Latency × b) - (Polling Frequency × c), where a, b, and c are preset weight coefficients. Request Success Rate = Number of Successful Requests / Total Number of Requests; Average Round-Trip Latency is the average round-trip latency per unit time; Polling Frequency is the number of requests per unit time.

[0135] Steps 2-7: After defining the elements of the reinforcement learning algorithm (including state space, action space, and reward function), the multiple network state parameters are first discretized to obtain the current state vector, which is then adapted to the reinforcement learning algorithm. Based on the current state vector, the pre-built state-action value table is used for iteration to obtain optimized policy parameter values, thereby achieving parameter tuning for various polling strategies.

[0136] For example, the continuous indicators obtained by the network quality awareness module, i.e., multiple network state parameters, are discretized to obtain the current state S. For instance, discretizing the current values ​​of round-trip latency, packet loss rate, and task progress among various network state parameters can yield a state S represented by a three-dimensional vector. For example, S = (2, 1, 0) indicates a packet loss rate ≥ 30%, a round-trip latency of 200ms to 500ms, and a task progress < 50%.

[0137] For example, the initialization process of the state-action value table can be as follows: set the initial policy parameter values ​​to: backoff_base=2, α=0.8, β=1.2, set the action value (Q value) of the action corresponding to the initial policy parameter value to the baseline value (e.g., 100), and set the Q value of other actions to 0. After each round of polling, according to the feedback result, calculate the reward R through the reward function, and update the state-action value table according to the reinforcement learning algorithm formula to obtain the optimized policy parameter values ​​(i.e., the updated Q values). Hereinafter, the reinforcement learning algorithm formula is shown in the following formula (8):

[0138] Q(S,A)=Q(S,A)+η[R+γ*max(Q(S',A'))-Q(S,A)](8);

[0139] Where Q(S,A) represents the action value (Q value) of action A in the current state S; η is the discount factor (0<η≤1, such as 0.9), which controls the degree of influence of new information on the action value (Q value), i.e., the learning rate; γ is the weight coefficient (0≤γ<1, such as 0.9), which determines the discounting degree of future rewards; R represents the reward obtained after performing action A in state S; S' is the next state; max(Q(S',A')) is the maximum Q value of all possible actions A' in S', i.e., the optimal action value in the next state S'.

[0140] In Q-Learning, the discount factor η and the weight coefficient γ jointly affect the convergence and stability of the algorithm. They need to be adjusted in tandem to balance the weights of short-term and long-term rewards as well as the update step size.

[0141] In practice, the reinforcement learning algorithm updates its internal state-action value table after each polling cycle to reflect the optimal action choice for the current state. In this way, the system can automatically adjust its policy parameters under different network environments, thereby improving the overall service quality.

[0142] For example, the implementation of the second part is shown in Table 2:

[0143] Table 2. Algorithms for parameter optimization based on reinforcement learning algorithms.

[0144]

[0145] The third part, described below, involves the dynamic strategy module selecting the optimal polling strategy from multiple polling strategies based on network state parameters. It then calculates the corresponding polling interval based on the optimized polling strategy parameter values. These multiple polling strategies can be three types: a preset first polling strategy, a preset second polling strategy, and a preset third polling strategy. Depending on the current network state, the optimal polling strategy can be any one of these three strategies. For example, the preset first polling strategy could be an enhanced exponential backoff strategy, dynamically increasing the interval when the packet loss rate is high. The preset second polling strategy could be a predictive polling strategy, adjusting the polling frequency based on the remaining workload. The preset third polling strategy could be an EMA (Exponential Backoff Strategy), optimizing the polling interval based on historical RTT (Round-Trip Time) when the network state is stable.

[0146] Steps 2-8 determine whether the current value of the packet loss rate among various network state parameters is greater than a preset threshold. When the current value of the packet loss rate among the network state parameters is greater than the preset threshold, the network state is unstable or high-load. In this case, the enhanced exponential backoff strategy is selected, and the backoff coefficient is dynamically adjusted with the packet loss rate exponentially, as shown in formula (1). Thus, when the network is usually in an unstable or high-load state, by adjusting the backoff coefficient, frequent polling can be avoided from increasing the network burden, the frequency of requests can be significantly reduced, and network stability can be improved. Furthermore, when the network recovers, successful requests will immediately reset the polling interval time, avoiding the recovery delay of traditional exponential backoff.

[0147] Steps 2-9 determine whether the current value of the packet loss rate in the network status parameters is not greater than a preset threshold, and whether the current value of the task progress in the network status parameters is greater than half of the total task volume. When it is determined that the current value of the packet loss rate in the network status parameters is not greater than the preset threshold, and the current value of the task progress in the network status parameters is greater than half of the total task volume, the network is in a stable state and the processing speed tends to stabilize. At this time, a predictive polling strategy is selected as the optimal polling strategy. The next polling interval is calculated by dividing the remaining task volume by the average processing speed. As the task approaches completion, the polling interval gradually increases. Compared with fixed-frequency polling, the preset second polling strategy can reduce a large number of invalid requests. Furthermore, if the processing speed (i.e., the current value of bandwidth) suddenly increases, the polling interval can be automatically shortened to maintain real-time synchronization.

[0148] Step 2-10: Determine whether the current value of the packet loss rate in the network status parameters is not greater than a preset threshold, and whether the current value of the task progress in the network status parameters is not greater than half of the total task volume. When it is determined that the current value of the packet loss rate in the network status parameters is not greater than the preset threshold, and the current value of the task progress in the network status parameters is not greater than half of the total task volume, the network is in a stable state, the RTT jitter is small, and the task processing speed has not yet reached a stable state. At this time, selecting EMA as the optimal polling strategy can achieve smooth adjustment, reduce the impact of instantaneous RTT jitter, and avoid frequent interval adjustments. In addition, when calculating the polling interval time, a 20% margin can be added (such as the formula ema_rtt*1.2) to reserve a buffer for network fluctuations.

[0149] For example, the implementation of the third part is shown in Table 3:

[0150] Table 3 Algorithm for Strategy Selection

[0151]

[0152] The fourth part, which addresses network anomaly scenarios handled by the state compensation module, is explained below. By monitoring network status at preset fixed intervals, connection activity is maintained during the polling interval. When the network is normal, polling tasks execute normally. However, when a network anomaly occurs (such as a network interruption), requests from the period of anomaly are added to a cache queue to await network recovery, preventing task loss due to network issues. Once the network recovers, tasks in the cache queue are restarted via a scheduled task. This improves the reliability of network service quality.

[0153] Step 2-11: Determine whether the connection status of either the server or the client is online;

[0154] Step 2-12: When either the server or the client is online, it indicates that there is no network anomaly (e.g., the network is normal). The client's pending requests are scheduled according to the polling interval corresponding to the optimal polling strategy.

[0155] Step 2-13: If the connection status of either the server or the client is not online, it indicates a network anomaly (such as a network interruption). The client's pending requests are added to the cache queue. The connection status is retrieved again at preset fixed intervals (e.g., every 5 seconds). Based on whether the re-retrieved connection status is online, the network anomaly is re-determined until it is determined that no network anomaly has occurred. Then, the pending requests in the cache queue are executed according to the polling interval corresponding to the optimal polling strategy.

[0156] For example, the implementation of the fourth part is shown in Table 4:

[0157] Table 4 Algorithm for State Compensation Mechanism

[0158]

[0159] Below, with Figure 2 The system architecture shown is used as an example to illustrate the interactive scenario of the polling strategy determination method. Figure 3 This is a schematic diagram of the interaction scenario of the polling strategy determination method provided in the embodiments of this application, such as... Figure 3 As shown, the interactive process of determining the polling strategy within the polling cycle may include the following steps:

[0160] Step 3-1: During the polling cycle, the client sends a request message to the server, which carries request information.

[0161] Step 3-2: The server sends a response message to the client. The response message carries response information, including: request result, round-trip time, number of bytes transmitted, bandwidth, etc.

[0162] In this way, network status parameters can be obtained by collecting network request data between the client and the server.

[0163] Step 3-3: Based on the response and request information, the client records the request data and sends the request data to the network quality awareness module;

[0164] Steps 3-4: The network quality awareness module updates the network status parameters based on the request data;

[0165] Steps 3-5: The network quality awareness module discretizes the current values ​​of round-trip delay, packet loss rate and task progress among various network state parameters to obtain the current state S corresponding to the current network state parameters, and sends the state S to the RL dynamic parameter tuning module.

[0166] Steps 3-6: The RL dynamic parameter tuning module, based on the current state S, uses a pre-built state-action value table to fine-tune the strategy parameters of various polling strategies to obtain optimized strategy parameter values, including the parameter values ​​of the backoff coefficient, prediction coefficient, and smoothing coefficient; and sends the optimized strategy parameter values ​​to the dynamic strategy module.

[0167] Steps 3-7: The dynamic strategy module selects the optimal polling strategy based on the network status parameters and calculates the polling interval corresponding to the optimal polling strategy; then sends the polling interval corresponding to the optimal polling strategy to the status compensation module.

[0168] Steps 3-8: When the network is normal, the state compensation module schedules the client's pending requests (i.e., the next request) according to the polling interval time corresponding to the optimal polling strategy.

[0169] Steps 3-9: When a network anomaly occurs (such as a network interruption), the status compensation module adds the client's pending requests to the cache queue and starts the heartbeat monitoring function. The heartbeat monitoring function refers to re-acquiring the connection status at preset fixed intervals (e.g., every 5 seconds) to confirm the client's connection status until the network returns to normal. Then, according to the polling interval corresponding to the optimal polling strategy, the pending requests in the cache queue are executed.

[0170] The polling strategy determination method provided in this application has at least the following beneficial effects:

[0171] By acquiring various network state parameters in real time (such as round-trip latency, packet loss rate, bandwidth, and task progress), and using reinforcement learning algorithms to dynamically optimize the policy parameters (such as backoff coefficient, smoothing coefficient, and prediction coefficient) of various polling strategies, it is possible to intelligently adjust the policy parameters according to the current network state, resulting in optimized policy parameter values ​​that are more adaptable to the current network environment. Based on these network state parameters and the optimized policy parameter values, the optimal polling strategy and its corresponding polling interval are determined. This allows for dynamic adaptive selection of the optimal polling strategy from among various options, ensuring that requests are polled using the optimal polling interval corresponding to the current network environment. Thus, it overcomes the limitations of fixed parameters, enabling automatic, timely, and intelligent adjustment of the polling strategy under different network scenarios (such as stable network quality, unstable network quality, high network load, and stable network processing speed). It maintains high adaptability in complex network scenarios, effectively improving network quality optimization, increasing polling efficiency, reducing the number of invalid requests, and minimizing resource waste. Consequently, it can reduce cloud computing call costs and bandwidth overhead. Furthermore, the reduction in invalid requests lowers the pressure on the server-side request processing, allowing for a corresponding reduction in the number of servers in the cluster. By monitoring network status at preset fixed intervals to maintain connection activity during polling intervals, polling tasks can be executed normally when the network is normal. However, when network anomalies occur (such as network interruptions), requests from the period of anomaly are added to a cache queue to await network recovery, preventing task loss due to network issues. Once the network recovers, tasks in the cache queue are restarted via a scheduled task. This improves the reliability of network service quality.

[0172] This application also provides a polling strategy determination device, such as... Figure 4 As shown, the device includes:

[0173] Network monitoring unit 401 is configured to obtain various network status parameters;

[0174] The parameter adjustment unit 402 is configured to optimize the policy parameters of various polling strategies based on multiple network state parameters using a reinforcement learning algorithm, thereby obtaining optimized policy parameter values.

[0175] The dynamic policy unit 403 is configured to determine the optimal polling policy and its corresponding polling interval based on multiple network state parameters and optimized policy parameter values.

[0176] Embodiments of this application provide an electronic device that can be applied to... Figure 1 In a corresponding embodiment, a polling strategy determination method is provided, such as... Figure 5As shown, the electronic device 500 includes: a processor 501, a memory 502, and a communication bus 503, wherein:

[0177] The communication bus 503 is configured to enable communication between the processor 501 and the memory 502.

[0178] The memory 502 is configured to store computer programs that can run on the processor 501;

[0179] Processor 501, when running a computer program, performs the following steps:

[0180] Obtain various network state parameters;

[0181] Based on the aforementioned network state parameters, reinforcement learning algorithms are used to fine-tune the policy parameters of various polling strategies to obtain optimized policy parameter values.

[0182] Based on the various network state parameters and the optimized strategy parameter values, the optimal polling strategy and its corresponding polling interval are determined.

[0183] In some exemplary embodiments, the processor 501, when running a computer program, implements the steps of the polling strategy determination method shown in one or more of the exemplary embodiments described above. Further details are omitted here; however, they can be understood by referring to the descriptions in the embodiments of the polling strategy determination method described above.

[0184] The various components in electronic device 500 are coupled together via communication bus 503. It is understood that communication bus 503 is used to achieve communication between these components. In addition to a data bus, communication bus 503 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 5 All buses are labeled as communication bus 503.

[0185] A processor can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, a general-purpose processor can be a microprocessor or any conventional processor.

[0186] Embodiments of this application provide a computer storage medium storing one or more programs, which can be executed by one or more processors to achieve, for example... Figure 1 The steps are shown.

[0187] It should be noted that the aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; it can also be various terminals that include one or any combination of the above-mentioned memory, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0188] Embodiments of this application provide a computer program product, including a computer program executable by a processor 501 of an electronic device 500, to perform tasks such as... Figure 1 The steps are shown.

[0189] It should be noted that the descriptions of the same steps and contents in the above-mentioned apparatus, equipment, storage medium, and product embodiments as in the method embodiments can be found in the descriptions in the method embodiments, and will not be repeated here.

[0190] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0191] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0192] Furthermore, in the various embodiments of this application, all functional units can be integrated into one processing module, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0193] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict. The features disclosed in the several product embodiments provided in this application can be arbitrarily combined to obtain new product embodiments without conflict. The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined to obtain new method embodiments or device embodiments without conflict.

[0194] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for determining a polling strategy, characterized in that, include: Obtain various network state parameters; Based on the aforementioned multiple network state parameters, reinforcement learning algorithms are used to fine-tune the policy parameters of various polling strategies to obtain optimized policy parameter values. The multiple network state parameters include various parameters such as round-trip time, packet loss rate, bandwidth, and task progress. The policy parameters of the various polling strategies include various parameters such as a backoff coefficient for controlling the aggressiveness of backoff in a preset first polling strategy, a prediction coefficient for correcting the calculation of remaining task volume in a preset second polling strategy, and a smoothing coefficient for adjusting the smoothing sensitivity of the round-trip time in a preset third polling strategy. Based on the various network state parameters and the optimized strategy parameter values, the optimal polling strategy and its corresponding polling interval are determined.

2. The method according to claim 1, characterized in that, The step of using reinforcement learning algorithms to fine-tune the policy parameters of various polling strategies based on the multiple network state parameters to obtain optimized policy parameters includes: Based on the round-trip delay, packet loss rate, and task progress among the various network state parameters, reinforcement learning algorithms are used to fine-tune the policy parameters of various polling strategies to obtain optimized policy parameter values.

3. The method according to claim 2, characterized in that, Based on the round-trip delay, packet loss rate, and task progress among the various network state parameters, a reinforcement learning algorithm is used to fine-tune the policy parameters of various polling strategies to obtain optimized policy parameter values, including: Discretize the current values ​​of round-trip delay, packet loss rate, and task progress among the various network state parameters to obtain the current state vector; Based on the current state vector, the policy parameters of various polling strategies are tuned using a pre-built state-action value table to obtain optimized policy parameter values.

4. The method according to claim 3, characterized in that, Before discretizing the current values ​​of round-trip delay, packet loss rate, and task progress among the various network state parameters to obtain the current state vector, the method further includes: The elements of a reinforcement learning algorithm are predefined to obtain the state-action value table. The elements include a state space, an action space, and a reward function. The state space is obtained by discretizing the round-trip time, packet loss rate, and task progress. Each state in the state space corresponds to a combination of network state values. The action space is obtained by discretizing the policy parameters of various polling strategies. Each action in the action space corresponds to a combination of policy parameter values. The reward function is used to update the state-action value table.

5. The method according to any one of claims 1 to 4, characterized in that, The step of determining the optimal polling strategy and its corresponding polling interval based on the various network state parameters and the optimized strategy parameter values ​​includes: Determine whether the various network state parameters meet a preset first condition; When it is determined that the multiple network state parameters meet a preset first condition, the preset first polling strategy corresponding to the preset first condition is determined as the optimal polling strategy. Based on the current value of the packet loss rate among the various network state parameters and the backoff coefficient value among the optimized strategy parameter values, the first polling interval time of the preset first polling strategy is determined as the polling interval time corresponding to the optimal polling strategy.

6. The method according to any one of claims 1 to 4, characterized in that, The step of determining the optimal polling strategy and its corresponding polling interval based on the various network state parameters and the optimized strategy parameter values ​​includes: Determine whether the various network state parameters meet the preset second condition; When it is determined that the multiple network state parameters meet the preset second condition, the preset second polling strategy corresponding to the preset second condition is determined as the optimal polling strategy; Based on the current values ​​of task progress and bandwidth among the various network state parameters, as well as the prediction coefficient values ​​among the optimized strategy parameter values, the second polling interval time of the preset second polling strategy is determined as the polling interval time corresponding to the optimal polling strategy.

7. The method according to any one of claims 1 to 4, characterized in that, The step of determining the optimal polling strategy and its corresponding polling interval based on the various network state parameters and the optimized strategy parameter values ​​includes: Determine whether the various network state parameters meet the preset third condition; When it is determined that the multiple network state parameters satisfy a preset third condition, the preset third polling strategy corresponding to the preset third condition is determined as the optimal polling strategy. Based on the current value of the round-trip delay time among the various network state parameters and the smoothing coefficient value among the optimized strategy parameter values, the third polling interval time of the preset third polling strategy is determined as the polling interval time corresponding to the optimal polling strategy.

8. The method according to any one of claims 1 to 4, characterized in that, After determining the optimal polling strategy and its corresponding polling interval based on the various network state parameters and the optimized strategy parameter values, the method further includes: The client's pending requests are scheduled according to the polling interval time corresponding to the optimal polling strategy.

9. The method according to claim 8, characterized in that, The step of scheduling the client's pending requests according to the polling interval time corresponding to the optimal polling strategy includes: Get the connection status of either the server or the client; Based on whether the connection status is online, determine whether a network anomaly has occurred; When it is determined that no network anomalies have occurred, the client's pending requests are scheduled according to the polling interval time corresponding to the optimal polling strategy.

10. The method according to claim 9, characterized in that, After determining whether a network anomaly has occurred based on whether the connection status is online, the method further includes: When a network anomaly is detected, the client's pending requests are added to the cache queue; The connection status is reacquired at preset fixed intervals, and the network anomaly is re-determined based on whether the reacquired connection status is online. This process continues until no network anomaly is detected. Then, the pending requests in the cache queue are executed according to the polling interval corresponding to the optimal polling strategy.

11. A polling strategy determination device, characterized in that, include: The network monitoring unit is configured to obtain various network status parameters; The parameter adjustment unit is configured to fine-tune the policy parameters of various polling strategies based on the aforementioned multiple network state parameters using a reinforcement learning algorithm to obtain optimized policy parameter values. The multiple network state parameters include various parameters such as round-trip time, packet loss rate, bandwidth, and task progress. The policy parameters of the multiple polling strategies include various parameters such as a backoff coefficient for a preset first polling strategy to control the aggressiveness of backoff, a prediction coefficient for a preset second polling strategy to correct the calculation of remaining task volume, and a smoothing coefficient for a preset third polling strategy to adjust the smoothing sensitivity of the round-trip time. The dynamic strategy unit is configured to determine the optimal polling strategy and its corresponding polling interval based on the various network state parameters and the optimized strategy parameter values.

12. An electronic device, characterized in that, include: A processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of the method as claimed in any one of claims 1 to 10.

13. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as claimed in any one of claims 1 to 10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as claimed in any one of claims 1 to 10.

Images