Methods and apparatus for configuring cache queue thresholds, storage media and electronic devices

Abstract

This application discloses a method, apparatus, storage medium, and electronic device for configuring a buffer queue threshold. The method includes: when the target control period for the target buffer queue of a target device has been reached, obtaining the queue information of the target buffer queue after adjusting the queue threshold of the target buffer queue according to an initial adjustment operation during a reference control period prior to the current time; updating the initial prediction model based on the queue information to obtain a target prediction model; inputting the current target queue information of the target buffer queue into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model; determining the target adjustment operation from multiple threshold adjustment operations based on the operation information; and adjusting the current target threshold value of the target buffer queue according to the target adjustment operation. This technical solution solves the problem of low configuration efficiency of queue thresholds in related technologies.

Description

Technical Field

[0001] This application relates to the field of communications, and more specifically, to a method and apparatus for configuring a buffer queue threshold, a storage medium, and an electronic device. Background Technology

[0002] To prevent packet loss during data transmission, switching equipment uses buffer queues to cache received but not yet processed traffic. However, when there's a gap between the receiving and processing speeds, the buffer queue may be full while still receiving pending traffic, leading to further packet loss. To avoid this, congestion control mechanisms are implemented, setting queue thresholds and buffer limits. When the amount of traffic in the buffer queue exceeds the threshold, the receiving speed of pending traffic is reduced; when the amount exceeds the threshold, acceptance of pending traffic is stopped, improving data transmission efficiency. However, current methods for determining queue thresholds rely on known scenarios. A reasonable queue threshold can only be determined when multiple parameters of the switching equipment fall within these known scenarios. In unknown scenarios, it's difficult to determine a suitable queue threshold using simple parameters.

[0003] No effective solution has yet been proposed to address the issues of low configuration efficiency of queue thresholds in related technologies. Summary of the Invention

[0004] This application provides a method and apparatus for configuring a cache queue threshold, a storage medium, and an electronic device to at least solve the problem of low configuration efficiency of queue thresholds in related technologies.

[0005] According to one embodiment of this application, a method for configuring a cache queue threshold is provided, comprising: when the target control period for a target cache queue of a target device is reached, obtaining queue information of the target cache queue after adjusting the queue threshold of the target cache queue according to an initial adjustment operation during a reference control period prior to the current time, wherein the target cache queue is used to cache data transmitted from other devices to the target device, the queue information is used to indicate the transmission efficiency of transmitting data from other devices to the target device through the target cache queue, and the initial adjustment operation is selected from multiple threshold adjustment operations using an initial prediction model, the initial prediction model being used to predict... The process involves: executing the threshold adjustment operation to achieve the adjustment effect on the adjustment target, which includes improving the data transmission efficiency from the other devices to the target device; updating the initial prediction model based on the queue information to obtain a target prediction model; inputting the current target queue information of the target buffer queue into the target prediction model to obtain operation information for each threshold adjustment operation output by the target prediction model, wherein the operation information indicates the adjustment effect of each adjustment operation on the adjustment target; determining the target adjustment operation from multiple threshold adjustment operations based on the operation information; and adjusting the current target threshold value of the target buffer queue according to the target adjustment operation.

[0006] Optionally, updating the initial prediction model based on the queue information includes: using the initial prediction model to select reference adjustment operations from the plurality of threshold adjustment operations based on the queue information; calculating modified adjustment parameters of the initial adjustment operation based on reference adjustment parameters of the reference adjustment operation, initial adjustment parameters of the initial adjustment operation, and gain parameters of the initial adjustment operation, wherein the reference adjustment parameters are used to indicate the adjustment effect of the reference adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period, the initial adjustment parameters are used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the reference control period, the gain parameters are used to indicate the adjustment effect achieved on the adjustment target after executing the initial adjustment operation, and the modified adjustment parameters are used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period; and updating the initial prediction model using the modified adjustment parameters.

[0007] Optionally, calculating the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters of the reference adjustment operation, the initial adjustment parameters of the initial adjustment operation, and the gain parameters of the initial adjustment operation includes: converting the cache parameters of the target cache queue within the reference control period to a first queue state and converting the cache parameters of the target cache queue within the target control period to a second queue state, wherein the queue state of the target cache queue corresponds to the capacity range of the occupied cache capacity of the target cache queue, the cache parameters are used to indicate the cache space of the target cache queue occupied by the target cache queue storing the data, and the queue information includes the cache parameters; using the initial prediction model to predict the initial adjustment parameters of the initial adjustment operation in the first queue state, and the reference adjustment parameters of the reference adjustment operation in the second queue state; and calculating the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters, the initial adjustment parameters, and the gain parameters of the initial adjustment operation.

[0008] Optionally, calculating the modified adjustment parameter of the initial adjustment operation based on the reference adjustment parameter, the initial adjustment parameter, and the gain parameter of the initial adjustment operation includes: detecting the relationship between the cache parameter and the cache threshold of the target cache queue; if the cache parameter is detected to be greater than or equal to the cache threshold, obtaining the difference between the cache threshold and the cache parameter to obtain a first gain parameter; calculating the modified adjustment parameter of the initial adjustment operation based on the reference adjustment parameter, the initial adjustment parameter, and the first gain parameter; if the cache parameter is detected to be less than the cache threshold, obtaining the reciprocal of the difference between the cache threshold and the cache parameter, calculating the product of the reciprocal and the capacity of the target cache queue to obtain a second gain parameter, wherein the capacity is used to indicate the data cache space of the target cache queue; and calculating the modified adjustment parameter of the initial adjustment operation based on the reference adjustment parameter, the initial adjustment parameter, and the second gain parameter.

[0009] Optionally, calculating the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters of the reference adjustment operation, the initial adjustment parameters of the initial adjustment operation, and the gain parameters of the initial adjustment operation includes: calculating the modified adjustment parameters using the following formula:

[0010] Q (t+1) =Q t +α(r+γQ t '-Qt);

[0011] Among them, Q (t+1) The changed adjustment parameter, Q tIt is the initial adjustment parameter, Q t ' is the reference adjustment parameter, r is the gain parameter, α is the learning rate, and γ is the discount factor.

[0012] Optionally, the step of using the initial prediction model to select a reference adjustment operation from the plurality of threshold adjustment operations based on the queue information includes: generating a target random number; comparing the target random number with a current random number threshold; if the target random number is greater than or equal to the current random number threshold, selecting the threshold adjustment operation with the best adjustment effect predicted by the initial prediction model from the plurality of threshold adjustment operations as the reference adjustment operation; and if the target random number is less than the current random number threshold, randomly selecting one of the threshold adjustment operations from the plurality of threshold adjustment operations as the reference adjustment operation.

[0013] Optionally, determining the target adjustment operation from the plurality of threshold adjustment operations based on the operation information includes: selecting the threshold adjustment operation with the best adjustment effect predicted by the target prediction model from the plurality of threshold adjustment operations as the target adjustment operation.

[0014] According to another embodiment of this application, a configuration device for a cache queue threshold is also provided, comprising: an acquisition module, configured to, when the target control period for the target cache queue of a target device is reached, acquire queue information of the target cache queue after adjusting the queue threshold of the target cache queue according to an initial adjustment operation during a reference control period prior to the current time, wherein the target cache queue is used to cache data transmitted from other devices to the target device, the queue information is used to indicate the transmission efficiency of transmitting data from other devices to the target device through the target cache queue, and the initial adjustment operation is selected from multiple threshold adjustment operations using an initial prediction model, the initial prediction model being used to predict the execution of the threshold adjustment operation. The system comprises: an adjustment module for achieving the adjustment effect of the adjustment target, wherein the adjustment target includes improving the data transmission efficiency from the other devices to the target device; an update module for updating the initial prediction model according to the queue information to obtain a target prediction model; an input module for inputting the current target queue information of the target buffer queue into the target prediction model to obtain operation information of each threshold adjustment operation output by the target prediction model, wherein the operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target; a determination module for determining the target adjustment operation from multiple threshold adjustment operations according to the operation information; and an adjustment module for adjusting the current target threshold value of the target buffer queue according to the target adjustment operation.

[0015] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described configuration method for cache queue thresholds at runtime.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the configuration method of the cache queue threshold through the computer program.

[0017] In this embodiment, when the target control period for the target buffer queue of the target device is reached, the queue information of the target buffer queue is obtained after adjusting the queue threshold of the target buffer queue according to the initial adjustment operation during the reference control period prior to the current time; the initial prediction model is updated according to the queue information to obtain the target prediction model; the current target queue information of the target buffer queue is input into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model; the target adjustment operation is determined from multiple threshold adjustment operations according to the operation information; the current target threshold value of the target buffer queue is adjusted according to the target adjustment operation, that is, when the target control period for the target buffer queue is reached... In this case, after adjusting the queue threshold according to the initial adjustment operation obtained using the initial prediction model within the reference control cycle, the target device transmits data from other devices to the target device. Further, the improvement effect of the initial adjustment operation on the data transmission efficiency from other devices to the target device can be obtained. Using this improvement effect to update the initial prediction model, a more accurate target prediction model can be obtained to predict the adjustment effect of the threshold adjustment operation on improving data transmission efficiency. Then, using the target prediction model and current queue information to determine the target adjustment operation, a better adjustment effect on improving data transmission efficiency can be achieved, ultimately resulting in a more suitable queue threshold. The above technical solution solves the problem of low queue threshold configuration efficiency in related technologies, achieving the technical effect of improving queue threshold configuration efficiency. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the hardware environment for a method of configuring a cache queue threshold according to an embodiment of this application;

[0021] Figure 2 This is a flowchart of a method for configuring a cache queue threshold according to an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of a congestion control mechanism according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of a queue threshold state decomposition method according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of an ECN threshold adjustment operation setting method according to an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of a reward and punishment mechanism for ECN threshold adjustment actions according to an embodiment of this application. Figure 1 ;

[0026] Figure 7 This is a schematic diagram of a reward and punishment mechanism for ECN threshold adjustment actions according to an embodiment of this application. Figure 2 ;

[0027] Figure 8 This is a schematic diagram of an action value function table according to an embodiment of this application;

[0028] Figure 9 This is a structural block diagram of a cache queue threshold configuration device according to an embodiment of this application. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] The methods and embodiments provided in this application can be executed on a computer terminal, device terminal, or similar computing device. Taking running on a computer terminal as an example, Figure 1 This is a schematic diagram of a hardware environment for configuring a cache queue threshold according to an embodiment of this application. Figure 1 As shown, a computer terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. In one exemplary embodiment, the computer terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer terminal described above. For example, the computer terminal may also include components that are more complex than those described above. Figure 1 The more or fewer components shown, or having the same Figure 1 Equivalent functions or ratios shown Figure 1 The functions shown have more different configurations.

[0032] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the message push sending method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0033] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the computer terminal. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0034] This embodiment provides a method for configuring a cache queue threshold, applied to the aforementioned computer terminal. Figure 2 This is a flowchart of a method for configuring a cache queue threshold according to an embodiment of this application. The process includes the following steps:

[0035] Step S202: When the target control period for the target buffer queue of the target device is reached, obtain the queue information of the target buffer queue after adjusting the queue threshold of the target buffer queue according to the initial adjustment operation according to the reference control period before the current time. The target buffer queue is used to buffer data transmitted from other devices to the target device. The queue information is used to indicate the transmission efficiency of data transmission from other devices to the target device through the target buffer queue. The initial adjustment operation is selected from multiple threshold adjustment operations using an initial prediction model. The initial prediction model is used to predict the adjustment effect of executing the threshold adjustment operation on the adjustment target. The adjustment target includes improving the data transmission efficiency from other devices to the target device.

[0036] Step S204: Update the initial prediction model based on the queue information to obtain the target prediction model;

[0037] Step S206: Input the current target queue information of the target cache queue into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model. The operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target.

[0038] Step S208: Determine the target adjustment operation from multiple threshold adjustment operations based on the operation information;

[0039] Step S210: Adjust the current target threshold value of the target cache queue according to the target adjustment operation.

[0040] Through the above steps, when the target control cycle for the target buffer queue of the target device is reached, the queue information of the target buffer queue is obtained after adjusting the queue threshold of the target buffer queue according to the initial adjustment operation during the reference control cycle prior to the current time; the initial prediction model is updated based on the queue information to obtain the target prediction model; the current target queue information of the target buffer queue is input into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model; the target adjustment operation is determined from multiple threshold adjustment operations based on the operation information; the current target threshold value of the target buffer queue is adjusted according to the target adjustment operation, i.e., when the target control cycle of the target buffer queue is reached. The method involves collecting data from other devices to the target device within a reference control cycle, adjusting the queue threshold according to the initial adjustment operation obtained using the initial prediction model, and further obtaining the improvement effect of the initial adjustment operation on the data transmission efficiency from other devices to the target device. By updating the initial prediction model with this improvement effect, a more accurate target prediction model can be obtained, predicting the adjustment effect of the threshold adjustment operation on improving data transmission efficiency. Then, using the target prediction model and current queue information, the target adjustment operation is determined, achieving a better adjustment effect on improving data transmission efficiency, ultimately resulting in a more suitable queue threshold. This technical solution solves the problem of low queue threshold configuration efficiency in related technologies, achieving the technical effect of improving queue threshold configuration efficiency.

[0041] Optionally, in this embodiment of the application, a congestion control mechanism is provided to alleviate congestion in the transmission of packets by the switching equipment. Figure 3 This is a schematic diagram of a congestion control mechanism according to an embodiment of this application. Figure 3As shown, the target device (i.e., the network switch) receives packets from other devices (i.e., the sending server) and forwards the received packets to the receiving server. If there is congestion at the port's outgoing direction, packets need to be marked with ECN to notify the server that congestion has occurred on the link and the packet transmission rate needs to be appropriately reduced. The target device determines whether congestion has occurred by checking if the queue buffer in the target buffer queue exceeds the ECN threshold (i.e., the queue threshold). If the number of buffered packets in the queue exceeds the ECN threshold, it is considered that the queue is congested, and packets sent from the queue will be marked with ECN to notify the server. To achieve flow control of the lossless queue and alleviate buffer congestion, in addition to the ECN threshold, there is also a PFC threshold (i.e., the buffer threshold). When the network switch's queue buffer reaches the PFC threshold, it notifies the upstream device to stop sending traffic. Triggering ECN marking only notifies the server to slow down; therefore, when congestion occurs, it is desirable to first trigger the ECN threshold to notify the server to slow down and alleviate congestion, avoiding triggering PFC as much as possible to prevent transmission interruption. However, there is a time lag between the network switch detecting congestion in its queue buffer and triggering a congestion flag, and the sending server receiving a CNP congestion notification message and recognizing congestion on the network link, thus reducing its packet transmission rate. During this time, the sending server continues to send traffic to the switch at the original packet transmission rate, causing the device's queue buffer congestion to continuously change, eventually triggering PFC flow control and suspending traffic transmission. Therefore, it is necessary to set a reasonable ECN threshold so that the buffer space between the ECN threshold and the PFC threshold can accommodate the traffic sent during the period from triggering the congestion flag to the sending server's rate reduction. However, this ECN threshold value is difficult to determine and may be dynamically changing. Therefore, a method is needed to determine a reasonable ECN threshold value based on changes in the network environment. According to the method proposed in this application, the ECN threshold can be reasonably adjusted according to changes in the network environment through continuous learning of the predictive model.

[0042] In the technical solution provided in step S202 above, the target device may be, but is not limited to, a device that receives data sent by the front-end device and needs to process the received data. For example, it may be a switch used to receive data sent by the front-end device and transmit the data to the back-end device.

[0043] Optionally, in this embodiment, the target cache queue may be used, but is not limited to, to cache received data to be processed for the target device. The target device may be configured with, but is not limited to, multiple target cache queues to cache data from different services or data requiring different processing.

[0044] Optionally, in this embodiment, the threshold adjustment frequency can be set according to the target device's data reception and processing status. A fixed threshold adjustment frequency is used to set the timing for adjusting the queue threshold of the target buffer queue. The threshold adjustment timing is reached when the target control period is reached. The target control period is the time interval between the current threshold adjustment timing and the next threshold adjustment timing. The reference control period is the time interval between the previous threshold adjustment timing and the current threshold adjustment timing. The aforementioned time intervals need to exceed a certain duration to fully demonstrate the impact of the threshold adjustment operation on the transmission efficiency of the target buffer queue in receiving data sent by other devices.

[0045] Optionally, in the embodiments of this application, the queue information may be, but is not limited to, the amount of data stored in the target cache queue, or the proportion of the cache space occupied by the data stored in the target cache queue, etc. This application does not make specific limitations on this.

[0046] In the technical solution provided in step S204 above, the initial prediction model and the target prediction model may, but are not limited to, maintain the adjustment effect of different threshold adjustment operations on the adjustment target under different queue states.

[0047] In one exemplary embodiment, updating the initial prediction model based on queue information includes: selecting reference adjustment operations from multiple threshold adjustment operations using the initial prediction model based on the queue information; calculating modified adjustment parameters of the initial adjustment operation based on reference adjustment parameters of the reference adjustment operation, initial adjustment parameters of the initial adjustment operation, and gain parameters of the initial adjustment operation, wherein the reference adjustment parameters are used to indicate the adjustment effect of the reference adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period, the initial adjustment parameters are used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the reference control period, the gain parameters are used to indicate the adjustment effect achieved on the adjustment target after executing the initial adjustment operation, and the modified adjustment parameters are used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period; and updating the initial prediction model by modifying the adjustment parameters.

[0048] Optionally, in this embodiment, queue information may be input into the initial prediction model to obtain the operation information of each threshold adjustment operation output by the initial prediction model. The operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target. The threshold adjustment operation with the best adjustment effect indicated by the operation information is selected from multiple threshold adjustment operations as the reference adjustment operation.

[0049] Optionally, in the embodiments of this application, the reference adjustment parameter may be, but is not limited to, the operation information corresponding to the reference adjustment operation output by the initial prediction model when the queue information of the target control cycle is input to the initial prediction model, and the initial adjustment parameter may be, but is not limited to, the operation information corresponding to the initial adjustment operation output by the initial prediction model when the queue information of the reference control cycle is input to the initial prediction model.

[0050] Optionally, in this embodiment, updating the initial prediction model based on the queue information includes, but is not limited to, calculating the gain parameter of the initial adjustment operation based on the queue information and updating the initial prediction model using the gain parameter.

[0051] In one exemplary embodiment, calculating the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters of the reference adjustment operation, the initial adjustment parameters of the initial adjustment operation, and the gain parameters of the initial adjustment operation includes: converting the cache parameters of the target cache queue within the reference control period to a first queue state and converting the cache parameters of the target cache queue within the target control period to a second queue state, wherein the queue state of the target cache queue corresponds to the capacity range of the occupied cache capacity of the target cache queue, the cache parameters are used to indicate the cache space of the target cache queue occupied by the data stored in the target cache queue, and the queue information includes the cache parameters; predicting the initial adjustment parameters of the initial adjustment operation in the first queue state and the reference adjustment parameters of the reference adjustment operation in the second queue state using an initial prediction model; and calculating the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters, the initial adjustment parameters, and the gain parameters of the initial adjustment operation.

[0052] Optionally, in this embodiment, the target cache queue can be divided into multiple capacity ranges with the same cache capacity according to the cache capacity of the target cache queue. The cache parameters can be matched with different capacity ranges according to the cache space occupied by the data stored in the target cache queue as indicated by the cache parameters. Then, the cache parameters can be converted into the corresponding queue states by means of the correspondence between the capacity ranges and the queue states.

[0053] In one exemplary embodiment, calculating the modified adjustment parameters of the initial adjustment operation based on reference adjustment parameters, initial adjustment parameters, and gain parameters of the initial adjustment operation includes: detecting the relationship between cache parameters and a cache threshold of the target cache queue; if the cache parameters are detected to be greater than or equal to the cache threshold, obtaining the difference between the cache threshold and the cache parameters to obtain a first gain parameter; calculating the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters, initial adjustment parameters, and the first gain parameter; if the cache parameters are detected to be less than the cache threshold, obtaining the reciprocal of the difference between the cache threshold and the cache parameters, calculating the product of the reciprocal and the capacity of the target cache queue to obtain a second gain parameter, wherein the capacity is used to indicate the data cache space possessed by the target cache queue; and calculating the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters, initial adjustment parameters, and the second gain parameter.

[0054] Optionally, in this embodiment, the cache threshold may be, but is not limited to, the capacity of the target cache queue, or a portion thereof. The target device is configured to stop accepting data transmitted from other devices when the amount of data cached in the target cache queue exceeds the cache threshold. That is, when the cache threshold is exceeded, a data interruption or packet loss may occur. To ensure the efficiency of data reception by the target device, the occurrence of exceeding the cache threshold should be avoided.

[0055] In one exemplary embodiment, calculating the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters of the reference adjustment operation, the initial adjustment parameters of the initial adjustment operation, and the gain parameters of the initial adjustment operation includes: calculating the modified adjustment parameters using the following formula:

[0056] Q (t+1) =Q t +α(r+γQ t '-Q t );

[0057] Among them, Q (t+1) It involves changing the adjustment parameter, Q. t It is the initial adjustment parameter, Q t ' is the reference adjustment parameter, r is the gain parameter, α is the learning rate, and γ is the discount factor.

[0058] Optionally, in this embodiment, the Q-value represents an estimate of the cumulative reward expected to be obtained by performing a threshold adjustment operation under a given target cache queue cache condition. The reward is the adjustment effect achieved on the adjustment target. Specifically, the Q-value represents the sum of the expected long-term rewards that can be obtained after taking the selected threshold adjustment operation starting from the current cache condition. This sum includes the immediate reward r and the discounted rewards obtained from all future adjustment operations. α is a number between 0 and 1, which controls the degree of influence of new information on the old Q-value. γ is a number between 0 and 1, which determines the importance of future rewards relative to current rewards. If γ is close to 1, then future rewards are almost as important as current rewards; if γ is close to 0, then only immediate rewards are important.

[0059] In one exemplary embodiment, selecting a reference adjustment operation from multiple threshold adjustment operations using an initial prediction model based on queue information includes: generating a target random number; comparing the target random number with a current random number threshold; if the target random number is greater than or equal to the current random number threshold, selecting the threshold adjustment operation with the best adjustment effect predicted by the initial prediction model from the multiple threshold adjustment operations as a reference adjustment operation; if the target random number is less than the current random number threshold, randomly selecting a threshold adjustment operation from the multiple threshold adjustment operations as a reference adjustment operation.

[0060] Optionally, in this embodiment, this includes, but is not limited to, generating a target random number threshold that is less than the current random number threshold after each filtering and adjustment operation, as a random number threshold to be compared with the generated target random number during the next filtering and adjustment operation.

[0061] In the technical solution provided in step S206 above, it is possible, but not limited to, that after the current target queue information is input into the target prediction model, the target prediction model will output the predicted adjustment effect that can be achieved by taking different threshold adjustment operations on the current queue threshold value under the data transmission efficiency indicated by the current target queue information as operation information for different threshold adjustment operations.

[0062] In the technical solution provided in step S208 above, determining the target adjustment operation from multiple threshold adjustment operations based on the operation information can, but is not limited to, selecting the threshold adjustment operation with the best adjustment effect indicated by the operation information as the target adjustment operation with a higher probability, and selecting the threshold adjustment operation with the second best adjustment effect indicated by the operation information as the target adjustment operation with a lower probability.

[0063] In one exemplary embodiment, determining a target adjustment operation from multiple threshold adjustment operations based on operational information includes: selecting the threshold adjustment operation with the best adjustment effect predicted by the target prediction model from the multiple threshold adjustment operations as the target adjustment operation.

[0064] In the technical solution provided in step S210 above, adjusting the current target threshold value of the target buffer queue according to the target adjustment operation includes, but is not limited to, calculating the target threshold value after the target adjustment operation on the unadjusted queue threshold, and determining the calculated target threshold value as the new queue threshold.

[0065] To better understand the above process, the following description will be based on optional embodiments, but these are not intended to limit the technical solutions of the embodiments of this application.

[0066] Optionally, this application embodiment also provides a method for adjusting the threshold of a buffer queue based on the SARSA algorithm. The SARSA algorithm is an online learning algorithm in reinforcement learning that learns optimal behavioral policies through the interaction between the agent and the environment. This algorithm is an online policy iterative algorithm, the core of which lies in calculating the update formula of the Q-action value function. The Q-function is learned through online interaction, and its update rules are as follows:

[0067] In each iteration, the agent experiences state s, performs action x, observes reward r and a new state s', and takes action x', thereby obtaining the action-value function Q. The update formula for the Q function is:

[0068] Q (t+1) (s,x)=Q t (s,x)+α[r+γQ t (s',x')-Q t (s,x)];

[0069] Where α is the learning rate, r is the reward obtained by performing an action in the current state, γ is the discount factor, s' is the new state, and x' is the action in the new state.

[0070] The SARSA-based method for adjusting the threshold of a buffer queue includes the following steps:

[0071] Step S1, ECN waterline (queue threshold) state decomposition:

[0072] Figure 4 This is a schematic diagram of a queue threshold state decomposition method according to an embodiment of this application, as shown below. Figure 4As shown, the ECN threshold is set based on the target cache queue (Queue). The target cache queue is used to cache packets that need to be forwarded. When the number of cached packets exceeds the queue's ECN threshold, an ECN marking is triggered on the packets. The maximum number of packets (i.e., capacity) that the target cache queue can cache is N. The target cache queue is divided into 10 parts, and each part can be set as a state s in the SARSA learning algorithm: when the number of packets is between 1 and N / 10, it is set as state s1; when the number of packets (i.e., queue information) is between N / 10+1 and 2N / 10, it is set as state s2; when the number of packets is between 2N / 10+1 and 3N / 10, it is set as state s3; when the number of packets is between 3N / 10+1 and 4N / 10, it is set as state s4. s4; When the number of messages is between 4N / 10+1 and 5N / 10, set to state s5; when the number of messages is between 5N / 10+1 and 6N / 10, set to state s6; when the number of messages is between 6N / 10+1 and 7N / 10, set to state s7; when the number of messages is between 7N / 10+1 and 8N / 10, set to state s8; when the number of messages is between 8N / 10+1 and 9N / 10, set to state s9; when the number of messages is between 9N / 10+1 and N, set to state s10.

[0073] Step S2, ECN threshold adjustment operation settings:

[0074] Figure 5 This is a schematic diagram of an ECN threshold adjustment operation setting method according to an embodiment of this application. Figure 5 As shown, 'a' is the initial position of the ECN threshold; 'a+step' is the position after adding 'step' to the ECN threshold, where 'step' is the step size for ECN waterline adjustment; 'a+2step' is the position after adding 2 steps to the ECN threshold; 'a+3step' is the position after adding 3 steps to the ECN threshold; 'a-step' is the position after subtracting 'step' from the ECN threshold, where 'step' is the step size for ECN waterline adjustment; 'a-2step' is the position after subtracting 2 steps from the ECN threshold; and 'a-3step' is the position after subtracting 3 steps from the ECN threshold. Adding or subtracting 1 to 3 steps represents the selectable threshold adjustment operations.

[0075] Step S3, setting the reward and penalty mechanism after ECN threshold adjustment:

[0076] To achieve flow control over lossless queues and alleviate buffer congestion, in addition to the ECN threshold, there is also a PFC threshold. When the queue buffer of the switch (i.e., the target device) reaches the PFC threshold, it notifies the upstream device to stop sending traffic. Triggering the ECN flag only notifies the server (i.e., other devices) to slow down. Therefore, when transmission congestion occurs, it is desirable to first trigger the ECN threshold to notify the slowdown and alleviate congestion, minimizing the triggering of PFC to prevent transmission interruption. Thus, to maximize throughput while maintaining losslessness, it is desirable to have a large enough packet buffer, but without reaching the PFC threshold and being stopped from sending traffic. Therefore, a reward and penalty mechanism that balances these considerations is designed.

[0077] Figure 6 This is a schematic diagram of a reward and punishment mechanism for ECN threshold adjustment actions according to an embodiment of this application. Figure 1 .like Figure 6 As shown, the reward value r is the PFC threshold minus the reciprocal of the number of packets cached in the target buffer queue after the current ECN threshold adjustment, multiplied by N (i.e., the capacity of the target buffer queue). It can be observed that when the adjustment threshold is a+3step, the reciprocal of the PFC threshold minus the number of packets cached is larger than the reciprocals calculated for the adjustment thresholds of a+step and a+2step. Therefore, the adjustment threshold of a+3step yields the highest reward. Thus, in state s, there is a greater tendency to execute action +3step because it can obtain more rewards.

[0078] Figure 7 This is a schematic diagram of a reward and punishment mechanism for ECN threshold adjustment actions according to an embodiment of this application. Figure 2 .like Figure 7 As shown, in this reward and punishment mechanism, in addition to positive rewards, there are also negative rewards. When the ECN threshold value is adjusted to a+3step based on state s, causing the number of packets cached in the target cache queue to directly exceed the PFC threshold, the PFC threshold minus the number of cached packets becomes a negative value, indicating that it is not expected to execute action +3step in state s, that is, the probability of executing +3step is extremely small.

[0079] Step S4, the learning process of ECN threshold adjustment:

[0080] After the three key elements of the SARSA online learning algorithm—state s, action x, and reward r—have been set, the Q-function value can be learned based on the existing network environment. The learning process is as follows:

[0081] (1) The initial state of the target buffer queue packet buffer is s (corresponding to the reference control period). Assume that there are three selectable executable actions x (i.e., threshold adjustment operations), namely ECN threshold + step, ECN threshold + 2step, and ECN threshold + 3step. These three actions correspond to three Q action value functions Q. t (s,step),Q t (s,2step),Q t (s,3step); Based on a greedy strategy, the action with the largest Q-value function is selected with probability β, and another action is randomly selected with probability (1-β). β can be defined as 90%. At this point, the action-value function Q... t (s,x) (i.e., initial adjustment parameters), and simultaneously obtain the reward value r (i.e., gain parameter).

[0082] (2) Based on the selected action x (i.e., the initial adjustment operation), after execution, observe the number of packets in the buffer (i.e., queue information) to obtain the next state s' (corresponding to the target control cycle). In state s', also based on the greedy strategy, select an action x' (i.e., the reference adjustment operation), and then obtain Q. t (s',x') (i.e., reference adjustment parameters).

[0083] (3) Based on the SARSA learning algorithm: Q (t+1) (s,x)=Q t (s,x)+α[r+γQ t (s',x')-Q t [s,x], the new action value function Q can be calculated. (t+1) (s,x).

[0084] (4) Based on the existing network traffic data, the training and iteration are repeated to eventually obtain the action value function Q(s,x) for all actions in all states. Based on the Q action value function, the optimal ECN waterline adjustment method can be deduced. Figure 8 This is a schematic diagram of an action value function table according to an embodiment of this application. For example... Figure 8 As shown, after repeated training and iteration, the Q-action value function table (i.e., prediction model) is obtained. The optimal action (i.e. target adjustment operation) to be executed in different states s can be selected according to the principle of maximizing Q value to adjust the queue threshold, thereby achieving the purpose of dynamically adjusting the ECN threshold. Furthermore, the value of the dynamically adjusted ECN threshold is the optimal ECN threshold value in that state, which can achieve the maximum bandwidth throughput.

[0085] Through the above steps, an online learning algorithm based on SARSA was implemented. Only the number of cached packets in the target cache queue of the chip is needed. Without much other information, the algorithm can be trained to dynamically adjust the ECN threshold. Moreover, under any network traffic model, it can achieve the maximum network throughput while avoiding packet loss or interruption.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software and necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0087] Figure 9 This is a structural block diagram of a cache queue threshold configuration device according to an embodiment of this application; as shown... Figure 9 As shown, it includes:

[0088] The acquisition module 902 is used to acquire, when the target control period for the target buffer queue of the target device is reached, the queue information of the target buffer queue after adjusting the queue threshold of the target buffer queue according to the initial adjustment operation during the reference control period before the current time. The target buffer queue is used to buffer data transmitted from other devices to the target device. The queue information is used to indicate the transmission efficiency of data transmission from other devices to the target device through the target buffer queue. The initial adjustment operation is selected from multiple threshold adjustment operations using an initial prediction model. The initial prediction model is used to predict the adjustment effect of executing the threshold adjustment operation on the adjustment target. The adjustment target includes improving the data transmission efficiency from other devices to the target device.

[0089] The update module 904 is used to update the initial prediction model based on the queue information to obtain the target prediction model;

[0090] The input module 906 is used to input the current target queue information of the target cache queue into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model. The operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target.

[0091] The determination module 908 is used to determine the target adjustment operation from multiple threshold adjustment operations based on the operation information;

[0092] The adjustment module 910 is used to adjust the current target threshold value of the target buffer queue according to the target adjustment operation.

[0093] Through the above embodiments, when the target control cycle for the target buffer queue of the target device is reached, the queue information of the target buffer queue is obtained after adjusting the queue threshold of the target buffer queue according to the initial adjustment operation during the reference control cycle prior to the current time; the initial prediction model is updated according to the queue information to obtain the target prediction model; the current target queue information of the target buffer queue is input into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model; the target adjustment operation is determined from multiple threshold adjustment operations according to the operation information; the current target threshold value of the target buffer queue is adjusted according to the target adjustment operation, that is, when the target control cycle of the target buffer queue is reached. The method involves collecting data from other devices to the target device within a reference control cycle, adjusting the queue threshold according to the initial adjustment operation obtained using the initial prediction model, and further obtaining the improvement effect of the initial adjustment operation on the data transmission efficiency from other devices to the target device. By updating the initial prediction model with this improvement effect, a more accurate target prediction model can be obtained, predicting the adjustment effect of the threshold adjustment operation on improving data transmission efficiency. Then, using the target prediction model and current queue information, the target adjustment operation is determined, achieving a better adjustment effect on improving data transmission efficiency, ultimately resulting in a more suitable queue threshold. This technical solution solves the problem of low queue threshold configuration efficiency in related technologies, achieving the technical effect of improving queue threshold configuration efficiency.

[0094] In one exemplary embodiment, the update module includes: a first filtering unit, configured to filter reference adjustment operations from multiple threshold adjustment operations using an initial prediction model based on queue information; a calculation unit, configured to calculate modified adjustment parameters of the initial adjustment operation based on reference adjustment parameters of the reference adjustment operation, initial adjustment parameters of the initial adjustment operation, and gain parameters of the initial adjustment operation, wherein the reference adjustment parameters are used to indicate the adjustment effect of the reference adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period, the initial adjustment parameters are used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the reference control period, the gain parameters are used to indicate the adjustment effect achieved on the adjustment target after executing the initial adjustment operation, and the modified adjustment parameters are used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period; and an update unit, configured to update the initial prediction model by modifying the adjustment parameters.

[0095] In an exemplary embodiment, the computing unit is further configured to: convert the cache parameters of the target cache queue within a reference control period into a first queue state and convert the cache parameters of the target cache queue within a target control period into a second queue state, wherein the queue state of the target cache queue corresponds to the capacity range of the occupied cache capacity of the target cache queue, the cache parameters are used to indicate the cache space of the target cache queue occupied by the data stored in the target cache queue, and the queue information includes the cache parameters; predict the initial adjustment parameters of the initial adjustment operation in the first queue state and the reference adjustment parameters of the reference adjustment operation in the second queue state using an initial prediction model; and calculate the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters, the initial adjustment parameters, and the gain parameters of the initial adjustment operation.

[0096] In an exemplary embodiment, the calculation unit is further configured to: detect the relationship between the cache parameter and the cache threshold of the target cache queue; if the cache parameter is detected to be greater than or equal to the cache threshold, obtain the difference between the cache threshold and the cache parameter to obtain a first gain parameter; calculate a change adjustment parameter for the initial adjustment operation based on a reference adjustment parameter, an initial adjustment parameter, and the first gain parameter; if the cache parameter is detected to be less than the cache threshold, obtain the reciprocal of the difference between the cache threshold and the cache parameter, calculate the product of the reciprocal and the capacity of the target cache queue to obtain a second gain parameter, wherein the capacity is used to indicate the data cache space of the target cache queue; and calculate the change adjustment parameter for the initial adjustment operation based on the reference adjustment parameter, the initial adjustment parameter, and the second gain parameter.

[0097] In one exemplary embodiment, the calculation unit is further configured to calculate the change adjustment parameter using the following formula:

[0098] Q (t+1) =Q t +α(r+γQ t '-Q t );

[0099] Among them, Q (t+1) It involves changing the adjustment parameter, Q. t It is the initial adjustment parameter, Q t ' is the reference adjustment parameter, r is the gain parameter, α is the learning rate, and γ is the discount factor.

[0100] In an exemplary embodiment, the first filtering unit is further configured to: generate a target random number; compare the target random number with a current random number threshold; if the target random number is greater than or equal to the current random number threshold, select the threshold adjustment operation with the best adjustment effect predicted by the initial prediction model from multiple threshold adjustment operations as a reference adjustment operation; if the target random number is less than the current random number threshold, randomly select a threshold adjustment operation from multiple threshold adjustment operations as a reference adjustment operation.

[0101] In one exemplary embodiment, the determining module includes: a second filtering unit, configured to filter from multiple threshold adjustment operations the threshold adjustment operation with the best adjustment effect predicted by the target prediction model as the target adjustment operation.

[0102] Embodiments of this application also provide a storage medium including a stored program, wherein the program executes any of the above-described cache queue threshold configuration methods when it runs.

[0103] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps:

[0104] S1, when the target control period for the target buffer queue of the target device is reached, obtain the queue information of the target buffer queue after adjusting the queue threshold of the target buffer queue according to the initial adjustment operation according to the reference control period before the current time. The target buffer queue is used to buffer data transmitted from other devices to the target device. The queue information is used to indicate the transmission efficiency of transmitting data from other devices to the target device through the target buffer queue. The initial adjustment operation is selected from multiple threshold adjustment operations using an initial prediction model. The initial prediction model is used to predict the adjustment effect of executing the threshold adjustment operation on the adjustment target. The adjustment target includes improving the data transmission efficiency from other devices to the target device.

[0105] S2, Update the initial prediction model based on the queue information to obtain the target prediction model;

[0106] S3, input the current target queue information of the target cache queue into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model. The operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target.

[0107] S4, determine the target adjustment operation from multiple threshold adjustment operations based on the operation information;

[0108] S5, adjust the current target threshold value of the target cache queue according to the target adjustment operation.

[0109] Embodiments of this application also provide an electronic device including a memory and a processor, the memory storing a computer program, the processor being configured to run the computer program to perform the steps in any of the above embodiments of the cache queue threshold configuration method.

[0110] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0111] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0112] S1, when the target control period for the target buffer queue of the target device is reached, obtain the queue information of the target buffer queue after adjusting the queue threshold of the target buffer queue according to the initial adjustment operation according to the reference control period before the current time. The target buffer queue is used to buffer data transmitted from other devices to the target device. The queue information is used to indicate the transmission efficiency of transmitting data from other devices to the target device through the target buffer queue. The initial adjustment operation is selected from multiple threshold adjustment operations using an initial prediction model. The initial prediction model is used to predict the adjustment effect of executing the threshold adjustment operation on the adjustment target. The adjustment target includes improving the data transmission efficiency from other devices to the target device.

[0113] S2, Update the initial prediction model based on the queue information to obtain the target prediction model;

[0114] S3, input the current target queue information of the target cache queue into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model. The operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target.

[0115] S4, determine the target adjustment operation from multiple threshold adjustment operations based on the operation information;

[0116] S5, adjust the current target threshold value of the target cache queue according to the target adjustment operation.

[0117] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0118] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0119] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0120] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for configuring a cache queue threshold, characterized in that, include: When the target control period for the target buffer queue of the target device is reached, the queue information of the target buffer queue is obtained after adjusting the queue threshold of the target buffer queue according to the initial adjustment operation according to the reference control period before the current time. The target buffer queue is used to buffer data transmitted from other devices to the target device. The queue information is used to indicate the transmission efficiency of transmitting data from other devices to the target device through the target buffer queue. The initial adjustment operation is selected from multiple threshold adjustment operations using an initial prediction model. The initial prediction model is used to predict the adjustment effect of executing the threshold adjustment operation on the adjustment target. The adjustment target includes improving the data transmission efficiency from other devices to the target device. The initial prediction model is updated based on the queue information to obtain the target prediction model; The current target queue information of the target cache queue is input into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model, wherein the operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target; The target adjustment operation is determined from the plurality of threshold adjustment operations based on the operation information; Adjust the current target threshold value of the target cache queue according to the target adjustment operation; The step of updating the initial prediction model based on the queue information includes: using the initial prediction model to select reference adjustment operations from the plurality of threshold adjustment operations based on the queue information; calculating modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters of the reference adjustment operation, the initial adjustment parameters of the initial adjustment operation, and the gain parameters of the initial adjustment operation, wherein the reference adjustment parameters are used to indicate the adjustment effect of the reference adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period, the initial adjustment parameters are used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the reference control period, the gain parameters are used to indicate the adjustment effect achieved on the adjustment target after executing the initial adjustment operation, and the modified adjustment parameters are used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period; and updating the initial prediction model using the modified adjustment parameters. The step of calculating the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters of the reference adjustment operation, the initial adjustment parameters of the initial adjustment operation, and the gain parameters of the initial adjustment operation includes: converting the cache parameters of the target cache queue within the reference control period to a first queue state and converting the cache parameters of the target cache queue within the target control period to a second queue state, wherein the queue state of the target cache queue corresponds to the capacity range of the occupied cache capacity of the target cache queue, the cache parameters are used to indicate the cache space of the target cache queue occupied by the target cache queue storing the data, and the queue information includes the cache parameters; using the initial prediction model to predict the initial adjustment parameters of the initial adjustment operation in the first queue state, and the reference adjustment parameters of the reference adjustment operation in the second queue state; and calculating the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters, the initial adjustment parameters, and the gain parameters of the initial adjustment operation. The step of using the initial prediction model to select a reference adjustment operation from the plurality of threshold adjustment operations based on the queue information includes: generating a target random number; comparing the target random number with a current random number threshold; if the target random number is greater than or equal to the current random number threshold, selecting the threshold adjustment operation with the best adjustment effect predicted by the initial prediction model from the plurality of threshold adjustment operations as the reference adjustment operation; if the target random number is less than the current random number threshold, randomly selecting one of the threshold adjustment operations from the plurality of threshold adjustment operations as the reference adjustment operation.

2. The method according to claim 1, characterized in that, The step of calculating the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters, the initial adjustment parameters, and the gain parameters of the initial adjustment operation includes: Detect the relationship between the cache parameters and the cache threshold of the target cache queue; If the cache parameter is detected to be greater than or equal to the cache threshold, the difference between the cache threshold and the cache parameter is obtained to obtain a first gain parameter; the change adjustment parameter of the initial adjustment operation is calculated based on the reference adjustment parameter, the initial adjustment parameter and the first gain parameter. If the cache parameter is detected to be less than the cache threshold, the reciprocal of the difference between the cache threshold and the cache parameter is obtained, and the product of the reciprocal and the capacity of the target cache queue is calculated to obtain a second gain parameter, wherein the capacity is used to indicate the data cache space of the target cache queue; the change adjustment parameter of the initial adjustment operation is calculated based on the reference adjustment parameter, the initial adjustment parameter and the second gain parameter.

3. The method according to claim 1, characterized in that, The step of calculating the modified adjustment parameters of the initial adjustment operation based on the reference adjustment parameters of the reference adjustment operation, the initial adjustment parameters of the initial adjustment operation, and the gain parameters of the initial adjustment operation includes: The changed adjustment parameter is calculated using the following formula: Q （t+1） =Q t +α(r+γQ t '-Q t ); Among them, Q （t+1） The changed adjustment parameter, Q t It is the initial adjustment parameter, Q t ' is the reference adjustment parameter, r is the gain parameter, α is the learning rate, and γ is the discount factor.

4. The method according to claim 1, characterized in that, The step of determining the target adjustment operation from the plurality of threshold adjustment operations based on the operation information includes: The threshold adjustment operation with the best adjustment effect predicted by the target prediction model is selected from the multiple threshold adjustment operations and used as the target adjustment operation.

5. A configuration device for a cache queue threshold, characterized in that, include: The acquisition module is used to acquire, when the target control period for the target cache queue of the target device is reached, the queue information of the target cache queue after adjusting the queue threshold of the target cache queue according to the initial adjustment operation during the reference control period before the current time. The target cache queue is used to cache data transmitted from other devices to the target device. The queue information is used to indicate the transmission efficiency of transmitting data from other devices to the target device through the target cache queue. The initial adjustment operation is selected from multiple threshold adjustment operations using an initial prediction model. The initial prediction model is used to predict the adjustment effect of executing the threshold adjustment operation on the adjustment target. The adjustment target includes improving the data transmission efficiency from other devices to the target device. The update module is used to update the initial prediction model according to the queue information to obtain the target prediction model; The input module is used to input the current target queue information of the target cache queue into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model, wherein the operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target; A determining module is used to determine a target adjustment operation from a plurality of threshold adjustment operations based on the operation information; The adjustment module is used to adjust the current target threshold value of the target cache queue according to the target adjustment operation; The update module includes: a first filtering unit, configured to filter reference adjustment operations from the plurality of threshold adjustment operations using the initial prediction model based on the queue information; a calculation unit, configured to calculate a modified adjustment parameter of the initial adjustment operation based on the reference adjustment parameter of the reference adjustment operation, the initial adjustment parameter of the initial adjustment operation, and the gain parameter of the initial adjustment operation, wherein the reference adjustment parameter indicates the adjustment effect of the reference adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period, the initial adjustment parameter indicates the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the reference control period, the gain parameter indicates the adjustment effect achieved on the adjustment target after executing the initial adjustment operation, and the modified adjustment parameter indicates the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period; and an update unit, configured to update the initial prediction model using the modified adjustment parameter. The calculation unit is further configured to: convert the cache parameters of the target cache queue within the reference control period into a first queue state and convert the cache parameters of the target cache queue within the target control period into a second queue state, wherein the queue state of the target cache queue corresponds to the capacity range of the occupied cache capacity of the target cache queue, the cache parameters are used to indicate the cache space of the target cache queue occupied by the target cache queue storing the data, and the queue information includes the cache parameters; use the initial prediction model to predict the initial adjustment parameters of the initial adjustment operation in the first queue state, and the reference adjustment parameters of the reference adjustment operation in the second queue state; calculate the change adjustment parameters of the initial adjustment operation based on the reference adjustment parameters, the initial adjustment parameters, and the gain parameters of the initial adjustment operation; The first filtering unit is further configured to: generate a target random number; compare the target random number with a current random number threshold; if the target random number is greater than or equal to the current random number threshold, select the threshold adjustment operation with the best adjustment effect predicted by the initial prediction model from the plurality of threshold adjustment operations as the reference adjustment operation; if the target random number is less than the current random number threshold, randomly select one of the threshold adjustment operations from the plurality of threshold adjustment operations as the reference adjustment operation.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 4.

7. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 4 through the computer program.

Images