Cache queue threshold configuration method and device, storage medium and electronic device

Abstract

The invention discloses a cache queue threshold configuration method and device, a storage medium and an electronic device. The method comprises the following steps: when a target control period of a target cache queue of target equipment is reached at present, configuring a cache queue threshold of the target equipment; obtaining queue information of the target cache queue after the reference control period before the current moment adjusts the queue threshold of the target cache queue according to the initial adjustment operation; updating the initial prediction model according to the queue information to obtain a target prediction model; inputting current target queue information of the target cache queue into the target prediction model to obtain operation information of each threshold adjustment operation output by the target prediction model; determining a target adjustment operation from a plurality of threshold adjustment operations according to the operation information; and adjusting the current target threshold value of the target cache queue according to the target adjustment operation. By adopting the technical scheme, the problems of low configuration efficiency of the queue threshold and the like in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular, to a method and device for configuring a cache queue threshold, a storage medium, and an electronic device. Background Art

[0002] In order to avoid packet loss and other situations when transmitting message data, the switching device will set up a corresponding cache queue to cache the services that are received but cannot be processed in time. However, when there is a gap between the receiving speed and processing speed of the service, the cache queue may be full but the service to be processed is still being received, which will further lead to packet loss. In order to avoid the above problems, a congestion control mechanism is set, that is, the queue threshold and cache threshold of the cache queue are set, so that when the amount of service cached in the cache queue exceeds the queue threshold, the receiving speed of the service to be processed is reduced, and when the amount of service cached in the cache queue exceeds the cache threshold, the service to be processed is stopped, thereby improving the transmission efficiency of the message data. However, the current method for determining the queue threshold depends on known scenarios. Only when multiple parameters of the switching device fall within the known scenario can a reasonable queue threshold be determined. In an unknown scenario, it is difficult to determine a suitable queue threshold with the help of simple parameters.

[0003] With regard to the problems such as low efficiency in configuring queue thresholds in related technologies, no effective solutions have been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a method and device for configuring a cache queue threshold, a storage medium, and an electronic device, so as to at least solve the problem of low efficiency in configuring the queue threshold in the related art.

[0005] According to an embodiment of the present application, a method for configuring a cache queue threshold is provided, comprising: when a target control cycle of a target cache queue for a target device is currently reached, obtaining a reference control cycle before a current moment, adjusting the queue threshold of the target cache queue according to an initial adjustment operation, and then obtaining queue information of the target cache queue, wherein the target cache queue is used to cache data transmitted from other devices to the target device, the queue information is used to indicate the transmission efficiency of transmitting data from the other devices to the target device through the target cache queue, the initial adjustment operation is selected from multiple threshold adjustment operations using an initial prediction model, and the initial prediction model is used to predict Execute the threshold adjustment operation to obtain the adjustment effect on the adjustment target, wherein the adjustment target includes improving the data transmission efficiency from the other devices to the target device; update the initial prediction model according to the queue information to obtain the target prediction model; input the current target queue information of the target cache queue into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model, wherein the operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target; determine the target adjustment operation from the multiple threshold adjustment operations according to the operation information; and adjust the current target threshold value of the target cache queue according to the target adjustment operation.

[0006] Optionally, updating the initial prediction model according to the queue information includes: using the initial prediction model to filter a reference adjustment operation from the multiple threshold adjustment operations according to the queue information; calculating a change adjustment parameter of the initial adjustment operation according to a reference adjustment parameter of the reference adjustment operation, an initial adjustment parameter of the initial adjustment operation, and a gain parameter of the initial adjustment operation, wherein the reference adjustment parameter is used to indicate an adjustment effect of the reference adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period, the initial adjustment parameter is used to indicate an adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the reference control period, the gain parameter is used to indicate an adjustment effect achieved on the adjustment target after executing the initial adjustment operation, and the change adjustment parameter is used to indicate an adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period; and updating the initial prediction model by using the change adjustment parameter.

[0007] Optionally, the calculation of the changed adjustment parameters of the initial adjustment operation based on the reference adjustment parameters of the reference adjustment operation, the initial adjustment parameters of the initial adjustment operation, and the gain parameters of the initial adjustment operation includes: converting the cache parameters of the target cache queue within the reference control period to a first queue state and converting the cache parameters of the target cache queue within the target control period to a second queue state, wherein the queue state of the target cache queue has a corresponding relationship with the capacity interval of the occupied cache capacity of the target cache queue, the cache parameters are used to indicate the cache space of the target cache queue occupied by the target cache queue to store the data, and the queue information includes the cache parameters; using the initial prediction model to predict the initial adjustment parameters of the initial adjustment operation in the first queue state, and the reference adjustment parameters of the reference adjustment operation in the second queue state; calculating the changed adjustment parameters of the initial adjustment operation based on the reference adjustment parameters, the initial adjustment parameters, and the gain parameters of the initial adjustment operation.

[0008] Optionally, the calculation of the change adjustment parameter of the initial adjustment operation based on the reference adjustment parameter, the initial adjustment parameter and the gain parameter of the initial adjustment operation includes: detecting the size relationship between the cache parameter and the cache threshold of the target cache queue; when it is detected that the cache parameter is greater than or equal to the cache threshold, obtaining the difference between the cache threshold and the cache parameter to obtain a first gain parameter; calculating the change adjustment parameter of the initial adjustment operation based on the reference adjustment parameter, the initial adjustment parameter and the first gain parameter; when it is detected that the cache parameter is less than the cache threshold, obtaining the inverse of the difference between the cache threshold and the cache parameter, calculating the product of the inverse and the capacity of the target cache queue to obtain a second gain parameter, wherein the capacity is used to indicate the data cache space of the target cache queue; calculating the change adjustment parameter of the initial adjustment operation based on the reference adjustment parameter, the initial adjustment parameter and the second gain parameter.

[0009] Optionally, calculating the changed adjustment parameter of the initial adjustment operation according to the reference adjustment parameter of the reference adjustment operation, the initial adjustment parameter of the initial adjustment operation, and the gain parameter of the initial adjustment operation includes: calculating the changed adjustment parameter by the following formula:

[0010] Q (t+1) =Q t +α(r+γQ t '-Qt);

[0011] Among them, Q (t+1) is the change adjustment parameter, Q tis the initial tuning parameter, Q t ' is the reference adjustment parameter, r is the gain parameter, α is the learning rate, and γ is the discount factor.

[0012] Optionally, the use of the initial prediction model according to the queue information to select a reference adjustment operation from the multiple threshold adjustment operations includes: generating a target random number; comparing the target random number with a current random number threshold; when the target random number is greater than or equal to the current random number threshold, selecting the threshold adjustment operation with the best adjustment effect predicted by the initial prediction model from the multiple threshold adjustment operations as the reference adjustment operation; when the target random number is less than the current random number threshold, randomly selecting one of the threshold adjustment operations from the multiple threshold adjustment operations as the reference adjustment operation.

[0013] Optionally, determining a target adjustment operation from the multiple threshold adjustment operations according to the operation information includes: selecting the threshold adjustment operation with the best adjustment effect predicted by the target prediction model from the multiple threshold adjustment operations as the target adjustment operation.

[0014] According to another embodiment of the embodiment of the present application, a cache queue threshold configuration device is also provided, including: an acquisition module, which is used to obtain the queue information of the target cache queue after adjusting the queue threshold of the target cache queue according to the initial adjustment operation in the reference control cycle before the current moment when the target control cycle of the target cache queue for the target device is currently reached, wherein the target cache queue is used to cache data transmitted from other devices to the target device, and the queue information is used to indicate the transmission efficiency of transmitting data from the other devices to the target device through the target cache queue, and the initial adjustment operation is screened from multiple threshold adjustment operations using an initial prediction model, and the initial prediction model is used to predict the execution of the threshold adjustment operation. The invention relates to a method for adjusting the adjustment effect achieved on the adjustment target, wherein the adjustment target includes improving the data transmission efficiency from the other devices to the target device; an update module, which is used to update the initial prediction model according to the queue information to obtain a target prediction model; an input module, which is used to input the current target queue information of the target cache queue into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model, wherein the operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target; a determination module, which is used to determine a target adjustment operation from a plurality of threshold adjustment operations according to the operation information; and an adjustment module, which is used to adjust the current target threshold value of the target cache queue according to the target adjustment operation.

[0015] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned cache queue threshold configuration method when running.

[0016] According to another aspect of an embodiment of the present application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the cache queue threshold configuration method through the computer program.

[0017] In an embodiment of the present application, when a target control cycle of a target cache queue of a target device is currently reached, after the queue threshold of the target cache queue is adjusted according to the initial adjustment operation in the reference control cycle before the current moment, the queue information of the target cache queue is obtained; the initial prediction model is updated according to the queue information to obtain the target prediction model; the current target queue information of the target cache queue is input into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model; the target adjustment operation is determined from multiple threshold adjustment operations according to the operation information; the current target threshold value of the target cache queue is adjusted according to the target adjustment operation, that is, when the target control cycle of the target cache queue is reached, Under the circumstances, the transmission efficiency of the target device transmitting data from other devices to the target device after adjusting the queue threshold according to the initial adjustment operation obtained by using the initial prediction model during the reference control period is collected, and the improvement effect of the initial adjustment operation on improving the data transmission efficiency of data transmitted from other devices to the target device can be further obtained. The initial prediction model is updated with the obtained improvement effect, and a target prediction model that predicts the adjustment effect that can be achieved by the threshold adjustment operation on improving the data transmission efficiency can be obtained more accurately. Then, the target prediction model and the current queue information are used to determine the target adjustment operation, which can achieve a better adjustment effect of improving the data transmission efficiency, that is, a more suitable queue threshold can be finally obtained. The above technical solution solves the problem of low configuration efficiency of the queue threshold in the related technology, and achieves the technical effect of improving the configuration efficiency of the queue threshold. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 It is a hardware environment schematic diagram of a method for configuring a cache queue threshold according to an embodiment of the present application;

[0021] Figure 2 is a flow chart of a method for configuring a cache queue threshold according to an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of a congestion control mechanism according to an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of a queue threshold state decomposition method according to an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of an ECN threshold adjustment operation setting method according to an embodiment of the present application;

[0025] Figure 6 Schematic diagram of a reward and punishment mechanism for an ECN threshold adjustment action according to an embodiment of the present application Figure 1 ;

[0026] Figure 7 Schematic diagram of a reward and punishment mechanism for an ECN threshold adjustment action according to an embodiment of the present application Figure 2 ;

[0027] Figure 8 is a schematic diagram of an action value function table according to an embodiment of the present application;

[0028] Fig. 9 It is a structural block diagram of a cache queue threshold configuration device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal, a device terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 FIG. 1 is a schematic diagram of a hardware environment for configuring a cache queue threshold according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 (processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. In an exemplary embodiment, the computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0032] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for sending message push in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0033] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] In this embodiment, a method for configuring a cache queue threshold is provided, which is applied to the above-mentioned computer terminal. Figure 2 : is a flowchart of a method for configuring a cache queue threshold according to an embodiment of the present application, the process comprising the following steps:

[0035] Step S202, when the target control cycle of the target cache queue of the target device is currently reached, the queue information of the target cache queue after adjusting the queue threshold of the target cache queue according to the initial adjustment operation in the reference control cycle before the current moment is obtained, wherein the target cache queue is used to cache data transmitted from other devices to the target device, the queue information is used to indicate the transmission efficiency of transmitting data from other devices to the target device through the target cache queue, the initial adjustment operation is selected from multiple threshold adjustment operations using an initial prediction model, and the initial prediction model is used to predict the adjustment effect achieved by executing the threshold adjustment operation on the adjustment target, and the adjustment target includes improving the data transmission efficiency of other devices to the target device;

[0036] Step S204, updating the initial prediction model according to the queue information to obtain a target prediction model;

[0037] Step S206, inputting the current target queue information of the target cache queue into the target prediction model, and obtaining the operation information of each threshold adjustment operation output by the target prediction model, wherein the operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target;

[0038] Step S208, determining a target adjustment operation from a plurality of threshold adjustment operations according to the operation information;

[0039] Step S210: adjusting the current target threshold value of the target cache queue according to the target adjustment operation.

[0040] Through the above steps, when the target control cycle of the target cache queue of the target device is currently reached, the queue information of the target cache queue is obtained after the queue threshold of the target cache queue is adjusted according to the initial adjustment operation in the reference control cycle before the current moment; the initial prediction model is updated according to the queue information to obtain the target prediction model; the current target queue information of the target cache queue is input into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model; the target adjustment operation is determined from multiple threshold adjustment operations according to the operation information; the current target threshold value of the target cache queue is adjusted according to the target adjustment operation, that is, when the target control cycle of the target cache queue is reached Under the above, the transmission efficiency of the target device transmitting data from other devices to the target device after adjusting the queue threshold according to the initial adjustment operation obtained by using the initial prediction model within the reference control period can be collected. Further, the improvement effect of the initial adjustment operation on improving the data transmission efficiency of data transmitted from other devices to the target device can be obtained. The initial prediction model is updated with the help of the obtained improvement effect, and a target prediction model that predicts the adjustment effect that can be achieved by the threshold adjustment operation on improving the data transmission efficiency can be obtained more accurately. Then, the target prediction model and the current queue information are used to determine the target adjustment operation, which can achieve a better adjustment effect on improving the data transmission efficiency, that is, a more suitable queue threshold can be finally obtained. The above technical solution solves the problem of low configuration efficiency of the queue threshold in the related technology, and achieves the technical effect of improving the configuration efficiency of the queue threshold.

[0041] Optionally, in the embodiment of the present application, in order to alleviate the congestion of the switching device transmitting messages, a congestion control mechanism is set. Figure 3 FIG. 1 is a schematic diagram of a congestion control mechanism according to an embodiment of the present application. Figure 3As shown, the target device (i.e., the network switch) receives the message sent by other devices (i.e., the sending end server) and sends the received message to the receiving end server. If there is congestion in the outbound direction of the port, the message needs to be marked with ECN to notify the server that congestion has occurred on the current link and the message sending rate needs to be appropriately reduced. The target device determines whether congestion has occurred by determining whether the queue buffer in the target buffer queue exceeds the ECN threshold (i.e., the queue threshold). If the number of queue message buffers exceeds the ECN threshold, it is considered that the queue is congested at this time, and the message sent from the queue will be marked with ECN to notify the server. In order to achieve flow control of the lossless queue and alleviate the cache congestion of the lossless queue, in addition to the ECN threshold, there is also a PFC threshold (i.e., cache threshold). When the queue cache of the network switch reaches the PFC threshold, it will notify the upstream device to stop sending traffic. When the ECN mark is triggered, it only notifies the server to slow down. Therefore, when congestion occurs, it is expected to first trigger the ECN threshold to notify the speed reduction to alleviate the congestion, and try to avoid triggering PFC to prevent the transmission interruption. However, it takes some time from when the network switch discovers that congestion has occurred in the queue cache and triggers the congestion mark, to when the sending server receives the CNP congestion notification message and perceives that there is congestion on the network link and reduces the packet sending rate. During this period of time, the sending server will still send traffic to the switch at the original packet sending rate, which will cause the device queue cache congestion to continue to change, and will eventually trigger PFC flow control and suspend the sending of traffic. Therefore, it is necessary to set the ECN threshold reasonably so that the cache space between the ECN threshold and the PFC threshold can accommodate the traffic sent during the period from when the congestion mark is triggered to when the sending server slows down. However, this ECN threshold value is difficult to determine and may also change dynamically. Therefore, it is necessary to provide a way to determine a reasonable ECN threshold value according to changes in the network environment. According to the method provided in this application, the ECN threshold can be reasonably adjusted according to changes in the network environment through continuous learning of the prediction model.

[0042] In the technical solution provided in the above step S202, the target device may be, but is not limited to, a device that receives data sent by a front-end device and needs to process the received data, for example, a switch for receiving data sent by a front-end device and transmitting the data to a back-end device.

[0043] Optionally, in this embodiment, the target cache queue may be, but is not limited to, used to cache received data to be processed for the target device. The target device may be, but is not limited to, configured with multiple target cache queues to cache data from different services or requiring different processing.

[0044] Optionally, in the embodiment of the present application, the threshold adjustment frequency can be set, but not limited to, according to the reception and processing of data by the target device, and the threshold adjustment timing of the queue threshold of the target cache queue is set with a fixed threshold adjustment frequency. The target control period is reached, that is, the threshold adjustment timing is reached. The target control period is the time period between the moment of reaching the threshold adjustment timing this time and the moment of reaching the threshold adjustment timing next time. The reference control period is the time period between the moment of reaching the threshold adjustment timing last time and the moment of reaching the threshold adjustment timing this time. The above time period needs to exceed a certain length of time to fully demonstrate the impact of the threshold adjustment operation taken on the transmission efficiency of the target cache queue receiving data sent by other devices.

[0045] Optionally, in an embodiment of the present application, the queue information may be, but is not limited to, the amount of data stored in the target cache queue, or the proportion of the cache space of the target cache queue occupied by the data stored in the target cache queue, etc., and the present application does not make specific limitations on this.

[0046] In the technical solution provided in the above step S204, the initial prediction model and the target prediction model may, but are not limited to, maintain the adjustment effects of different threshold adjustment operations on the adjustment target under different queue states.

[0047] In an exemplary embodiment, the initial prediction model is updated according to the queue information, including: using the initial prediction model to filter a reference adjustment operation from multiple threshold adjustment operations according to the queue information; calculating a change adjustment parameter of the initial adjustment operation according to a reference adjustment parameter of the reference adjustment operation, an initial adjustment parameter of the initial adjustment operation, and a gain parameter of the initial adjustment operation, wherein the reference adjustment parameter is used to indicate the adjustment effect of the reference adjustment operation predicted by the initial prediction model on the data transmission efficiency within a target control period, the initial adjustment parameter is used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the reference control period, the gain parameter is used to indicate the adjustment effect achieved on the adjustment target after executing the initial adjustment operation, and the change adjustment parameter is used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period; the initial prediction model is updated by changing the adjustment parameter.

[0048] Optionally, in this embodiment, queue information can be input into the initial prediction model but is not limited to obtaining operation information of each threshold adjustment operation output by the initial prediction model, wherein the operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target; and the threshold adjustment operation with the best adjustment effect indicated by the operation information is searched from multiple threshold adjustment operations as a reference adjustment operation.

[0049] Optionally, in an embodiment of the present application, the reference adjustment parameter may be, but is not limited to, the operation information corresponding to the reference adjustment operation output by the initial prediction model when the queue information of the target control cycle is input into the initial prediction model, and the initial adjustment parameter may be, but is not limited to, the operation information corresponding to the initial adjustment operation output by the initial prediction model when the queue information of the reference control cycle is input into the initial prediction model.

[0050] Optionally, in an embodiment of the present application, updating the initial prediction model according to the queue information includes but is not limited to calculating a gain parameter of the initial adjustment operation according to the queue information, and using the gain parameter to update the initial prediction model.

[0051] In an exemplary embodiment, the changed adjustment parameters of the initial adjustment operation are calculated based on the reference adjustment parameters of the reference adjustment operation, the initial adjustment parameters of the initial adjustment operation, and the gain parameters of the initial adjustment operation, including: converting the cache parameters of the target cache queue within the reference control period to a first queue state and converting the cache parameters of the target cache queue within the target control period to a second queue state, wherein the queue state of the target cache queue corresponds to the capacity interval of the occupied cache capacity of the target cache queue, the cache parameters are used to indicate the cache space of the target cache queue occupied by the target cache queue to store data, and the queue information includes the cache parameters; using an initial prediction model to predict the initial adjustment parameters of the initial adjustment operation in the first queue state, and the reference adjustment parameters of the reference adjustment operation in the second queue state; calculating the changed adjustment parameters of the initial adjustment operation based on the reference adjustment parameters, the initial adjustment parameters, and the gain parameters of the initial adjustment operation.

[0052] Optionally, in this embodiment, the target cache queue can be divided into multiple capacity intervals with the same cache capacity according to the cache capacity of the target cache queue, but is not limited to it. The cache parameters can be matched to different capacity intervals according to the cache space occupied by the data stored in the target cache queue indicated by the cache parameters, and then the cache parameters can be converted into corresponding queue states with the help of the correspondence between the capacity interval and the queue state.

[0053] In an exemplary embodiment, a change adjustment parameter of an initial adjustment operation is calculated based on a reference adjustment parameter, an initial adjustment parameter, and a gain parameter of the initial adjustment operation, including: detecting a size relationship between a cache parameter and a cache threshold of a target cache queue; when it is detected that the cache parameter is greater than or equal to the cache threshold, obtaining a difference between the cache threshold and the cache parameter to obtain a first gain parameter; calculating the change adjustment parameter of the initial adjustment operation based on the reference adjustment parameter, the initial adjustment parameter, and the first gain parameter; when it is detected that the cache parameter is less than the cache threshold, obtaining a reciprocal of the difference between the cache threshold and the cache parameter, calculating the product of the reciprocal and the capacity of the target cache queue to obtain a second gain parameter, wherein the capacity is used to indicate the data cache space of the target cache queue; calculating the change adjustment parameter of the initial adjustment operation based on the reference adjustment parameter, the initial adjustment parameter, and the second gain parameter.

[0054] Optionally, in this embodiment, the cache threshold may be, but is not limited to, the capacity of the target cache queue, or a portion of the capacity of the target cache queue. The target device is configured to stop accepting data transmitted to the target device by other devices when the amount of data cached in the target cache queue exceeds the cache threshold, that is, when the cache threshold is exceeded, a flow interruption or packet loss may occur. In order to ensure the efficiency of data reception by the target device, it is necessary to avoid exceeding the cache threshold.

[0055] In an exemplary embodiment, calculating the change adjustment parameter of the initial adjustment operation according to the reference adjustment parameter of the reference adjustment operation, the initial adjustment parameter of the initial adjustment operation, and the gain parameter of the initial adjustment operation includes: calculating the change adjustment parameter by the following formula:

[0056] Q (t+1) =Q t +α(r+γQ t '-Q t );

[0057] Among them, Q (t+1) is the change adjustment parameter, Q t is the initial adjustment parameter, Q t ' is the reference tuning parameter, r is the gain parameter, α is the learning rate, and γ is the discount factor.

[0058] Optionally, in this embodiment, the Q value represents an estimate of the cumulative reward expected to be obtained by performing a threshold adjustment operation under a given target cache queue cache condition. The reward is the adjustment effect achieved on the adjustment target. Specifically, the Q value represents the sum of the long-term rewards expected to be obtained after taking the selected threshold adjustment operation based on the current cache condition. This sum includes the immediate reward r and the discounted reward obtained in all future adjustment operations. α is a number between 0 and 1, which controls the degree of influence of new information on the old Q value. γ is a number between 0 and 1, which determines the importance of future rewards relative to current rewards. If γ is close to 1, then future rewards are almost as important as current rewards. If γ is close to 0, then only immediate rewards are important.

[0059] In an exemplary embodiment, an initial prediction model is used to screen a reference adjustment operation from multiple threshold adjustment operations based on queue information, including: generating a target random number; comparing the target random number with a current random number threshold; when the target random number is greater than or equal to the current random number threshold, screening out from the multiple threshold adjustment operations a threshold adjustment operation with the best adjustment effect predicted by the initial prediction model as a reference adjustment operation; when the target random number is less than the current random number threshold, randomly screening out a threshold adjustment operation from the multiple threshold adjustment operations as a reference adjustment operation.

[0060] Optionally, in this embodiment, including but not limited to, after each screening reference adjustment operation, a target random number threshold value less than the current random number threshold value is generated as the random number threshold value for comparison with the generated target random number in the next screening reference adjustment operation.

[0061] In the technical solution provided in the above step S206, after the current target queue information is input into the target prediction model, the target prediction model may, but is not limited to, output the predicted adjustment effect that can be achieved on the adjustment target by taking different threshold adjustment operations on the current queue threshold value under the data transmission efficiency indicated by the current target queue information as operation information of different threshold adjustment operations.

[0062] In the technical solution provided in the above step S208, determining the target adjustment operation from multiple threshold adjustment operations based on the operation information can be, but is not limited to, selecting the threshold adjustment operation with the best adjustment effect indicated by the operation information as the target adjustment operation with a greater probability, and selecting the threshold adjustment operation with the second best adjustment effect indicated by the operation information as the target adjustment operation with a smaller probability.

[0063] In an exemplary embodiment, determining a target adjustment operation from a plurality of threshold adjustment operations according to the operation information includes: selecting a threshold adjustment operation with the best adjustment effect predicted by a target prediction model from the plurality of threshold adjustment operations as the target adjustment operation.

[0064] In the technical solution provided in the above step S210, adjusting the current target threshold value of the target cache queue according to the target adjustment operation includes but is not limited to calculating the target threshold value after the unadjusted queue threshold is adjusted by the target adjustment operation, and determining the calculated target threshold value as the new queue threshold.

[0065] In order to better understand the above process, the above process is described below in conjunction with an optional embodiment, but it is not intended to limit the technical solution of the embodiment of the present application.

[0066] Optionally, a cache queue threshold adjustment method based on the SARSA algorithm is also provided in the embodiment of the present application. The SARSA algorithm is an online learning algorithm in reinforcement learning, which learns the optimal behavior strategy through the interaction between the agent and the environment. The algorithm is an online policy iteration algorithm, the core of which is to calculate the update formula of the Q action value function, and the Q function is learned through online interaction, and its update rules are as follows:

[0067] In each iteration, the agent experiences state s, performs action x, observes reward r and new state s', and takes action x', thereby obtaining the action value function Q. The update formula of the Q function is:

[0068] Q (t+1) (s,x)=Q t (s,x)+α[r+γQ t (s',x')-Q t (s,x)];

[0069] Where α is the learning rate, r is the reward for taking an action in the current state, γ is the discount factor, s' is the new state, and x' is the action in the new state.

[0070] The cache queue threshold adjustment method based on the SARSA algorithm includes the following steps:

[0071] Step S1, ECN waterline (queue threshold) state decomposition:

[0072] Figure 4 is a schematic diagram of a queue threshold state decomposition method according to an embodiment of the present application, such as Figure 4As shown, the ECN threshold is set based on the target cache queue Queue. The target cache queue is used to cache messages that need to be forwarded. When the number of cached messages exceeds the queue's ECN threshold, i.e., the queue threshold, it will trigger ECN marking of the message. The maximum number of cached messages (i.e., capacity) of the target cache queue is N. The target cache queue is divided into 10 parts, each of which can be set to a state s in the SARSA learning algorithm: when the number of messages is between 1 and N / 10, it is set to state s1; when the number of messages (i.e., queue information) is between N / 10+1 and 2N / 10, it is set to state s2; when the number of messages is between 2N / 10+1 and 3N / 10, it is set to state s3; when the number of messages is between 3N / 10+1 and 4N / 10, it is set to state s4. s4; when the number of messages is between 4N / 10+1 and 5N / 10, the state is set to s5; when the number of messages is between 5N / 10+1 and 6N / 10, the state is set to s6; when the number of messages is between 6N / 10+1 and 7N / 10, the state is set to s7; when the number of messages is between 7N / 10+1 and 8N / 10, the state is set to s8; when the number of messages is between 8N / 10+1 and 9N / 10, the state is set to s9; when the number of messages is between 9N / 10+1 and N, the state is set to s10.

[0073] Step S2, ECN threshold adjustment operation setting:

[0074] Figure 5 FIG. 1 is a schematic diagram of an ECN threshold adjustment operation setting method according to an embodiment of the present application. Figure 5 As shown in the figure, a is the initial position of the ECN threshold, a+step is the position after adding step to the ECN threshold, and step is the step size of the ECN waterline adjustment; a+2step is the position after adding 2 steps to the ECN threshold; a+3step is the position after adding 3 steps to the ECN threshold; a-step is the position after subtracting step from the ECN threshold, and step is the step size of the ECN waterline adjustment; a-2step is the position after subtracting 2 steps from the ECN threshold; a-3step is the position after subtracting 3 steps from the ECN threshold. Adding or subtracting 1 to 3 steps above is the selectable threshold adjustment operation.

[0075] Step S3, setting of reward and punishment mechanism after ECN threshold adjustment:

[0076] In order to achieve flow control of lossless queues and alleviate cache congestion of lossless queues, in addition to the ECN threshold, there is also a PFC threshold. When the queue cache of the switch (i.e., the target device) reaches the PFC threshold, it will notify the upstream device to stop sending traffic. When the ECN mark is triggered, it only notifies the server (i.e., other devices) to slow down. It can be seen that when sending congestion, it is expected to first trigger the ECN threshold to notify the speed reduction to alleviate the congestion and try to avoid triggering PFC to prevent interruption of sending. Therefore, in order to expand the throughput as much as possible while maintaining losslessness, it is hoped that the number of message caches is as large as possible, but it is not hoped that the PFC threshold will be reached and the flow will be stopped. Therefore, a reward and punishment mechanism that takes into account the above considerations is designed.

[0077] Figure 6 Schematic diagram of a reward and punishment mechanism for an ECN threshold adjustment action according to an embodiment of the present application Figure 1 .like Figure 6 As shown in the figure, the reward value r is the reciprocal of the number of packets cached in the target cache queue after the current ECN threshold adjustment is subtracted from the PFC threshold, multiplied by N (i.e., the capacity of the target cache queue); it can be found that when the threshold value is adjusted to a+3step, the reciprocal of the PFC threshold minus the number of packet caches is larger than the reciprocal calculated when the threshold value is adjusted to a+step and a+2step, and the reward obtained when the threshold value is adjusted to a+3step is the largest. Therefore, in state s, the tendency to execute action +3step is greater because more rewards can be obtained.

[0078] Figure 7 Schematic diagram of a reward and punishment mechanism for an ECN threshold adjustment action according to an embodiment of the present application Figure 2 .like Figure 7 As shown in the figure, in this reward and punishment mechanism, in addition to positive rewards, there are also negative rewards. When the ECN threshold is adjusted to a+3step based on state s, the number of cached messages in the target cache queue directly exceeds the PFC threshold. Then the PFC threshold minus the number of cached messages is a negative value, indicating that it is not expected to execute action +3step in state s, that is, there is a very small probability of executing +3step.

[0079] Step S4, the learning process of ECN threshold adjustment:

[0080] After the three key elements of the SARSA online learning algorithm, state s, action x, and reward r, have been set, the Q function value can be learned based on the existing network environment. The learning process is as follows:

[0081] (1) The initial state of the target cache queue message cache is s (corresponding to the reference control period). Assume that there are three optional executable actions x (i.e., threshold adjustment operations), namely ECN threshold + step, ECN threshold + 2step, and ECN threshold + 3step. These three actions correspond to three Q action value functions Q t (s,step),Q t (s,2step),Q t (s, 3step); Based on the greedy strategy, the action with the largest Q value function is selected according to the probability of β, and an action is randomly selected according to the probability of (1-β). β can be defined as 90%. At this time, there is an action value function Q t (s,x) (i.e., initial adjustment parameters), and at the same time obtain the reward value r (i.e., gain parameter).

[0082] (2) Based on the selected action x (i.e., the initial adjustment operation), the number of message buffers (i.e., queue information) is observed after execution to obtain the next state s' (corresponding to the target control cycle). In the state s', an action x' (i.e., the reference adjustment operation) is also selected based on the greedy strategy. Then, Q is obtained. t (s',x') (i.e., reference adjustment parameters).

[0083] (3) Based on SARSA learning algorithm: Q (t+1) (s,x)=Q t (s,x)+α[r+γQ t (s',x')-Q t (s,x)], the new action value function Q can be calculated (t+1) (s,x).

[0084] (4) Based on the existing network traffic data, through repeated training and iteration, we can eventually derive the action value function Q(s,x) of all actions in all states. Based on the Q action value function, we can infer the optimal ECN waterline adjustment method. Figure 8 is a schematic diagram of an action value function table according to an embodiment of the present application. Figure 8 As shown in the figure, after repeated training and iteration, the Q action value function table (i.e., prediction model) is obtained. The optimal action (i.e., target adjustment operation) performed under different states s can be selected according to the maximum Q value principle to adjust the queue threshold, thereby achieving the purpose of dynamically adjusting the ECN threshold. Moreover, the value of the dynamically adjusted ECN threshold is the optimal ECN threshold value under this state, which can achieve the maximum bandwidth throughput.

[0085] Through the above steps, the SARSA online learning algorithm is implemented. It only needs to obtain the number of cached messages in the target cache queue of the chip without too much information. Algorithm learning and training can be performed to realize the method of dynamically adjusting the ECN threshold. In any network traffic model, it can obtain the maximum network throughput while avoiding packet loss or interruption.

[0086] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software and a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0087] Fig. 9 is a structural block diagram of a cache queue threshold configuration device according to an embodiment of the present application; Fig. 9 As shown, including:

[0088] An acquisition module 902 is used to obtain, when a target control cycle of a target cache queue of a target device is currently reached, queue information of a target cache queue after adjusting a queue threshold of the target cache queue according to an initial adjustment operation in a reference control cycle before a current moment, wherein the target cache queue is used to cache data transmitted from other devices to the target device, and the queue information is used to indicate a transmission efficiency of transmitting data from other devices to the target device through the target cache queue, and the initial adjustment operation is selected from a plurality of threshold adjustment operations using an initial prediction model, and the initial prediction model is used to predict an adjustment effect achieved by executing the threshold adjustment operation on an adjustment target, and the adjustment target includes improving the efficiency of data transmission from other devices to the target device;

[0089] An updating module 904 is used to update the initial prediction model according to the queue information to obtain a target prediction model;

[0090] An input module 906 is used to input the current target queue information of the target cache queue into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model, wherein the operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target;

[0091] A determination module 908, configured to determine a target adjustment operation from a plurality of threshold adjustment operations according to the operation information;

[0092] The adjustment module 910 is used to adjust the current target threshold value of the target cache queue according to the target adjustment operation.

[0093] Through the above-mentioned embodiment, when the target control cycle of the target cache queue of the target device is currently reached, the queue information of the target cache queue is obtained after the queue threshold of the target cache queue is adjusted according to the initial adjustment operation in the reference control cycle before the current moment; the initial prediction model is updated according to the queue information to obtain the target prediction model; the current target queue information of the target cache queue is input into the target prediction model to obtain the operation information of each threshold adjustment operation output by the target prediction model; the target adjustment operation is determined from multiple threshold adjustment operations according to the operation information; the current target threshold value of the target cache queue is adjusted according to the target adjustment operation, that is, when the target control cycle of the target cache queue is reached Under the above, the transmission efficiency of the target device transmitting data from other devices to the target device after adjusting the queue threshold according to the initial adjustment operation obtained by using the initial prediction model within the reference control period can be collected. Further, the improvement effect of the initial adjustment operation on improving the data transmission efficiency of data transmitted from other devices to the target device can be obtained. The initial prediction model is updated with the help of the obtained improvement effect, and a target prediction model that predicts the adjustment effect that can be achieved by the threshold adjustment operation on improving the data transmission efficiency can be obtained more accurately. Then, the target prediction model and the current queue information are used to determine the target adjustment operation, which can achieve a better adjustment effect on improving the data transmission efficiency, that is, a more suitable queue threshold can be finally obtained. The above technical solution solves the problem of low configuration efficiency of the queue threshold in the related technology, and achieves the technical effect of improving the configuration efficiency of the queue threshold.

[0094] In an exemplary embodiment, an update module includes: a first screening unit, used to screen a reference adjustment operation from multiple threshold adjustment operations using an initial prediction model according to queue information; a calculation unit, used to calculate a change adjustment parameter of the initial adjustment operation according to a reference adjustment parameter of the reference adjustment operation, an initial adjustment parameter of the initial adjustment operation, and a gain parameter of the initial adjustment operation, wherein the reference adjustment parameter is used to indicate the adjustment effect of the reference adjustment operation predicted by the initial prediction model on the data transmission efficiency within a target control period, the initial adjustment parameter is used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the reference control period, the gain parameter is used to indicate the adjustment effect achieved on the adjustment target after executing the initial adjustment operation, and the change adjustment parameter is used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period; an update unit, used to update the initial prediction model by changing the adjustment parameter.

[0095] In an exemplary embodiment, the computing unit is further used to: convert the cache parameters of the target cache queue within a reference control cycle into a first queue state and convert the cache parameters of the target cache queue within a target control cycle into a second queue state, wherein the queue state of the target cache queue corresponds to the capacity interval of the occupied cache capacity of the target cache queue, the cache parameters are used to indicate the cache space of the target cache queue occupied by the target cache queue to store data, and the queue information includes the cache parameters; use an initial prediction model to predict initial adjustment parameters of an initial adjustment operation in a first queue state, and reference adjustment parameters of a reference adjustment operation in a second queue state; and calculate the change adjustment parameters of the initial adjustment operation based on the reference adjustment parameters, the initial adjustment parameters, and the gain parameters of the initial adjustment operation.

[0096] In an exemplary embodiment, the calculation unit is also used to: detect the size relationship between the cache parameter and the cache threshold of the target cache queue; when it is detected that the cache parameter is greater than or equal to the cache threshold, obtain the difference between the cache threshold and the cache parameter to obtain a first gain parameter; calculate the change adjustment parameter of the initial adjustment operation based on the reference adjustment parameter, the initial adjustment parameter and the first gain parameter; when it is detected that the cache parameter is less than the cache threshold, obtain the inverse of the difference between the cache threshold and the cache parameter, calculate the product of the inverse and the capacity of the target cache queue to obtain a second gain parameter, wherein the capacity is used to indicate the data cache space of the target cache queue; calculate the change adjustment parameter of the initial adjustment operation based on the reference adjustment parameter, the initial adjustment parameter and the second gain parameter.

[0097] In an exemplary embodiment, the calculation unit is further configured to calculate the change adjustment parameter by the following formula:

[0098] Q (t+1) =Q t +α(r+γQ t '-Q t );

[0099] Among them, Q (t+1) is the change adjustment parameter, Q t is the initial adjustment parameter, Q t ' is the reference tuning parameter, r is the gain parameter, α is the learning rate, and γ is the discount factor.

[0100] In an exemplary embodiment, the first screening unit is also used to: generate a target random number; compare the target random number with the current random number threshold; when the target random number is greater than or equal to the current random number threshold, screen out the threshold adjustment operation with the best adjustment effect predicted by the initial prediction model from multiple threshold adjustment operations as a reference adjustment operation; when the target random number is less than the current random number threshold, randomly screen out a threshold adjustment operation from multiple threshold adjustment operations as a reference adjustment operation.

[0101] In an exemplary embodiment, the determination module includes: a second screening unit, configured to screen out, from a plurality of threshold adjustment operations, a threshold adjustment operation with the best adjustment effect predicted by a target prediction model as a target adjustment operation.

[0102] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein when the program is run, any of the above-mentioned methods for configuring a cache queue threshold is executed.

[0103] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0104] S1, when a target control cycle of a target cache queue of a target device is currently reached, queue information of the target cache queue after adjusting the queue threshold of the target cache queue according to an initial adjustment operation in a reference control cycle before the current moment, wherein the target cache queue is used to cache data transmitted from other devices to the target device, and the queue information is used to indicate the transmission efficiency of data transmitted from other devices to the target device through the target cache queue, and the initial adjustment operation is selected from multiple threshold adjustment operations using an initial prediction model, and the initial prediction model is used to predict the adjustment effect achieved by executing the threshold adjustment operation on the adjustment target, and the adjustment target includes improving the data transmission efficiency of other devices to the target device;

[0105] S2, updating the initial prediction model according to the queue information to obtain the target prediction model;

[0106] S3, inputting the current target queue information of the target cache queue into the target prediction model, and obtaining the operation information of each threshold adjustment operation output by the target prediction model, wherein the operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target;

[0107] S4, determining a target adjustment operation from a plurality of threshold adjustment operations according to the operation information;

[0108] S5: Adjust the current target threshold value of the target cache queue according to the target adjustment operation.

[0109] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned cache queue threshold configuration method embodiments.

[0110] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0111] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0112] S1, when a target control cycle of a target cache queue of a target device is currently reached, queue information of the target cache queue after adjusting the queue threshold of the target cache queue according to an initial adjustment operation in a reference control cycle before the current moment, wherein the target cache queue is used to cache data transmitted from other devices to the target device, and the queue information is used to indicate the transmission efficiency of data transmitted from other devices to the target device through the target cache queue, and the initial adjustment operation is selected from multiple threshold adjustment operations using an initial prediction model, and the initial prediction model is used to predict the adjustment effect achieved by executing the threshold adjustment operation on the adjustment target, and the adjustment target includes improving the data transmission efficiency of other devices to the target device;

[0113] S2, updating the initial prediction model according to the queue information to obtain the target prediction model;

[0114] S3, inputting the current target queue information of the target cache queue into the target prediction model, and obtaining the operation information of each threshold adjustment operation output by the target prediction model, wherein the operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target;

[0115] S4, determining a target adjustment operation from a plurality of threshold adjustment operations according to the operation information;

[0116] S5: Adjust the current target threshold value of the target cache queue according to the target adjustment operation.

[0117] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0118] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0119] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0120] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for configuring a cache queue threshold, characterized in that: include: In the case where a target control cycle of a target cache queue of a target device is currently reached, queue information of the target cache queue after adjusting the queue threshold of the target cache queue according to an initial adjustment operation in a reference control cycle before the current moment, wherein the target cache queue is used to cache data transmitted from other devices to the target device, and the queue information is used to indicate a transmission efficiency of transmitting data from the other devices to the target device through the target cache queue, and the initial adjustment operation is selected from a plurality of threshold adjustment operations using an initial prediction model, and the initial prediction model is used to predict an adjustment effect achieved by executing the threshold adjustment operation on an adjustment target, and the adjustment target includes improving the data transmission efficiency from the other devices to the target device; The initial prediction model is updated according to the queue information to obtain a target prediction model; Inputting the current target queue information of the target cache queue into the target prediction model to obtain operation information of each threshold adjustment operation output by the target prediction model, wherein the operation information is used to indicate the adjustment effect achieved by each adjustment operation on the adjustment target; Determining a target adjustment operation from the plurality of threshold adjustment operations according to the operation information; The current target threshold value of the target cache queue is adjusted according to the target adjustment operation.

2. The method according to claim 1, characterized in that The updating of the initial prediction model according to the queue information includes: Filtering a reference adjustment operation from the plurality of threshold adjustment operations using the initial prediction model according to the queue information; The changed adjustment parameters of the initial adjustment operation are calculated according to the reference adjustment parameters of the reference adjustment operation, the initial adjustment parameters of the initial adjustment operation, and the gain parameters of the initial adjustment operation, wherein the reference adjustment parameters are used to indicate the adjustment effect of the reference adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period, the initial adjustment parameters are used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the reference control period, the gain parameters are used to indicate the adjustment effect achieved on the adjustment target after executing the initial adjustment operation, and the changed adjustment parameters are used to indicate the adjustment effect of the initial adjustment operation predicted by the initial prediction model on the data transmission efficiency within the target control period; The initial prediction model is updated by changing the adjustment parameters.

3. The method according to claim 2, characterized in that The calculating the changed adjustment parameter of the initial adjustment operation according to the reference adjustment parameter of the reference adjustment operation, the initial adjustment parameter of the initial adjustment operation and the gain parameter of the initial adjustment operation comprises: Converting the cache parameters of the target cache queue in the reference control period into a first queue state and converting the cache parameters of the target cache queue in the target control period into a second queue state, wherein the queue state of the target cache queue has a corresponding relationship with a capacity interval of an occupied cache capacity of the target cache queue, the cache parameters are used to indicate the cache space of the target cache queue occupied by the target cache queue to store the data, and the queue information includes the cache parameters; Using the initial prediction model to predict the initial adjustment parameters of the initial adjustment operation in the first queue state, and the reference adjustment parameters of the reference adjustment operation in the second queue state; The modified adjustment parameter of the initial adjustment operation is calculated according to the reference adjustment parameter, the initial adjustment parameter and the gain parameter of the initial adjustment operation.

4. The method according to claim 3, characterized in that The calculating the changed adjustment parameter of the initial adjustment operation according to the reference adjustment parameter, the initial adjustment parameter and the gain parameter of the initial adjustment operation comprises: Detecting a magnitude relationship between the cache parameter and a cache threshold of the target cache queue; When it is detected that the cache parameter is greater than or equal to the cache threshold, obtaining a difference between the cache threshold and the cache parameter to obtain a first gain parameter; calculating a changed adjustment parameter of the initial adjustment operation according to the reference adjustment parameter, the initial adjustment parameter and the first gain parameter; When it is detected that the cache parameter is less than the cache threshold, the inverse of the difference between the cache threshold and the cache parameter is obtained, and the product of the inverse and the capacity of the target cache queue is calculated to obtain a second gain parameter, wherein the capacity is used to indicate the data cache space of the target cache queue; and the change adjustment parameter of the initial adjustment operation is calculated according to the reference adjustment parameter, the initial adjustment parameter and the second gain parameter.

5. The method according to claim 2, characterized in that: The calculating the changed adjustment parameter of the initial adjustment operation according to the reference adjustment parameter of the reference adjustment operation, the initial adjustment parameter of the initial adjustment operation and the gain parameter of the initial adjustment operation comprises: The change adjustment parameter is calculated by the following formula: Q (t+1) =Q t +α(r+γQ t '-Q t ); Among them, Q (t+1) is the change adjustment parameter, Q t is the initial tuning parameter, Q t ' is the reference adjustment parameter, r is the gain parameter, α is the learning rate, and γ is the discount factor.

6. The method according to claim 2, characterized in that The step of selecting a reference adjustment operation from the plurality of threshold adjustment operations using the initial prediction model according to the queue information includes: Generate target random number; Compare the target random number with the current random number threshold; In a case where the target random number is greater than or equal to the current random number threshold, selecting the threshold adjustment operation with the best adjustment effect predicted by the initial prediction model from the multiple threshold adjustment operations as the reference adjustment operation; In a case where the target random number is less than the current random number threshold, one of the threshold adjustment operations is randomly selected from the multiple threshold adjustment operations as the reference adjustment operation.

7. The method according to claim 1, characterized in that The step of determining a target adjustment operation from the plurality of threshold adjustment operations according to the operation information includes: The threshold adjustment operation with the best adjustment effect predicted by the target prediction model is selected from the multiple threshold adjustment operations as the target adjustment operation.

8. A cache queue threshold configuration device, characterized in that: include: an acquisition module, configured to, when a target control cycle of a target cache queue of a target device is currently reached, acquire queue information of the target cache queue after adjusting the queue threshold of the target cache queue according to an initial adjustment operation in a reference control cycle before a current moment, wherein the target cache queue is used to cache data transmitted from other devices to the target device, and the queue information is used to indicate a transmission efficiency of transmitting data from the other devices to the target device through the target cache queue, and the initial adjustment operation is selected from a plurality of threshold adjustment operations using an initial prediction model, and the initial prediction model is used to predict an adjustment effect achieved by executing the threshold adjustment operation on an adjustment target, and the adjustment target includes improving the data transmission efficiency from the other devices to the target device; An updating module, used for updating the initial prediction model according to the queue information to obtain a target prediction model; An input module, used for inputting the current target queue information of the target cache queue into the target prediction model, and obtaining the operation information of each threshold adjustment operation output by the target prediction model, wherein the operation information is used for indicating the adjustment effect achieved by each adjustment operation on the adjustment target; A determination module, configured to determine a target adjustment operation from the plurality of threshold adjustment operations according to the operation information; An adjustment module is used to adjust the current target threshold value of the target cache queue according to the target adjustment operation.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images