Data management method and device, storage medium and computer equipment

Abstract

The embodiment of the invention provides a data management method and device, a storage medium and computer equipment, and the method comprises the steps: periodically receiving congestion control transmission type data, and carrying out the statistics of the buffer occupation information of the congestion control transmission type data; if so, integrating current period related data information to obtain current period integrated data, and inputting the current period integrated data into a prediction model in combination with historical period integrated data to obtain a predicted target data volume; and if the target occupancy reaches a larger second occupancy threshold, obtaining the current total occupancy of the buffer area and the occupancy of each type of current data. And finally, integrating the target data volume, the current total occupancy and the current data occupancy, scientifically managing buffer data, optimizing resource utilization, preventing mutual preemption of different types of data, improving data transmission efficiency, and ensuring stable and efficient operation of the network so as to cope with network congestion challenges and meet diversified data transmission requirements.

Description

Technical Field

[0001] The present application relates to the field of data management technology, and in particular to a data management method, device, storage medium and computer equipment. Background Art

[0002] With the enrichment of application scenarios, the research on network congestion has become more and more in-depth, and various types of congestion control algorithms have emerged. At present, traditional congestion control transmission algorithms are mainly divided into three categories: 1. Transmission Control Protocol (TCP) based on packet loss feedback, 2. TCP protocol based on delay feedback, and 3. High-speed bandwidth algorithm based on packet loss feedback. There are large differences in the perception and processing schemes of these algorithms for congestion. Taking the high-speed algorithm of packet loss feedback as an example, these algorithms generally consider that there is packet loss after receiving three repeated acknowledgment packets (ACK) and perform corresponding congestion control. The congestion control protocol based on delay feedback will sense whether there is congestion in the current network by observing the changes in data packet delay, and intervene in advance to reduce the negative impact of packet loss.

[0003] In the related art, the use of different strategies will cause unfairness when algorithms compete with each other. Through research, it is found that one of the main reasons for this unfairness is that the buffer management of the router adopts the same management scheme for all algorithms. When the buffer setting in the router is large, the data packet delay is susceptible to fluctuations but packet loss is not easy to occur. This will cause the TCP protocol based on delay feedback to lack competitive advantages, and the available bandwidth will be occupied by the TCP protocol based on packet loss feedback. On the contrary, when the buffer setting is small, a burst flow may cause a large number of data packets to be lost, while the delay of data packets that can be sent normally is not greatly affected. This will cause the TCP protocol based on packet loss feedback to lack competitive advantages, and the available bandwidth will be occupied by the TCP protocol based on delay feedback, thereby causing the data transmitted by different congestion algorithms to preempt each other, resulting in low data transmission efficiency. Therefore, the related art urgently needs to propose a data management method to solve the above technical problems. Summary of the invention

[0004] The main purpose of this application is to provide a data management method, device, storage medium and computer equipment, which can comprehensively consider the various data information obtained and analyzed, scientifically and reasonably manage the data in the buffer, optimize the utilization of buffer resources, avoid mutual preemption between data transmitted by different congestion algorithms, and improve the efficiency of data transmission.

[0005] In a first aspect, an embodiment of the present application provides a data management method, including:

[0006] Periodically receiving data corresponding to at least one congestion control transmission type transmitted at each time point, and counting buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point;

[0007] If at the current time point there is data of at least one target congestion control transmission type whose target occupancy in the buffer is greater than or equal to the first occupancy threshold, then obtaining data information of the data transmitted at each time point in the current cycle at the current time point, as well as buffer occupancy information at each time point;

[0008] Integrate each piece of data information and each piece of buffer occupancy information to obtain current cycle integrated data corresponding to the current cycle;

[0009] Obtain historical period integrated data of the target congestion control transmission type, and input the current period integrated data and the historical period integrated data into a data volume prediction model corresponding to the target congestion control transmission type to obtain a predicted target data volume;

[0010] When the target occupancy is greater than or equal to a second occupancy threshold, obtaining a current total occupancy of the buffer and a current data occupancy of data of each congestion control transmission type in the buffer, and the second occupancy threshold is greater than the first occupancy threshold;

[0011] The data in the buffer is managed based on the target data volume, the current total occupancy and the current data occupancy.

[0012] In a second aspect, an embodiment of the present application provides a data management device, including:

[0013] A statistical unit, used to periodically receive data corresponding to at least one congestion control transmission type transmitted at each time point, and count buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point;

[0014] A first acquisition unit is used to acquire data information of data transmitted at each time point in the current cycle at the current time point, and buffer occupancy information at each time point, if at a current time point there is data of at least one target congestion control transmission type whose target occupancy in the buffer is greater than or equal to a first occupancy threshold;

[0015] An integration unit, used to integrate each of the data information and each of the buffer occupancy information to obtain current cycle integration data corresponding to the current cycle;

[0016] An input unit, used to obtain historical period integrated data of the target congestion control transmission type, and input the current period integrated data and the historical period integrated data into a data volume prediction model corresponding to the target congestion control transmission type to obtain a predicted target data volume;

[0017] A second acquisition unit, configured to acquire a current total occupancy of the buffer and a current data occupancy of data of each congestion control transmission type in the buffer when the target occupancy is greater than or equal to a second occupancy threshold, and the second occupancy threshold is greater than the first occupancy threshold;

[0018] A management unit is used to manage the data in the buffer based on the target data volume, the current total occupancy and the current data occupancy.

[0019] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a plurality of instructions suitable for loading by a processor to execute any of the above data management methods.

[0020] In a fourth aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above data management methods when executing the computer program.

[0021] In an embodiment of the present application, data corresponding to at least one congestion control transmission type transmitted at each time point is periodically received, and buffer occupancy information of the data corresponding to each congestion control transmission type at each time point in the buffer is counted; if there is at least one target congestion control transmission type at the current time point whose target occupancy in the buffer is greater than or equal to a first occupancy threshold, data information of the data transmitted at each time point in the current cycle at the current time point and buffer occupancy information at each time point are obtained; each of the data information and each of the buffer occupancy information are integrated to obtain current cycle integrated data corresponding to the current cycle; historical cycle integrated data of the target congestion control transmission type is obtained, and the current cycle integrated data and the historical cycle integrated data are input into the data volume prediction corresponding to the target congestion control transmission type. A measurement model is used to obtain a predicted target data volume; when the target occupancy is greater than or equal to a second occupancy threshold, the current total occupancy of the buffer zone and the current data occupancy of data of each congestion control transmission type in the buffer zone are obtained, and the second occupancy threshold is greater than the first occupancy threshold; based on the target data volume, the current total occupancy and the current data occupancy, the data in the buffer zone is managed. Compared with the related art, the buffer management of the router adopts the same management scheme for all algorithms, which leads to the mutual preemption of data transmitted by different congestion algorithms, resulting in low data transmission efficiency. The embodiment of the present application can comprehensively consider the various data information obtained by acquisition and analysis, scientifically and reasonably manage the data in the buffer zone, optimize the utilization of buffer zone resources, avoid the mutual preemption of data transmitted by different congestion control transmission algorithms, and improve the efficiency of data transmission.

[0022] Other features and advantages of the present disclosure will be described in the following description, and partly become apparent from the description, or understood by practicing the present disclosure. The purpose and other advantages of the present disclosure can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A schematic diagram of a scenario of a data management system provided in an embodiment of the present application.

[0025] Figure 2A flowchart of a data management method provided in an embodiment of the present application.

[0026] Figure 3 A schematic diagram of the structure of a data management method and device provided in an embodiment of the present application.

[0027] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0029] It should be noted that in some processes described in the specification, claims and the above-mentioned drawings, multiple steps appearing in a specific order are included, but it should be clearly understood that these steps may not be executed in the order in which they appear in this document or may be executed in parallel. The step numbers are only used to distinguish different steps, and the numbers themselves do not represent any execution order. In addition, descriptions such as "first", "second" or "target" in this document are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0030] Before further describing the embodiments of the present disclosure in detail, the nouns and terms involved in the embodiments of the present disclosure are described. The nouns and terms involved in the embodiments of the present disclosure are subject to the following interpretations:

[0031] TCP protocol based on packet loss feedback:

[0032] 1. Congestion perception: When three repeated ACK packets are received, it is considered that there is packet loss, which is used as a signal of network congestion. For example, during data transmission, if the sender receives three consecutive ACK confirmation packets for the same sequence number, it is determined that there is packet congestion in the network.

[0033] 2. Congestion handling: Once congestion is determined, corresponding congestion control measures will be taken, such as reducing the sending rate. A common practice is to halve the sending window size to reduce the amount of data entering the network and alleviate congestion.

[0034] For example, the original sending window size is 100. After congestion is detected, the sending window is adjusted to 50 to reduce the speed of sending data.

[0035] TCP protocol based on delayed feedback:

[0036] 1. Congestion perception: By observing the changes in packet delay, we can sense whether there is congestion in the current network. We will continuously monitor the time it takes for a packet to be sent and received. If we find that the delay increases significantly, we think that the network may be congested. For example, if the round-trip time of a packet is 10ms under normal circumstances and it suddenly increases to 50ms, it will cause concern.

[0037] 2. Congestion handling: Intervene in advance to reduce the negative impact of packet loss. When an increase in latency is detected, the sending rate will be adjusted appropriately to avoid further network deterioration and packet loss. Usually, a more conservative approach is adopted to reduce the sending rate, rather than aggressively halving the window like the TCP protocol based on packet loss feedback.

[0038] For example, according to the degree of delay increase, the sending rate is reduced by a certain proportion, such as 10%-20%.

[0039] High-speed bandwidth algorithm based on packet loss feedback:

[0040] 1. Congestion perception: Similar to the TCP protocol based on packet loss feedback, it also considers packet loss after receiving a certain number (such as three) of repeated ACK packets, thereby judging network congestion. This is because in a high-speed network environment, packet loss is still an important congestion indication signal.

[0041] 2. Congestion handling: When handling congestion, optimization will be performed based on the characteristics of the high-speed network. For example, the sending rate and window size may be adjusted more quickly to meet the data transmission requirements under high-speed bandwidth. At the same time, more complex algorithms may be used to calculate new sending rates and window sizes to improve network utilization and transmission efficiency.

[0042] For example, in addition to simply halving the window, factors such as the remaining bandwidth of the network and the current number of connections are also considered to dynamically adjust the window size so that the network can recover to an efficient transmission state more quickly after congestion.

[0043] However, although the terminal device as the data sender can use different congestion control transmission algorithms to intervene in advance in the network congestion situation, thereby reducing the negative impact of packet loss, after the data is transmitted to the router's buffer, since the buffer management strategy for each type of congestion control transmission algorithm is consistent, when the buffer setting in the router is large, the packet delay is prone to fluctuations but packet loss is not easy to occur, which will cause the TCP protocol based on delay feedback to lack a competitive advantage, and the available bandwidth will be preempted by the TCP protocol based on packet loss feedback. On the contrary, when the buffer setting is small, a burst flow may cause a large number of packet losses, while the delay of the packets that can be sent normally is not greatly affected. This will cause the TCP protocol based on packet loss feedback to lack a competitive advantage, and the available bandwidth will be preempted by the TCP protocol based on delay feedback, resulting in low data transmission efficiency.

[0044] TOS (Type of Service) field: It is a field in the IP datagram header, with a length of 8 bits, used to specify the service type of the IP data packet. It is mainly used to convey the priority of the data packet and the requirements for different quality of service (QoS) to network devices (such as routers), helping network devices to better manage traffic and allocate resources.

[0045] rx_trace function: usually used in the software system of network devices (such as routers, network interface cards, etc.), it is a trace function related to receiving (receive, rx) data packets. Its main function is to record various information when the data packet enters the network device, such as the source of the data packet (source IP address, source MAC address), arrival time, data packet size, protocol used, and many other details.

[0046] XGBoost (eXtreme Gradient Boosting): is an efficient gradient boosting algorithm based on decision trees, which is widely used in data mining and machine learning, especially in the classification and regression tasks of structured data. It is an optimization and extension of the traditional gradient boosting algorithm, and is favored for its efficiency, accuracy and scalability.

[0047] Its working principle includes:

[0048] 1. Gradient Boosting Framework:

[0049] XGBoost belongs to the family of gradient boosting algorithms in ensemble learning. Its basic idea is to solve the problem by building multiple weak learners (mainly decision trees in XGBoost) and combining them into a strong learner. Each weak learner tries to correct the mistakes of the previous learner.

[0050] For example, in a regression task, we first initialize a prediction value (such as simply taking the mean of the target variable), and then build the first decision tree to fit the residual (error) between the target variable and this initial prediction value. After that, we add the prediction result of this decision tree to the initial prediction value to get a new prediction value, and then build the next decision tree to fit the new residual, and so on.

[0051] 2. Decision tree construction:

[0052] The decision tree in XGBoost is its core component. The decision tree construction process is achieved by performing feature selection and splitting nodes on the training data. It will select the best features and splitting points based on a certain splitting criterion (such as Gini coefficient, information gain, etc.) to make the data purity of the child nodes after the split higher.

[0053] For example, in a classification task to determine whether a user will buy a product, for data containing features such as user age, income, and purchase history, the decision tree may select the "income" feature as the first split point based on information gain, and divide the users into high-income and low-income groups, and then continue to select other features in each group for splitting until the stopping condition is met (such as the depth of the tree reaches the limit, the number of samples in the child node is too small, etc.).

[0054] 3. Regularization and prevention of overfitting:

[0055] XGBoost uses a variety of regularization techniques to prevent overfitting. These include penalizing the complexity of the decision tree, such as by controlling parameters such as the depth of the tree and the number of leaf nodes. At the same time, it also introduces a shrinkage mechanism, which uses only a small portion (usually controlled by the learning rate) of the prediction results of the new decision tree each time the prediction results are updated, making the model training process more robust.

[0056] For example, when training a model on a complex dataset, without regularization, the model may overfit the noise in the training data, resulting in poor performance on new data. XGBoost's regularization mechanism can limit the complexity of the model, allowing the model to generalize better to unknown data.

[0057] Low-wave filter queue: is a mechanism for processing network data queues. Its main purpose is to smooth the fluctuations in queue length data, so that network devices (such as routers) can manage and control data traffic more effectively. It filters the original queue length data through certain algorithms and rules to reduce high-frequency fluctuations caused by factors such as instantaneous traffic changes and measurement errors.

[0058] In order to solve the above-mentioned problem, the embodiment of the present application periodically receives data corresponding to at least one congestion control transmission type transmitted at each time point, and counts the buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point; if there is at least one target congestion control transmission type at the current time point and the target occupancy of the data in the buffer is greater than or equal to the first occupancy threshold, then obtain the data information of the data transmitted at each time point in the current cycle of the current time point, and the buffer occupancy information of each time point; integrate each of the data information and each of the buffer occupancy information to obtain the current cycle integrated data corresponding to the current cycle; obtain the historical cycle integrated data of the target congestion control transmission type, and input the current cycle integrated data and the historical cycle integrated data into the data corresponding to the target congestion control transmission type. According to the data volume prediction model, the predicted target data volume is obtained; when the target occupancy is greater than or equal to the second occupancy threshold, the current total occupancy of the buffer and the current data occupancy of the data of each congestion control transmission type in the buffer are obtained, and the second occupancy threshold is greater than the first occupancy threshold; based on the target data volume, the current total occupancy and the current data occupancy, the data in the buffer is managed. Compared with the related art, the buffer management of the router adopts the same set of management schemes for all algorithms, resulting in the mutual preemption of data transmitted by different congestion algorithms, resulting in low data transmission efficiency. The embodiments of the present application can comprehensively consider the various data information obtained and analyzed, manage the data in the buffer scientifically and reasonably, optimize the utilization of buffer resources, avoid the mutual preemption of data transmitted by different congestion control transmission algorithms, and improve the efficiency of data transmission. Please continue to refer to the following specific embodiments for details.

[0059] See also Figure 1 , Figure 1 A schematic diagram of a scenario of a data management system provided in an embodiment of the present application, which includes a terminal device 130, a router 120, a server 110, etc.

[0060] The terminal device 130 includes but is not limited to a pre-configured personal computer, or a tablet computer, a desktop computer, or other electronic device with data transmission capability. In addition, it can be a single device or a collection of multiple devices. The terminal device 130 can communicate with the router 120 in a wired or wireless manner to exchange data.

[0061] The server 110 refers to a computer system that can transmit data (such as training data) to the terminal device 130. Compared with ordinary terminals, the server 110 has higher requirements in terms of stability, security, performance, etc. The server 110 can be a high-performance computer in a network platform, a cluster of multiple high-performance computers, a part of a high-performance computer (such as a virtual machine), a combination of parts of multiple high-performance computers (such as virtual machines), etc.

[0062] The router 120 is a network device for connecting different networks, and it can forward data packets between multiple networks. Its main function is to send data packets from one network to another network according to the destination IP address in the data packet.

[0063] For example, in Figure 1 In the example, the router 120 may forward the data transmitted by the terminal device 130 to the server 110, so that the server 110 acquires the data transmitted by the terminal device 130. The data management method of the embodiment of the present disclosure may be implemented by the router 120.

[0064] It should be noted that Figure 1 The scenario diagram of the data management system shown is merely an example. The data management system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can appreciate that with the evolution of data management technology and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0065] In this embodiment, the description will be made from the perspective of a data management device, which can be specifically integrated in a computer device having a storage unit and a microprocessor installed therein and having computing capabilities.

[0066] See also Figure 2 , Figure 2 A flow chart of a data management method provided in an embodiment of the present application. The data management method includes:

[0067] In step 201, data corresponding to at least one congestion control transmission type transmitted at each time point is periodically received, and buffer occupancy information of data corresponding to each congestion control transmission type in the buffer at each time point is counted.

[0068] Among them, when the router receives the transmitted data from the terminal device, it will first store it in the buffer inside the router and wait for subsequent transmission. In order to observe and analyze the network data transmission and buffer status in stages, a cycle is set, which is a time interval, for example, every 10 minutes is set as a cycle. Congestion control transmission types include at least one of the transmission control protocol (TCP) based on packet loss feedback, the TCP protocol based on delay feedback, and the high-speed bandwidth algorithm based on packet loss feedback. The buffer occupancy information includes the buffer occupancy size of the data corresponding to each congestion control transmission type in the buffer at each time t in the current cycle, the total number of data in the buffer, the total number of data, the length of the corresponding type queue, the number of queues, etc.

[0069] Specifically, when a terminal device transmits data, the terminal device uses different congestion control transmission types to control the transmission of data. Therefore, the terminal device marks the data it controls according to different congestion control transmission types. The specific content of the mark is the corresponding congestion control transmission type. The router parses the data to obtain the mark corresponding to each data, thereby determining the congestion control transmission type corresponding to each data.

[0070] For example, "0111" is used to identify the congestion control transmission type based on packet loss feedback, and "0101" is used to identify the congestion control transmission type based on delay feedback. There are two data, A and B, where A controls the transmission of data A through the congestion control transmission type based on packet loss feedback, and B controls the transmission of data B through the congestion control transmission type based on delay feedback; the terminal device will write "0111" in the TOS field of the data A message header, and write "0101" in the TOS field of the data B message header. The router obtains each data through the rx_trace function, and confirms the congestion control transmission type used by the terminal device during data transmission by parsing the TOS field, thereby informing the router that data A and data B are controlled by different congestion control transmission types for transmission.

[0071] In this way, the router receives data transmitted from various data sources at each time point according to the pre-set cycle through the built-in monitoring module. At the same time, it uses its own statistical function to count the occupancy of each type of congestion control transmission data received at each time point in the buffer.

[0072] In step 202, if at the current time point there is at least one target congestion control transmission type of data whose target occupancy in the buffer is greater than or equal to the first occupancy threshold, then the data information of the data transmitted at each time point in the current cycle at the current time point and the buffer occupancy information at each time point are obtained.

[0073] Among them, because the traffic in the data center is large, if the data volume corresponding to each congestion control transmission type is predicted periodically or in real time, it will easily occupy too many computing resources. Therefore, a first occupancy threshold will be set for each congestion control transmission type. Only when the buffer occupancy of the congestion control transmission type exceeds the first occupancy threshold, the corresponding data volume prediction will be performed.

[0074] Specifically, at each time point, the router will continuously monitor the occupancy of data of each congestion control transmission type in the buffer. Once it is found that the target occupancy of data of at least one target congestion control transmission type in the buffer at the current time point reaches or exceeds the first occupancy threshold, it means that the data of the router's target congestion control transmission type occupies a large proportion in the buffer. It will use its storage and query functions to obtain the data information of all data transmitted from each time point in the current cycle at the current time point (such as the total size of the data, the total number of data, the congestion control transmission type, etc.), as well as the buffer occupancy information corresponding to each time point (that is, the length of the corresponding type queue, the number of queues, etc. of the target congestion control transmission type in the buffer at each time point that has been counted previously).

[0075] In step 203, each piece of data information and each piece of buffer occupancy information is integrated to obtain current period integrated data corresponding to the current period.

[0076] Please refer to Table 1, which shows what data the current cycle integrated data includes.

[0077] Table 1

[0078]

[0079]

[0080] Specifically, the current cycle integrated data includes three types of information about the target congestion control transmission type in the current cycle. The first type is data integration information, which specifically includes the total data size, total number of data, and target congestion control transmission type of the target congestion control transmission type counted in the current cycle; the second type is buffer usage integration information, which specifically includes the queue length, queue type (i.e., target congestion control transmission type), average queue length (i.e., the ratio of the sum of the queue lengths counted at each time point to the total number of time points in the current cycle), buffer occupancy size, and average buffer occupancy size (i.e., the ratio of the sum of the buffer occupancy sizes at each time point to the total number of time points in the current cycle) of the target congestion control transmission type counted in the current cycle. The third type is integration time, which is the time for integrating the current cycle integrated data.

[0081] In step 204, historical cycle integrated data of the target congestion control transmission type is obtained, and the current cycle integrated data and the historical cycle integrated data are input into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume.

[0082] The historical cycle integrated data may be a preset number of target congestion control transmission type cycles before the current cycle, such as the first 10 cycles; or all cycles before the current cycle, which is not limited here. The purpose of using the current cycle integrated data and the historical cycle integrated data as input data is to find through experiments that combining the current cycle integrated data and the historical cycle integrated data can effectively improve the accuracy of the prediction results.

[0083] Specifically, each congestion control transmission type corresponds to a data volume prediction model, which is used to predict the expected data volume result (i.e., data volume) of the corresponding congestion control transmission type in a certain time period in the future. The data volume prediction model is essentially the XGboost algorithm, which is used to predict the target data volume of the target congestion control transmission type in a certain time period in the future.

[0084] Please refer to Table 2, which shows the data structure of the target data volume predicted by the data volume prediction model.

[0085] Table 2

[0086]

[0087] The data structure of the target data volume includes the congestion control transmission type of the data, the predicted time, and the predicted specific data volume.

[0088] In some implementations, after inputting the current period integrated data and the historical period integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume, the method further includes:

[0089] (1) recording the actual data volume of the data corresponding to the target congestion control transmission type;

[0090] (2) using the current period integrated data and the historical period integrated data as current sample data, and using the actual data volume as the label value of the current sample data;

[0091] (3) Training a data volume prediction model corresponding to the target congestion control transmission type through historical sample data and corresponding label values, the current sample data and corresponding label values.

[0092] Among them, after predicting the target data volume through the data volume prediction model corresponding to the target congestion control transmission type, the actual data volume of the data corresponding to the target congestion control transmission type can be recorded to collect the current sample data of the current cycle for the data volume prediction model. The prediction accuracy of the data volume prediction model can be further improved through offline training.

[0093] Specifically, the optimization method of the data volume prediction model can refer to the following formula:

[0094]

[0095] Among them, y i represents the actual observation value recorded in the i-th period, Indicates the amount of data corresponding to the prediction of the i-th period, It represents the average value of the actual observations corresponding to all periods. The value range of the formula result is 0-1. When the result is 1, it means that the prediction result is completely consistent with the observation value and the goodness of fit is the highest.

[0096] In terms of model optimization, the newly obtained statistical information will be found and written into the corresponding training data storage file according to the congestion control transmission type. Then, the new training data will be used in combination with the results of quantitative evaluation to adjust the model parameters to achieve the purpose of model optimization.

[0097] In step 205, when the target occupancy is greater than or equal to the second occupancy threshold, the current total occupancy of the buffer and the current data occupancy of each congestion control transmission type in the buffer are obtained, and the second occupancy threshold is greater than the first occupancy threshold.

[0098] Among them, after obtaining the predicted target data volume, the process enters the buffer management stage. Buffer management compares the occupancy of each congestion control transmission type in the current buffer with the total occupancy of all congestion control transmission types in the buffer to confirm which type of traffic is currently over-occupying the cache, and imposes certain penalties on the over-occupier to ensure that the effect of all data is fair. In order to avoid unnecessary waste caused by frequent calculations in buffer management, a penalty threshold is set (i.e., the second occupancy threshold, which is greater than the first occupancy threshold). Only when the buffer occupancy is greater than or equal to the second occupancy threshold, buffer management will punish the traffic of the dominant party to reduce the corresponding amount of data sent.

[0099] Specifically, when the target occupancy is greater than or equal to the second occupancy threshold, obtain the current total occupancy B of the buffer, the current data occupancy of each congestion control transmission type in the buffer (including the buffer size B occupied by the packet loss feedback protocol at the current time point t) l (t), and the buffer size N occupied by the delay feedback protocol at the current time point t d (t);

[0100] In step 206, the data in the buffer is managed based on the target data volume, the current total occupancy, and the current data occupancy.

[0101] In step 204, the predicted target data volume N is obtained. l (t), and step 205 obtains the current total occupancy B, the buffer size B occupied by the protocol with packet loss as feedback at the current time point t l (t), and the buffer size N occupied by the delay feedback protocol at the current time point t d (t) Afterwards, based on the above parameters, determine which of the two congestion control transmission types has an advantage in the buffer, causing the other congestion control transmission type to be at a disadvantage, and then perform corresponding data management.

[0102] In some implementations, managing the data in the buffer based on the target data volume, the current total occupancy, and the current data occupancy includes:

[0103] (1) when the target occupancy is greater than or equal to the second occupancy threshold and less than the third occupancy threshold, calculating the ratio of the current total occupancy to the current target data occupancy of the data of the target congestion control transmission type in the buffer to obtain an occupancy multiple;

[0104] (2) obtaining a sum of the target data volume and the current occupancy of other data in the buffer zone by data of other congestion control transmission types to obtain a first predicted total occupancy;

[0105] (3) calculating a ratio of the target data volume to the first predicted total occupancy to obtain a first predicted occupancy ratio;

[0106] (4) obtaining a product of the occupancy multiple and the first predicted occupancy ratio to obtain a calculation result;

[0107] (5) comparing the calculation result with one to obtain a comparison result;

[0108] (6) Managing the data in the buffer according to the comparison result.

[0109] Among them, a discard threshold is also set, namely, the third occupancy threshold. When the target occupancy is greater than or equal to the second occupancy threshold and less than the third occupancy threshold, it means that although the target congestion control transmission type occupies a large part in the buffer, it has not reached the level where its data needs to be discarded. Therefore, at this time, it is necessary to determine which congestion control transmission type has an advantage in the buffer in order to reduce the amount of data transmitted.

[0110] Specifically, the method for determining the dominant congestion control transmission type can refer to the following formula:

[0111]

[0112] in, = The current total occupancy B and the current target data occupancy B of the target congestion control transmission type in the buffer l (t), i.e., the occupancy multiple; N l (t)+N d (t) is the target data volume N l (t) The current other data in the buffer occupies N with data of other congestion control transmission types. d The sum of (t), i.e., the first predicted total occupancy; The target data volume N l (t) and the first predicted total occupancy N l (t)+N d (t), i.e., the first predicted occupancy ratio; occupancy multiple Occupancy ratio with the first prediction The product of By comparing the calculation result with one, a comparison result is obtained, and the data in the buffer is managed according to the comparison result.

[0113] For example, the current total occupancy is B, and the target congestion control transmission type of data in the buffer currently occupies B. l (t) is 80, then the occupancy multiple is 100 / 80=1.25; the target data volume N l (t) is 30, and data of other congestion control transmission types occupy N of the current other data in the buffer. d (t) is 50, then the first predicted total occupancy is 30+50=80; the first predicted occupancy ratio is 30 / 80=1.875; the calculation result is 1.25×1.875=2.34375.

[0114] In some implementations, managing the data in the buffer according to the comparison result includes:

[0115] (1.1) when the comparison result is that the calculation result is less than one, reducing the transmission amount of data of the target congestion control transmission type in the buffer;

[0116] (1.2) When the comparison result is that the calculation result is greater than one, reducing the transmission volume of data of other congestion control transmission types in the buffer.

[0117] Among them, if the comparison result is that the calculation result is less than one, it means that the data of the current target congestion control transmission type has an advantage in the buffer, and the other congestion control transmission type falls into a disadvantage, then the transmission volume of the target congestion control transmission type is reduced; if the comparison result is that the calculation result is greater than one, it means that the data of another congestion control transmission type has an advantage in the buffer, and the other target congestion control transmission type falls into a disadvantage, then the transmission volume of the data of other congestion control transmission types is reduced.

[0118] In some embodiments, the method further comprises:

[0119] (1) when the target occupancy is greater than or equal to a third occupancy threshold, obtaining a sum of the target data volume and the current occupancy of other data of other congestion control transmission types in the buffer zone to obtain a second predicted total occupancy;

[0120] (2) calculating a ratio of the target data volume to the second predicted total occupancy to obtain a second predicted occupancy ratio;

[0121] (3) calculating the ratio of the current target data occupancy of the target congestion control transmission type in the buffer to the current total occupancy to obtain a current occupancy ratio;

[0122] (4) performing a weighted summation of the second predicted occupancy ratio and the current occupancy ratio to obtain an expected occupancy ratio;

[0123] (5) obtaining the remaining storage capacity of the buffer;

[0124] (6) determining, based on the expected occupancy ratio and the remaining storage capacity, an allowable amount of data for each of the congestion control transmission types to enter the buffer;

[0125] (7) Discarding data of each of the congestion control transmission types until the amount of data of each of the congestion control transmission types reaches the corresponding allowed amount of data.

[0126] Among them, if the target occupancy is greater than or equal to the third occupancy threshold, it means that the target congestion control transmission type occupies a very large part in the buffer, and it is necessary to partially discard the data to be transmitted to the buffer. The specific amount of data to be discarded needs to be calculated.

[0127] Specifically, before calculating the specific data to be discarded, it is necessary to calculate the expected occupancy ratio E b , the specific calculation method can refer to the following formula:

[0128]

[0129] in, is a custom parameter used to set the expected buffer's impact on the data share of the target congestion control transmission type; the second predicted total occupancy N l (t)+N d (t) is the target data volume N l (t) The current other data in the buffer occupies N with data of other congestion control transmission types. d (t) and the second predicted occupancy ratio The target data volume N l (t) and the second predicted total occupancy N l (t)+N d (t) ratio; current occupancy ratio That is, the data of the target congestion control transmission type occupies B in the current target data of the buffer. l (t) and the current total occupancy B. After obtaining the second predicted occupancy ratio and the current occupancy ratio, combined with The weighted sum of the two parameters is used to obtain the expected occupancy ratio E b .

[0130] Specifically, after calculating the expected occupancy ratio E bThen, combined with the current remaining storage capacity of the buffer, determine the allowed data volume of each congestion control transmission type entering the buffer; and discard the data of each congestion control transmission type until the data volume of each congestion control transmission type reaches the corresponding allowed data volume.

[0131] In some implementations, determining the allowed data volume of each of the congestion management transmission types that is allowed to enter the buffer based on the expected occupancy ratio and the remaining storage volume includes:

[0132] (1.1) determining an allowable occupancy ratio of the remaining storage amount based on the expected occupancy ratio and the second predicted total occupancy;

[0133] (1.2) calculating the product of the remaining storage capacity and the allowed occupancy ratio to obtain the allowed data volume for allowing data of the target congestion control transmission type to enter the buffer;

[0134] (1.3) Calculate the difference between the remaining storage capacity and the allowed data volume of the target congestion control transmission type that can enter the buffer, and obtain the allowed data volume of the other congestion control transmission types that can enter the buffer.

[0135] The calculation method of the allowed data volume of the target congestion control transmission type that is allowed to enter the buffer can refer to the following formula:

[0136]

[0137] Specifically, D l (t) is the allowed data volume of the target congestion control transmission type allowed to enter the buffer at time t; C is the remaining storage capacity; by substituting the parameters into the formula, the allowed data volume of the target congestion control transmission type allowed to enter the buffer can be obtained. This is the permitted occupancy ratio.

[0138] The calculation method for the amount of data allowed to enter the buffer for other congestion control transmission types can refer to the following formula:

[0139] D d (t) = CD l (t);

[0140] The allowed data volume D of the target congestion control transmission type allowed to enter the buffer at time t is calculated. l (t), then the amount of data allowed to enter the buffer for other congestion control transmission types is the remaining storage capacity C minus D l (t), the result is the remaining storage C divided by Dl (t) is the amount of data entering the buffer for storing data of other congestion control transmission types other than those specified in the buffer, that is, the amount of data allowed.

[0141] In some embodiments, the method further comprises:

[0142] (1) obtaining the historical low wave filter queue length of the low wave filter queue corresponding to each queue in the buffer at the previous time point, and the actual length of each queue in the buffer at the current time point;

[0143] (2) performing a weighted summation of the historical low-wave filter queue length and the actual length to obtain a current low-wave filter queue length of each low-wave filter queue at a current time point;

[0144] (3) deriving the current low-wave filter queue length of each of the low-wave filter queues to obtain a length growth trend;

[0145] (4) determining a length threshold of each queue based on a length growth trend of each low-wave filter queue and a third occupancy threshold;

[0146] (5) The data in the queue whose actual length is greater than the corresponding length threshold is discarded.

[0147] In data center networks, bursty data flows often occur. These data will fill up the router buffer in a short period of time, but will be emptied quickly. In order to cope with bursty flows and avoid unnecessary packet loss, a portion of space is often reserved for dealing with bursty flows when managing router buffers. In traditional methods, the real-time queue length is generally obtained and its gradient change is calculated to determine whether a bursty flow is currently being experienced. However, in actual use, it is difficult to obtain the accurate queue length in real time. If the real-time queue length is directly used to calculate the gradient to determine whether the queue is receiving a bursty flow, as in the traditional way, it is very easy to cause misjudgment. Therefore, the gradient performance of the low-wave filtering queue length is used to control the bursty flow.

[0148] Specifically, the current low-wave filter queue length can be calculated by referring to the following formula:

[0149]

[0150] in, is the length of the i-th low-wave filter queue with priority p at time t, Then the length of the i-th low-wave filter queue with priority p at time t-1 is the historical low-wave filter queue length of the low-wave filter queue corresponding to the queue at the previous time point; represents the actual length of the i-th queue with priority p at time t; γ is a preset fixed parameter. The current low-wave filter queue length of each low-wave filter queue at the current time point is obtained by weighted summing the historical low-wave filter queue length and the actual length.

[0151] By taking the derivative of the current low-wave filter queue length of each low-wave filter queue, the length growth trend is obtained, which is expressed as

[0152] The third occupancy threshold (i.e., the discard threshold) is represented by T(t), and the queue length threshold is K is a fixed parameter. If the actual length of the queue is less than or equal to the corresponding length threshold, no data will be discarded; if the actual length of the queue is greater than the corresponding length threshold, the data in it will be discarded.

[0153] From the above, it can be seen that the embodiment of the present application periodically receives data corresponding to at least one congestion control transmission type transmitted at each time point, and counts the buffer occupancy information of the data corresponding to each congestion control transmission type at each time point in the buffer; if there is at least one target congestion control transmission type at the current time point whose target occupancy in the buffer is greater than or equal to the first occupancy threshold, then obtains the data information of the data transmitted at each time point in the current cycle of the current time point, and the buffer occupancy information of each time point; integrates each of the data information and each of the buffer occupancy information to obtain the current cycle integrated data corresponding to the current cycle; obtains the historical cycle integrated data of the target congestion control transmission type, and inputs the current cycle integrated data and the historical cycle integrated data into the data corresponding to the target congestion control transmission type. A quantity prediction model is used to obtain the predicted target data quantity; when the target occupancy is greater than or equal to a second occupancy threshold, the current total occupancy of the buffer and the current data occupancy of data of each congestion control transmission type in the buffer are obtained, and the second occupancy threshold is greater than the first occupancy threshold; based on the target data quantity, the current total occupancy and the current data occupancy, the data in the buffer is managed. Compared with the related art, the buffer management of the router adopts the same management scheme for all algorithms, which causes the data transmitted by different congestion algorithms to preempt each other, resulting in low data transmission efficiency, the embodiment of the present application can comprehensively consider the various data information obtained by acquisition and analysis, manage the data in the buffer scientifically and reasonably, optimize the utilization of buffer resources, avoid the mutual preemption of data transmitted by different congestion control transmission algorithms, and improve the efficiency of data transmission.

[0154] The specific implementation of the above steps can be found in the previous embodiments, which will not be described in detail here.

[0155] In order to better implement the data management method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the above data management method. The meanings of the terms are the same as those in the above data management method, and the specific implementation details can refer to the description in the method embodiment.

[0156] See also Figure 3 , Figure 3 A structural schematic diagram of a data management method device provided in an embodiment of the present application, wherein the data management method device is applied to a computer device serving as a router, wherein the data management method device may include a statistical unit 601, a first acquisition unit 602, an integration unit 603, an input unit 604, a second acquisition unit 605, and a management unit 606, etc.

[0157] A statistical unit 601 is used to periodically receive data corresponding to at least one congestion control transmission type transmitted at each time point, and count the buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point;

[0158] A first acquisition unit 602 is used to acquire data information of data transmitted at each time point in the current cycle at the current time point, and buffer occupancy information at each time point, if at a current time point there is data of at least one target congestion control transmission type whose target occupancy in the buffer is greater than or equal to a first occupancy threshold;

[0159] An integration unit 603 is used to integrate each piece of data information and each piece of buffer occupancy information to obtain current period integration data corresponding to the current period;

[0160] An input unit 604 is used to obtain historical period integrated data of the target congestion control transmission type, and input the current period integrated data and the historical period integrated data into a data volume prediction model corresponding to the target congestion control transmission type to obtain a predicted target data volume;

[0161] A second acquisition unit 605 is configured to acquire a current total occupancy of the buffer and a current data occupancy of data of each congestion control transmission type in the buffer when the target occupancy is greater than or equal to a second occupancy threshold, and the second occupancy threshold is greater than the first occupancy threshold;

[0162] The management unit 606 is used to manage the data in the buffer based on the target data volume, the current total occupancy and the current data occupancy.

[0163] In some embodiments, the management unit 606 includes:

[0164] A first calculation subunit is used to calculate the ratio of the current total occupancy to the current target data occupancy of the data of the target congestion control transmission type in the buffer zone when the target occupancy is greater than or equal to the second occupancy threshold and less than the third occupancy threshold, to obtain an occupancy multiple;

[0165] A first acquisition subunit is used to obtain the target data volume and the sum of the current occupancy of other data of other congestion control transmission types in the buffer zone to obtain a first predicted total occupancy;

[0166] A second calculation subunit is used to calculate the ratio of the target data volume to the first predicted total occupancy to obtain a first predicted occupancy ratio;

[0167] a second acquisition subunit, configured to acquire a product of the occupancy multiple and the first predicted occupancy ratio to obtain a calculation result;

[0168] A comparison subunit, used for comparing the calculation result with one to obtain a comparison result;

[0169] The management subunit is used to manage the data in the buffer according to the comparison result.

[0170] In some embodiments, the management subunit is used to:

[0171] When the comparison result is that the calculation result is less than one, reducing the transmission amount of data of the target congestion control transmission type in the buffer;

[0172] When the comparison result is that the calculation result is greater than one, the transmission volume of data of other congestion control transmission types in the buffer is reduced.

[0173] In some embodiments, the management unit 606 further includes:

[0174] A third acquisition subunit is used to obtain the sum of the target data volume and the current other data occupancy of data of other congestion control transmission types in the buffer zone when the target occupancy is greater than or equal to a third occupancy threshold, to obtain a second predicted total occupancy;

[0175] A third calculation subunit is used to calculate the ratio of the target data volume to the second predicted total occupancy to obtain a second predicted occupancy ratio;

[0176] A fourth calculation subunit is used to calculate the ratio of the current target data occupancy of the data of the target congestion control transmission type in the buffer to the current total occupancy, so as to obtain a current occupancy ratio;

[0177] A first operation subunit is configured to perform a weighted summation of the second predicted occupancy ratio and the current occupancy ratio to obtain an expected occupancy ratio;

[0178] A fourth acquisition subunit, used to acquire the remaining storage capacity of the buffer;

[0179] A determination subunit, configured to determine, based on the expected occupancy ratio and the remaining storage amount, an allowable data amount of each of the congestion control transmission types that is allowed to enter the buffer;

[0180] The discarding subunit is used to discard the data of each congestion control transmission type until the data volume of each congestion control transmission type reaches the corresponding allowed data volume.

[0181] In some embodiments, the determining subunit is used to:

[0182] Determining an allowable occupancy ratio of the remaining storage amount based on the expected occupancy ratio and the second predicted total occupancy;

[0183] Calculate the product of the remaining storage capacity and the allowed occupancy ratio to obtain the allowed data volume for allowing data of the target congestion control transmission type to enter the buffer;

[0184] The difference between the remaining storage capacity and the allowed data volume of the target congestion management transmission type entering the buffer is calculated to obtain the allowed data volume of the other congestion management transmission types entering the buffer.

[0185] In some embodiments, the data management device further includes:

[0186] A third acquisition unit is used to obtain the historical low wave filter queue length of the low wave filter queue corresponding to each queue in the buffer at a previous time point, and the actual length of each queue in the buffer at a current time point;

[0187] A first operation unit is used to perform weighted summation on the historical low-wave filter queue length and the actual length to obtain a current low-wave filter queue length of each low-wave filter queue at a current time point;

[0188] A second operation unit is used to derive the current low-wave filter queue length of each of the low-wave filter queues to obtain a length growth trend;

[0189] A first determining unit, configured to determine a length threshold of each queue based on a length growth trend of each low-wave filter queue and a third occupancy threshold;

[0190] The discarding unit is used to discard data in a queue whose actual length is greater than a corresponding length threshold.

[0191] In some embodiments, the data management device further includes:

[0192] A recording unit, used to record the actual data volume of the data corresponding to the target congestion control transmission type;

[0193] A second determining unit is used to use the current period integrated data and the historical period integrated data as current sample data, and use the actual data volume as a label value of the current sample data;

[0194] A training unit is used to train a data volume prediction model corresponding to the target congestion control transmission type through historical sample data and corresponding label values, the current sample data and corresponding label values.

[0195] The specific implementation of each of the above units can be found in the previous embodiments, which will not be described in detail here.

[0196] As can be seen from the above, the embodiment of the present application periodically receives data corresponding to at least one congestion control transmission type transmitted at each time point through a statistical unit 601, and counts the buffer occupancy information of the data corresponding to each congestion control transmission type at each time point in the buffer; the first acquisition unit 602 obtains data information of the data transmitted at each time point in the current cycle in which the current time point is located, and the buffer occupancy information of each time point if there is data of at least one target congestion control transmission type at the current time point whose target occupancy in the buffer is greater than or equal to the first occupancy threshold; the integration unit 603 is used to integrate each of the data information and each of the buffer occupancy information to obtain the current cycle integrated data corresponding to the current cycle;

[0197] The input unit 604 obtains the historical period integrated data of the target congestion control transmission type, and inputs the current period integrated data and the historical period integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume; the second acquisition unit 605 obtains the current total occupancy of the buffer and the current data occupancy of the data of each congestion control transmission type in the buffer when the target occupancy is greater than or equal to the second occupancy threshold, and the second occupancy threshold is greater than the first occupancy threshold; the management unit 606 manages the data in the buffer based on the target data volume, the current total occupancy and the current data occupancy. Compared with the related art, the buffer management of the router adopts the same set of management schemes for all algorithms, resulting in the data transmitted by different congestion algorithms preempting each other, resulting in low data transmission efficiency. The embodiment of the present application can comprehensively consider the various data information obtained and analyzed, manage the data in the buffer scientifically and reasonably, optimize the buffer resource utilization, avoid the data transmitted by different congestion control transmission algorithms preempting each other, and improve the efficiency of data transmission.

[0198] The specific implementation of each of the above units can be found in the previous embodiments, which will not be described in detail here.

[0199] Reference Figure 4 , Figure 4 The block diagram of the structure of a part of a computer device 1000 as a router for implementing an embodiment of the present disclosure. The computer device 1000 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 622 (for example, one or more processors) and a memory 632, and one or more storage media 630 (for example, one or more mass storage devices) storing application programs 642 or data 644. Among them, the memory 632 and the storage medium 630 may be temporary storage or permanent storage. The program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server 600. Furthermore, the central processing unit 622 may be configured to communicate with the storage medium 630 and execute a series of instruction operations in the storage medium 630 on the server 600.

[0200] The computer device 1000 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input and output interfaces 658, and / or one or more operating systems 641, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0201] The central processor 622 in the computer device 1000 may be used to execute the data management method of the embodiment of the present disclosure, for example:

[0202] Periodically receiving data corresponding to at least one congestion control transmission type transmitted at each time point, and counting buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point;

[0203] If at the current time point there is data of at least one target congestion control transmission type whose target occupancy in the buffer is greater than or equal to the first occupancy threshold, then obtaining data information of the data transmitted at each time point in the current cycle at the current time point, as well as buffer occupancy information at each time point;

[0204] Integrate each piece of data information and each piece of buffer occupancy information to obtain current cycle integrated data corresponding to the current cycle;

[0205] Obtain historical period integrated data of the target congestion control transmission type, and input the current period integrated data and the historical period integrated data into a data volume prediction model corresponding to the target congestion control transmission type to obtain a predicted target data volume;

[0206] When the target occupancy is greater than or equal to a second occupancy threshold, obtaining a current total occupancy of the buffer and a current data occupancy of data of each congestion control transmission type in the buffer, and the second occupancy threshold is greater than the first occupancy threshold;

[0207] The data in the buffer is managed based on the target data volume, the current total occupancy and the current data occupancy.

[0208] The embodiments of the present disclosure further provide a computer-readable storage medium, which is used to store program codes, and the program codes are used to execute the data management methods of the aforementioned embodiments.

[0209] The present disclosure also provides a computer program product, which includes a computer program. The processor of a computer device reads and executes the computer program, so that the computer device executes the above-mentioned data management method. For example:

[0210] Periodically receiving data corresponding to at least one congestion control transmission type transmitted at each time point, and counting buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point;

[0211] If at the current time point there is data of at least one target congestion control transmission type whose target occupancy in the buffer is greater than or equal to the first occupancy threshold, then obtaining data information of the data transmitted at each time point in the current cycle at the current time point, as well as buffer occupancy information at each time point;

[0212] Integrate each piece of data information and each piece of buffer occupancy information to obtain current cycle integrated data corresponding to the current cycle;

[0213] Obtain historical period integrated data of the target congestion control transmission type, and input the current period integrated data and the historical period integrated data into a data volume prediction model corresponding to the target congestion control transmission type to obtain a predicted target data volume;

[0214] When the target occupancy is greater than or equal to a second occupancy threshold, obtaining a current total occupancy of the buffer and a current data occupancy of data of each congestion control transmission type in the buffer, and the second occupancy threshold is greater than the first occupancy threshold;

[0215] The data in the buffer is managed based on the target data volume, the current total occupancy and the current data occupancy.

[0216] In addition, the terms "comprises" and "includes" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, product or apparatus.

[0217] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0218] It should be understood that in the description of the embodiments of the present application, the meaning of multiple (or multiple items) is more than two, greater than, less than, exceed, etc. are understood to not include the number, and above, below, within, etc. are understood to include the number.

[0219] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0220] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0221] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0222] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0223] It should also be understood that the various implementations provided in the embodiments of the present application can be combined arbitrarily to achieve different technical effects.

[0224] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0225] The above is a specific description of the implementation method of the present application, but the present application is not limited to the above-mentioned implementation method. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A data management method, characterized in that: include: Periodically receiving data corresponding to at least one congestion control transmission type transmitted at each time point, and counting buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point; If at the current time point there is data of at least one target congestion control transmission type whose target occupancy in the buffer is greater than or equal to the first occupancy threshold, then obtaining data information of the data transmitted at each time point in the current cycle at the current time point, as well as buffer occupancy information at each time point; Integrate each piece of data information and each piece of buffer occupancy information to obtain current cycle integrated data corresponding to the current cycle; Obtain historical period integrated data of the target congestion control transmission type, and input the current period integrated data and the historical period integrated data into a data volume prediction model corresponding to the target congestion control transmission type to obtain a predicted target data volume; When the target occupancy is greater than or equal to a second occupancy threshold, obtaining a current total occupancy of the buffer and a current data occupancy of data of each congestion control transmission type in the buffer, and the second occupancy threshold is greater than the first occupancy threshold; The data in the buffer is managed based on the target data volume, the current total occupancy and the current data occupancy.

2. The data management method according to claim 1, characterized in that: The managing the data in the buffer based on the target data volume, the current total occupancy and the current data occupancy includes: When the target occupancy is greater than or equal to the second occupancy threshold and less than the third occupancy threshold, calculate the ratio of the current total occupancy to the current target data occupancy of the data of the target congestion control transmission type in the buffer to obtain the occupancy multiple; Obtaining the target data volume and the sum of the current occupancy of other data of other congestion control transmission types in the buffer zone to obtain a first predicted total occupancy; Calculating a ratio of the target data volume to the first predicted total occupancy to obtain a first predicted occupancy ratio; Obtaining a product of the occupancy multiple and the first predicted occupancy ratio to obtain a calculation result; Comparing the calculation result with one to obtain a comparison result; The data in the buffer is managed according to the comparison result.

3. The data management method according to claim 2, characterized in that: The managing the data in the buffer according to the comparison result includes: When the comparison result is that the calculation result is less than one, reducing the transmission amount of data of the target congestion control transmission type in the buffer; When the comparison result is that the calculation result is greater than one, the transmission volume of data of other congestion control transmission types in the buffer is reduced.

4. The data management method according to claim 2, characterized in that: The method further comprises: When the target occupancy is greater than or equal to a third occupancy threshold, obtaining a sum of the target data volume and the current occupancy of other data of other congestion control transmission types in the buffer zone to obtain a second predicted total occupancy; Calculating a ratio of the target data volume to the second predicted total occupancy to obtain a second predicted occupancy ratio; Calculate the ratio of the current target data occupancy of the target congestion control transmission type in the buffer to the current total occupancy to obtain a current occupancy ratio; Taking a weighted sum of the second predicted occupancy ratio and the current occupancy ratio to obtain an expected occupancy ratio; Obtaining the remaining storage capacity of the buffer; Based on the expected occupancy ratio and the remaining storage amount, determining an allowable amount of data for each of the congestion management transmission types to enter the buffer; The data of each of the congestion control transmission types is discarded until the data volume of each of the congestion control transmission types reaches the corresponding allowed data volume.

5. The data management method according to claim 4, characterized in that: The determining, based on the expected occupancy ratio and the remaining storage amount, the amount of data allowed to enter the buffer for each of the congestion control transmission types includes: Determining an allowable occupancy ratio of the remaining storage amount based on the expected occupancy ratio and the second predicted total occupancy; Calculate the product of the remaining storage capacity and the allowed occupancy ratio to obtain the allowed data volume for allowing data of the target congestion control transmission type to enter the buffer; The difference between the remaining storage capacity and the allowed data volume of the target congestion management transmission type entering the buffer is calculated to obtain the allowed data volume of the other congestion management transmission types entering the buffer.

6. The data management method according to claim 1, characterized in that: The method further comprises: Get the historical low wave filter queue length of the low wave filter queue corresponding to each queue in the buffer at the previous time point, and the actual length of each queue in the buffer at the current time point; Performing a weighted summation on the historical low-wave filter queue length and the actual length to obtain a current low-wave filter queue length of each low-wave filter queue at a current time point; Derivatively calculating the current low-wave filter queue length of each low-wave filter queue to obtain a length growth trend; Determine a length threshold of each queue based on a length growth trend of each low-wave filter queue and a third occupancy threshold; The data in the queue whose actual length is greater than the corresponding length threshold is discarded.

7. The data management method according to claim 1, characterized in that: After inputting the current period integrated data and the historical period integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume, the method further includes: Record the actual data volume of the data corresponding to the target congestion control transmission type; The current period integrated data and the historical period integrated data are used as current sample data, and the actual data volume is used as a label value of the current sample data; The data volume prediction model corresponding to the target congestion control transmission type is trained using historical sample data and corresponding label values, the current sample data and corresponding label values.

8. A data management device, characterized in that: include: A statistical unit, used to periodically receive data corresponding to at least one congestion control transmission type transmitted at each time point, and count buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point; A first acquisition unit is used to acquire data information of data transmitted at each time point in the current cycle at the current time point, and buffer occupancy information at each time point, if at a current time point there is data of at least one target congestion control transmission type whose target occupancy in the buffer is greater than or equal to a first occupancy threshold; An integration unit, used to integrate each of the data information and each of the buffer occupancy information to obtain current cycle integrated data corresponding to the current cycle; An input unit, used to obtain historical period integrated data of the target congestion control transmission type, and input the current period integrated data and the historical period integrated data into a data volume prediction model corresponding to the target congestion control transmission type to obtain a predicted target data volume; A second acquisition unit, configured to acquire a current total occupancy of the buffer and a current data occupancy of data of each congestion control transmission type in the buffer when the target occupancy is greater than or equal to a second occupancy threshold, and the second occupancy threshold is greater than the first occupancy threshold; A management unit is used to manage the data in the buffer based on the target data volume, the current total occupancy and the current data occupancy.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the data management method according to any one of claims 1 to 7.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the data management method according to any one of claims 1 to 7 is implemented.

Images