Data management method and device, storage medium and computer device

Abstract

The embodiment of the application provides a data management method, device, storage medium and computer equipment, through periodically receiving congestion control transmission type data and counting buffer occupation information, if the target type data reaches a first occupation threshold in the target occupation of the buffer, the current period related data information is integrated to obtain current period integrated data, and the prediction target data volume is obtained by combining the historical period integrated data and inputting the prediction model; if the target occupation reaches a larger second occupation threshold, the current total occupation of the buffer and the current data occupation of each type are obtained. Finally, the target data volume, the current total occupation and the current data occupation are comprehensively considered to scientifically manage the buffer data, optimize resource utilization, prevent different types of data from occupying each other, improve data transmission efficiency, ensure stable and efficient network operation, and meet the diversified data transmission demand to cope with network congestion challenges.

Description

Technical Field

[0001] This application relates to the field of data management technology, and in particular to a data management method, apparatus, storage medium and computer equipment. Background Technology

[0002] With the diversification of application scenarios, research on network congestion has become increasingly in-depth, leading to the emergence of various types of congestion control algorithms. Currently, traditional congestion management transmission algorithms are mainly divided into three categories: 1. Transmission Control Protocol (TCP) based on packet loss feedback; 2. TCP protocol based on delay feedback; and 3. High-speed bandwidth algorithms based on packet loss feedback. These algorithms differ significantly in their methods of detecting and handling congestion. Taking high-speed algorithms based on packet loss feedback as an example, these algorithms generally consider packet loss to exist after receiving three duplicate acknowledgment packets (ACKs) and perform corresponding congestion control. In contrast, congestion control protocols based on delay feedback detect network congestion by observing changes in packet delay and intervene in advance to reduce the negative impact of packet loss.

[0003] In related technologies, the use of different strategies can lead to unfair competition between algorithms. Research has found that a major reason for this unfairness is that router buffer management uses the same management scheme for all algorithms. When the buffer in the router is set large, packet latency is more susceptible to fluctuations but packet loss is less likely. This results in TCP protocols based on latency feedback lacking a competitive advantage, and available bandwidth is preempted by TCP protocols based on packet loss feedback. Conversely, when the buffer is set small, a burst of traffic can cause a large number of packet losses, while having little impact on the latency of normally transmitted packets. This also leads to TCP protocols based on packet loss feedback lacking a competitive advantage, and available bandwidth is preempted by TCP protocols based on latency feedback. Consequently, data transmission between different congestion algorithms becomes a competition for data, resulting in low data transmission efficiency. Therefore, there is an urgent need to propose a data management method to solve these technical problems. Summary of the Invention

[0004] The main objective of this application is to provide a data management method, apparatus, storage medium, and computer equipment that can comprehensively consider various data information obtained and analyzed, scientifically and rationally manage the data in the buffer, optimize the utilization of buffer resources, avoid data contention between different congestion algorithms, and improve the efficiency of data transmission.

[0005] In a first aspect, embodiments of this application provide a data management method, including:

[0006] Periodically receive data corresponding to at least one congestion control transmission type transmitted at each time point, and count the buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point;

[0007] If at the current time point there is at least one data of the target congestion control transmission type whose target occupancy in the buffer is greater than or equal to the first occupancy threshold, then obtain the data information of the data transmitted at each time point in the current period, as well as the buffer occupancy information at each time point;

[0008] Each piece of data information and each piece of buffer occupancy information are integrated to obtain the current period integrated data corresponding to the current period.

[0009] Obtain historical periodic integrated data of the target congestion control transmission type, and input the current periodic integrated data and the historical periodic integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume;

[0010] When the target occupancy is greater than or equal to the second occupancy threshold, the current total occupancy of the buffer and the current data occupancy of each congestion control transmission type in the buffer are obtained, and the second occupancy threshold is greater than the first occupancy threshold.

[0011] The data in the buffer is managed based on the target data volume, the current total occupancy, and the current data occupancy.

[0012] Secondly, embodiments of this application provide a data management device, including:

[0013] The statistics unit is used to periodically receive data corresponding to at least one congestion control transmission type transmitted at each time point, and to count the buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point.

[0014] The first acquisition unit is used to acquire data information of data transmitted at each time point in the current period and buffer occupancy information at each time point if the target occupancy of at least one data of the target congestion control transmission type in the buffer is greater than or equal to a first occupancy threshold at the current time point.

[0015] An integration unit is used to integrate each of the data information and each of the buffer occupancy information to obtain the current period integrated data corresponding to the current period;

[0016] The input unit is used to acquire historical periodic integrated data of the target congestion control transmission type, and input the current periodic integrated data and the historical periodic integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume.

[0017] The second acquisition unit is used to acquire the current total occupancy of the buffer and the current data occupancy of each congestion control transmission type in the buffer when the target occupancy is greater than or equal to the second occupancy threshold, wherein the second occupancy threshold is greater than the first occupancy threshold.

[0018] The management unit is used to manage the data in the buffer based on the target data volume, the current total occupancy, and the current data occupancy.

[0019] Thirdly, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute any of the above data management methods.

[0020] Fourthly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data management method as described above.

[0021] In this embodiment, data corresponding to at least one congestion control transmission type transmitted at each time point is received periodically, and buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer is statistically analyzed at each time point. If the target occupancy of data of at least one target congestion control transmission type in the buffer is greater than or equal to a first occupancy threshold at the current time point, the data information of the data transmitted at each time point in the current period and the buffer occupancy information of each time point are obtained. Each data information and each buffer occupancy information are integrated to obtain the current period integrated data corresponding to the current period. The historical period integrated data of the target congestion control transmission type is obtained, and the current period integrated data and the historical period integrated data are input into the data volume pre-setup corresponding to the target congestion control transmission type. The model is tested to obtain the predicted target data volume. When the target occupancy is greater than or equal to a second occupancy threshold, the current total occupancy of the buffer and the current data occupancy of each congestion control transmission type in the buffer are obtained, where the second occupancy threshold is greater than the first occupancy threshold. Based on the target data volume, the current total occupancy, and the current data occupancy, the data in the buffer is managed. Compared with related technologies, where router buffer management uses the same management scheme for all algorithms, leading to data contention between different congestion control algorithms and resulting in low data transmission efficiency, the embodiments of this application can comprehensively consider the various data information obtained and analyzed to scientifically and rationally manage the data in the buffer, optimize buffer resource utilization, avoid data contention between different congestion control transmission algorithms, and improve data transmission efficiency.

[0022] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of a data management system provided in an embodiment of this application.

[0025] Figure 2This is a flowchart illustrating the data management method provided in an embodiment of this application.

[0026] Figure 3 This is a schematic diagram of the data management method apparatus provided in the embodiments of this application.

[0027] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the solutions of this application, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] It should be noted that while some processes described in the specification, claims, and accompanying drawings contain multiple steps that appear in a specific order, it should be clearly understood that these steps may not be performed in the order they appear herein, or may be performed in parallel. The step numbers are merely used to distinguish different steps and do not represent any particular order of execution. Furthermore, descriptions such as "first," "second," or "objective" in this document are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0030] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0031] TCP protocol based on packet loss feedback:

[0032] 1. Congestion Awareness: Receiving three identical ACK packets is considered a sign of packet loss and serves as a signal of network congestion. For example, during data transmission, if the sender receives three consecutive ACK packets for the same sequence number, it determines that congestion has occurred in the network.

[0033] 2. Congestion Management: Once congestion is detected, corresponding congestion control measures will be taken, such as reducing the transmission rate. A common practice is to halve the transmission window size, thereby reducing the amount of data entering the network and alleviating congestion.

[0034] For example, if the original sending window size is 100, after congestion is detected, the sending window size is adjusted to 50 to reduce the speed of sending data.

[0035] TCP protocol based on delayed feedback:

[0036] 1. Congestion Awareness: This involves observing changes in data packet latency to detect network congestion. It continuously monitors the time it takes for data packets to travel from send to receive. If a significant increase in latency is detected, it's considered a potential congestion trend. For example, if the normal round-trip time for a data packet is 10ms, a sudden increase to 50ms would raise concerns.

[0037] 2. Congestion Management: Proactive intervention to reduce the negative impact of packet loss. When an increase in latency is detected, the sending rate is adjusted appropriately to prevent further network deterioration and packet loss. A more conservative approach is typically used to reduce the sending rate, rather than aggressively halving the window as is done in TCP protocols based on packet loss feedback.

[0038] For example, depending on the degree of increase in latency, the transmission rate can be reduced by a certain percentage, such as 10%-20%.

[0039] High-speed bandwidth algorithm based on packet loss feedback:

[0040] 1. Congestion Awareness: Similar to the TCP protocol based on packet loss feedback, it also considers packet loss to exist after receiving a certain number (e.g., three) of duplicate ACK packets, thus determining network congestion. This is because packet loss remains an important congestion indicator in high-speed network environments.

[0041] 2. Congestion Management: When handling congestion, optimizations are made based on the characteristics of high-speed networks. For example, the sending rate and window size may be adjusted more quickly to adapt to the data transmission demands under high bandwidth. Simultaneously, more complex algorithms may be employed to calculate new sending rates and window sizes to improve network utilization and transmission efficiency.

[0042] For example, in addition to simply halving the window size, factors such as the network's remaining bandwidth and the number of current connections are also considered to dynamically adjust the window size, so that the network can recover to an efficient transmission state more quickly after congestion.

[0043] However, although terminal devices sending data can proactively intervene in network congestion using different congestion control transmission algorithms to reduce the negative impact of packet loss, once data is transmitted to the router's buffer, the management strategy for each congestion control transmission algorithm type is consistent. When the buffer is set large in the router, packet latency is prone to fluctuations but packet loss is less likely. This results in a lack of competitive advantage for delay-based TCP protocols, with available bandwidth being preempted by loss-based TCP protocols. Conversely, when the buffer is set small, a sudden surge may cause a large number of packet losses, while having little impact on the latency of normally transmitted packets. This also leads to a lack of competitive advantage for loss-based TCP protocols, with available bandwidth being preempted by delay-based TCP protocols, resulting in lower data transmission efficiency.

[0044] The TOS (Type of Service) field is an 8-bit field in the IP datagram header used to specify the type of service (QoS) of the IP packet. Its primary purpose is to communicate packet priority and QoS requirements to network devices (such as routers), helping them to better manage traffic and allocate resources.

[0045] The `rx_trace` function is typically used in the software systems of network devices (such as routers and network interface cards). It is a tracing function related to receiving (rx) data packets. Its main function is to record various information about data packets entering the network device, such as the source of the data packet (source IP address, source MAC address), arrival time, data packet size, and the protocol used, among other details.

[0046] XGBoost (eXtreme Gradient Boosting) is a highly efficient gradient boosting algorithm based on decision trees, widely used in data mining and machine learning, particularly excelling in classification and regression tasks for structured data. It is an optimization and extension of the traditional gradient boosting algorithm, favored for its efficiency, accuracy, and scalability.

[0047] Its working principle includes:

[0048] 1. Gradient Boosting Framework:

[0049] XGBoost belongs to the gradient boosting algorithm family within ensemble learning. Its basic idea is to solve a problem by constructing multiple weak learners (primarily decision trees in XGBoost) and combining them into a single strong learner. Each weak learner attempts to correct the errors of the preceding learners.

[0050] For example, in a regression task, a predicted value is first initialized (e.g., simply taking the mean of the target variable). Then, a first decision tree is built to fit the residual (error) between the target variable and this initial predicted value. After that, the prediction result of this decision tree is added to the initial predicted value to obtain a new predicted value. Then, a new decision tree is built to fit the new residual, and so on.

[0051] 2. Decision tree construction:

[0052] In XGBoost, the decision tree is its core component. The construction of a decision tree is achieved by selecting features and splitting nodes on the training data. It selects the optimal features and split points based on certain splitting criteria (such as the Gini coefficient, information gain, etc.), resulting in higher purity of the child node data after splitting.

[0053] For example, in a classification task to determine whether a user will buy a product, for data containing features such as user age, income, and purchase history, a decision tree might select "income" as the first split point based on information gain, dividing users into high-income and low-income groups. Then, it would continue to select other features in each group for splitting until the stopping condition is met (such as the tree depth reaching a limit or the number of samples in the child nodes being too small).

[0054] 3. Regularization and prevention of overfitting:

[0055] XGBoost employs various regularization techniques to prevent overfitting. These include penalizing the complexity of the decision tree, for example, by controlling parameters such as tree depth and the number of leaf nodes. It also introduces a shrinkage mechanism, which uses only a small subset (usually controlled by the learning rate) of the new decision tree's predictions each time a prediction is updated, making the model's training process more robust.

[0056] For example, when training a model on a complex dataset, without regularization, the model may overfit to the noise in the training data, leading to a decline in performance on new data. XGBoost's regularization mechanism can limit the complexity of the model, enabling it to generalize better to unknown data.

[0057] Low-frequency filter queues are mechanisms used to process network data queues. Their primary purpose is to smooth out fluctuations in queue length data, enabling network devices (such as routers) to manage and control data traffic more effectively. They use specific algorithms and rules to filter raw queue length data, reducing high-frequency fluctuations caused by instantaneous traffic changes, measurement errors, and other factors.

[0058] To address the aforementioned problems, this application embodiment periodically receives data corresponding to at least one congestion control transmission type transmitted at each time point, and statistically analyzes the buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point. If, at the current time point, the target occupancy of data of at least one target congestion control transmission type in the buffer is greater than or equal to a first occupancy threshold, then the data information of the data transmitted at each time point in the current period, as well as the buffer occupancy information of each time point, are obtained. Each data information and each buffer occupancy information are integrated to obtain the current period integrated data corresponding to the current period. Historical period integrated data of the target congestion control transmission type is obtained, and the current period integrated data and the historical period integrated data are input into the data corresponding to the target congestion control transmission type. Based on the data volume prediction model, the predicted target data volume is obtained. When the target data volume is greater than or equal to a second data volume threshold, the current total data volume of the buffer and the current data volume of each congestion control transmission type in the buffer are obtained, where the second data volume threshold is greater than the first data volume threshold. Based on the target data volume, the current total data volume, and the current data volume, the data in the buffer is managed. Compared to related technologies, where router buffer management uses the same management scheme for all algorithms, leading to data contention between different congestion control algorithms and resulting in low data transmission efficiency, this application embodiment can comprehensively consider the various data information obtained and analyzed to scientifically and rationally manage the data in the buffer, optimize buffer resource utilization, avoid data contention between different congestion control transmission algorithms, and improve data transmission efficiency. Please refer to the following specific embodiments for details.

[0059] Please see Figure 1 , Figure 1 This is a schematic diagram of a data management system provided in an embodiment of this application. It includes a terminal device 130, a router 120, a server 110, etc.

[0060] Terminal device 130 includes, but is not limited to, pre-configured personal computers, tablet computers, desktop computers, or other electronic devices with data transmission capabilities. Furthermore, it can be a single device or a collection of multiple devices. Terminal device 130 can communicate with router 120 via wired or wireless means to exchange data.

[0061] Server 110 refers to a computer system capable of transmitting data (such as training data) to terminal device 130. Compared to ordinary terminals, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (such as a virtual machine), or a combination of portions of multiple high-performance computers (such as virtual machines).

[0062] A router 120 is a network device used to connect different networks, capable of forwarding data packets between multiple networks. Its main function is to send data packets from one network to another based on the destination IP address in the data packet.

[0063] For example, in Figure 1 In this embodiment, router 120 can forward data transmitted by terminal device 130 to server 110, enabling server 110 to obtain the data transmitted by terminal device 130. The data management method of this disclosure embodiment can be implemented by router 120.

[0064] It should be noted that, Figure 1 The schematic diagram of the data management system shown is merely an example. The data management system and scenario described in this application embodiment are intended to more clearly illustrate the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided in this application embodiment. As those skilled in the art will know, with the evolution of data management technology and the emergence of new business scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems.

[0065] In this embodiment, the description will be from the perspective of a data management device, which can be integrated into a computer device that has a storage unit and is equipped with a microprocessor and has computing capabilities.

[0066] Please see Figure 2 , Figure 2 A flowchart illustrating the data management method provided in this application embodiment. The data management method includes:

[0067] In step 201, data corresponding to at least one congestion control transmission type transmitted at each time point is received periodically, and the buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point is calculated.

[0068] When a router receives data from a terminal device, it first stores it in an internal buffer, awaiting subsequent transmission. To periodically observe and analyze network data transmission and buffer status, a time period is defined, such as 10 minutes. Congestion control transmission types include at least one of the following: Transmission Control Protocol (TCP) based on packet loss feedback, TCP based on delay feedback, and high-speed bandwidth algorithm based on packet loss feedback. Buffer occupancy information includes, at each time t within the current period, the buffer size for each congestion control transmission type, the total number of data items in the buffer, the total number of data items, the length of the corresponding queue, and the number of queues.

[0069] Specifically, when terminal devices transmit data, they use different congestion control transmission types to manage the data transmission. Therefore, the terminal devices mark the data they manage according to different congestion control transmission types. The specific content of the mark is the corresponding congestion control transmission type. The router obtains the mark corresponding to each data by parsing the data, thereby determining the congestion control transmission type corresponding to each data.

[0070] For example, "0111" is used to identify the congestion control transmission type based on packet loss feedback, and "0101" is used to identify the congestion control transmission type based on delay feedback. Given two data sets, A and B, where data A is controlled by the congestion control transmission type based on packet loss feedback, and data B is controlled by the congestion control transmission type based on delay feedback, the terminal device will write "0111" in the TOS field of the header of data A and "0101" in the TOS field of the header of data B. The router obtains each data set using the rx_trace function and parses the TOS field to confirm the congestion control transmission type used by the terminal device during data transmission. This informs the router that data A and data B are controlled by different congestion control transmission types.

[0071] In this way, the router, through its built-in monitoring module, receives data transmitted from various data sources at each pre-set time point. Simultaneously, using its own statistical functions, it analyzes the buffer occupancy of each type of congestion control transmission received at each time point.

[0072] In step 202, if at the current time point there is at least one data of the target congestion control transmission type whose target occupancy in the buffer is greater than or equal to the first occupancy threshold, then the data information of the data transmitted at each time point in the current period and the buffer occupancy information at each time point are obtained.

[0073] Because data center traffic is high, predicting the data volume for each congestion control transmission type on a regular or real-time basis could easily consume too many computing resources. Therefore, a first occupancy threshold is set for each congestion control transmission type. Data volume prediction will only be performed when the buffer occupancy of the congestion control transmission type exceeds the first occupancy threshold.

[0074] Specifically, at each point in time, the router continuously monitors the data occupancy of each congestion control transmission type in the buffer. Once it detects that at the current point in time, the target occupancy of at least one type of congestion control transmission data in the buffer reaches or exceeds the first occupancy threshold, indicating that the router's target congestion control transmission data occupies a large proportion of the buffer, it will use its storage and query functions to obtain the data information of all data transmitted at each point in time within the current period (such as the total size of the data, the total number of data items, the congestion control transmission type, etc.), as well as the buffer occupancy information corresponding to each point in time (i.e., the length and number of queues of the corresponding type of target congestion control transmission type in the buffer at each point in time, as previously statistically analyzed).

[0075] In step 203, each data information and each buffer occupancy information are integrated to obtain the current period integrated data corresponding to the current period.

[0076] Please refer to Table 1, which shows what data is included in the current period's integrated data.

[0077] Table 1

[0078]

[0079]

[0080] Specifically, the current period's integrated data includes three types of information about the target congestion control transmission type within the current period. The first type is data integration information, which specifically includes the total size and total number of data items for the target congestion control transmission type, as well as the target congestion control transmission type itself, as statistically analyzed in the current period. The second type is buffer usage integration information, which specifically includes the queue length, queue type (i.e., the target congestion control transmission type), average queue length (i.e., the ratio of the sum of queue lengths at each time point to the total number of time points in the current period), buffer occupancy size, and average buffer occupancy size (i.e., the ratio of the sum of buffer occupancy sizes at each time point to the total number of time points in the current period), as statistically analyzed in the current period. The third type is integration time, which is the time it takes to integrate the current period's integrated data.

[0081] In step 204, historical periodic integrated data of the target congestion control transmission type is obtained, and the current periodic integrated data and historical periodic integrated data are input into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume.

[0082] The historical period integration data can be a preset number of periods for the target congestion control transmission type prior to the current period, such as the previous 10 periods; or all periods prior to the current period, without limitation. The purpose of using the current period integration data and the historical period integration data as input data is to demonstrate through experiments that combining the current period integration data and the historical period integration data can effectively improve the accuracy of the prediction results.

[0083] Specifically, each congestion control transmission type corresponds to a data volume prediction model, which is used to predict the expected data volume (i.e., data quantity) of the corresponding congestion control transmission type within a certain future time period. The data volume prediction model is essentially the XGboost algorithm, which is used to predict the target data volume of the target congestion control transmission type within a certain future time period.

[0084] Please refer to Table 2, which shows the data structure of the target data volume predicted by the data volume prediction model.

[0085] Table 2

[0086]

[0087] The data structure for the target data volume includes the data congestion control transmission type, the prediction time, and the predicted specific data volume.

[0088] In some implementations, after inputting the current periodic integrated data and the historical periodic integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume, the method further includes:

[0089] (1) Record the actual data volume corresponding to the target congestion control transmission type;

[0090] (2) The current period integrated data and the historical period integrated data are used as the current sample data, and the actual data volume is used as the label value of the current sample data;

[0091] (3) The data volume prediction model corresponding to the target congestion control transmission type is trained by using historical sample data and corresponding label values, the current sample data and corresponding label values.

[0092] Specifically, after predicting the target data volume using the data volume prediction model corresponding to the target congestion control transmission type, the model can collect current-period sample data by recording the actual data volume corresponding to the target congestion control transmission type. This allows for offline training to further improve the prediction accuracy of the data volume prediction model.

[0093] Specifically, the optimization method for the data volume prediction model can refer to the following formula:

[0094]

[0095] Among them, y i This represents the actual observation value recorded in the i-th period. This represents the amount of data predicted for the i-th period. This represents the average of the actual observed values ​​across all periods. The formula result ranges from 0 to 1. A result of 1 indicates that the predicted result is completely consistent with the observed value, representing the highest goodness of fit.

[0096] In terms of model optimization, the newly acquired statistical information is stored in the corresponding training data file based on the congestion control transmission type and written to it. Then, the model parameters are adjusted using the new training data and the results of quantitative evaluation to achieve model optimization.

[0097] In step 205, when the target occupancy is greater than or equal to the second occupancy threshold, the current total occupancy of the buffer and the current data occupancy of each congestion control transmission type in the buffer are obtained, and the second occupancy threshold is greater than the first occupancy threshold.

[0098] Once the predicted target data volume is obtained, the process enters the buffer management phase. Buffer management compares the current occupancy of each congestion control transmission type in the buffer with the total occupancy of all congestion control transmission types in the buffer to identify which type of traffic is excessively occupying the buffer and imposes a penalty on the excessively occupying traffic to ensure fair data transmission across all data. To avoid unnecessary waste caused by frequent calculations in buffer management, a penalty threshold (i.e., a second occupancy threshold, which is greater than the first occupancy threshold) is set. Only when the buffer occupancy is greater than or equal to the second occupancy threshold will buffer management penalize the dominant traffic to reduce the corresponding amount of data being sent.

[0099] Specifically, when the target occupancy is greater than or equal to the second occupancy threshold, the current total occupancy B of the buffer and the current data occupancy of each congestion control transmission type in the buffer are obtained (including the buffer size B occupied by the packet loss feedback protocol at the current time t). l (t), and the buffer size N occupied by the delay-based feedback protocol at the current time t. d (t);

[0100] In step 206, the data in the buffer is managed based on the target data volume, the current total occupancy, and the current data occupancy.

[0101] In step 204, the predicted target data volume N is obtained. l (t), and step 205 obtains the current total usage B and the buffer size B occupied by the packet loss feedback protocol at the current time t. l (t), and the buffer size N occupied by the delay-based feedback protocol at the current time t. d After (t), based on the above parameters, determine which of the two congestion control transmission types has an advantage in the buffer, causing the other congestion control transmission type to fall into a disadvantage, and then perform corresponding data management.

[0102] In some implementations, managing the data in the buffer based on the target data volume, the current total occupancy, and the current data occupancy includes:

[0103] (1) When the target occupancy is greater than or equal to the second occupancy threshold and less than the third occupancy threshold, calculate the ratio of the current total occupancy to the current target data occupancy of the data of the target congestion control transmission type in the buffer to obtain the occupancy multiple;

[0104] (2) Obtain the sum of the target data volume and the current data occupancy of other congestion control transmission types in the buffer to obtain the first predicted total occupancy;

[0105] (3) Calculate the ratio of the target data volume to the first predicted total occupancy to obtain the first predicted occupancy ratio;

[0106] (4) Obtain the product of the occupancy multiple and the first predicted occupancy ratio to get the calculation result;

[0107] (5) Compare the calculation result with one to obtain the comparison result;

[0108] (6) Based on the comparison results, manage the data in the buffer.

[0109] There is also a drop threshold, namely the third occupancy threshold. When the target occupancy is greater than or equal to the second occupancy threshold and less than the third occupancy threshold, it means that although the target congestion control transmission type occupies a large part of the buffer, it has not reached the level where the data needs to be dropped. Therefore, at this time, it is necessary to determine which congestion control transmission type is dominant in the buffer in order to reduce the amount of data transmitted.

[0110] Specifically, the method for determining the dominant congestion control transmission type can be referenced by the following formula:

[0111]

[0112] in, The current total occupied B and the target congestion control transmission type of the data in the buffer are the current target data occupied B. l The ratio of (t), i.e., the occupancy multiple; N l (t)+N d (t) represents the target data volume N. l (t) The current data occupies N in the buffer along with other congestion control transmission types. d The sum of (t), i.e., the total occupancy of the first prediction; For the target data volume N l (t) and the first prediction total occupancy N l (t)+N d The ratio of (t) to the first predicted occupancy ratio; occupancy multiple. The proportion of the first prediction The product of these is the result of the calculation. By comparing the calculation result with one, a comparison result is obtained, and the data in the buffer is managed according to the comparison result.

[0113] For example, if the current total occupied B is 100 bytes, and the data of the target congestion control transmission type occupies B bytes in the buffer, then the current target data occupies B bytes in the buffer. l If (t) is 80, then the occupancy multiple is 100 / 80 = 1.25; target data volume N l (t) is 30, and other congestion control transmission types occupy N in the buffer. d If (t) is 50, then the total occupancy of the first prediction is 30 + 50 = 80; the occupancy ratio of the first prediction is 30 / 80 = 1.875; the calculation result is 1.25 × 1.875 = 2.34375.

[0114] In some implementations, managing the data in the buffer based on the comparison result includes:

[0115] (1.1) When the comparison result is less than one, reduce the amount of data of the target congestion control transmission type in the buffer.

[0116] (1.2) When the comparison result is greater than the calculation result, reduce the amount of data transmitted in the buffer for other congestion control transmission types.

[0117] If the comparison result is less than one, it means that the data of the current target congestion control transmission type has an advantage in the buffer, while the other congestion control transmission type has a disadvantage, so the transmission amount of the target congestion control transmission type is reduced; if the comparison result is greater than one, it means that the data of the other congestion control transmission type has an advantage in the buffer, while the other target congestion control transmission type has a disadvantage, so the transmission amount of other congestion control transmission types is reduced.

[0118] In some embodiments, the method further includes:

[0119] (1) When the target occupancy is greater than or equal to the third occupancy threshold, the sum of the target data volume and the current occupancy of other data of other congestion control transmission types in the buffer is obtained to obtain the second predicted total occupancy;

[0120] (2) Calculate the ratio of the target data volume to the second predicted total occupancy to obtain the second predicted occupancy ratio;

[0121] (3) Calculate the ratio of the current target data occupancy in the buffer to the current total occupancy of the data of the target congestion control transmission type, and obtain the current occupancy ratio;

[0122] (4) The second predicted occupancy ratio and the current occupancy ratio are weighted and summed to obtain the expected occupancy ratio;

[0123] (5) Obtain the remaining storage capacity of the buffer;

[0124] (6) Based on the expected occupancy ratio and the remaining storage capacity, determine the amount of data allowed to enter the buffer for each congestion control transmission type;

[0125] (7) Discard the data for each congestion control transmission type until the data volume of each congestion control transmission type reaches the corresponding allowed data volume.

[0126] If the target occupancy is greater than or equal to the third occupancy threshold, it means that the target congestion control transmission type occupies a large portion of the buffer, and some of the data about to be transmitted to the buffer needs to be discarded. The specific amount of data to be discarded needs to be calculated.

[0127] Specifically, before calculating the specific data that needs to be discarded, it is necessary to calculate the expected occupancy ratio E. b The specific calculation method can be found in the following formula:

[0128]

[0129] in, This is a custom parameter used to set the expected impact of the buffer on the proportion of data in the target congestion control transmission type; the second predicted total occupancy N l (t)+N d (t) represents the target data volume N. l (t) The current data occupies N in the buffer along with other congestion control transmission types. d The sum of (t); the second predicted occupancy ratio That is, the target data volume N l (t) and the second predicted total occupancy N l (t)+N d The ratio of (t); the current occupancy rate That is, the current target data occupies B in the buffer for data of the target congestion control transmission type. l (t) is the ratio of the current total occupancy B. After obtaining the second predicted occupancy ratio and the current occupancy ratio, it is combined with... The desired occupancy ratio E is obtained by weighted summation of the two parameters. b .

[0130] Specifically, after calculating the expected occupancy ratio E bThen, based on the current remaining storage capacity of the buffer, the allowed amount of data for each congestion control transmission type to enter the buffer is determined; data for each congestion control transmission type is discarded until the amount of data for each congestion control transmission type reaches the corresponding allowed amount.

[0131] In some implementations, determining the allowable amount of data for each congestion-controlled transmission type to enter the buffer based on the expected occupancy ratio and the remaining storage capacity includes:

[0132] (1.1) Based on the expected occupancy ratio and the second predicted total occupancy, determine the allowable occupancy ratio of the remaining storage;

[0133] (1.2) Calculate the product of the remaining storage amount and the allowed occupancy ratio to obtain the amount of data allowed to enter the buffer for the target congestion control transmission type;

[0134] (1.3) Calculate the difference between the remaining storage amount and the allowable data amount of the target congestion control transmission type entering the buffer to obtain the allowable data amount of the other congestion control transmission types entering the buffer.

[0135] The calculation method for the allowable data amount of the target congestion control transmission type entering the buffer can refer to the following formula:

[0136]

[0137] Specifically, D l (t) represents the amount of data allowed to enter the buffer at time t for the target congestion control transmission type; C represents the remaining storage capacity; by substituting the parameters into the formula, the amount of data allowed to enter the buffer for the target congestion control transmission type can be obtained. This refers to the allowed occupancy ratio.

[0138] The calculation method for the allowed amount of data entering the buffer for other congestion control transmission types can refer to the following formula:

[0139] D d (t)=CD l (t);

[0140] The allowed data volume D of the target congestion control transmission type is calculated to enter the buffer at time t. l After (t), the amount of data allowed to enter the buffer for other congestion-controlled transmission types is the remaining storage C minus D. l (t), which is the remaining storage C excluding D.l The amount of data other than (t) used to store other congestion control transmission types entering the buffer, i.e., the allowed amount of data.

[0141] In some embodiments, the method further includes:

[0142] (1) Obtain the historical low-wave filter queue length of each queue in the buffer corresponding to the low-wave filter queue in the previous time point, and the actual length of each queue in the buffer in the current time point;

[0143] (2) The historical low-wave filter queue length and the actual length are weighted and summed to obtain the current low-wave filter queue length of each low-wave filter queue at the current time point;

[0144] (3) Differentiate the current low-wave filter queue length of each of the low-wave filter queues to obtain the length growth trend;

[0145] (4) Determine the length threshold of each queue based on the length growth trend of each low-wave filter queue and the third occupancy threshold;

[0146] (5) Discard data in queues whose actual length is greater than the corresponding length threshold.

[0147] In data center networks, bursts of data frequently occur, quickly filling router buffers but also being emptied just as fast. To handle these bursts and avoid unnecessary packet loss, router buffer management typically reserves some space for them. Traditional methods usually determine whether a burst is occurring by obtaining the real-time queue length and calculating its gradient. However, in practice, obtaining accurate real-time queue lengths is difficult. Directly using the real-time queue length to calculate the gradient to determine if a queue is receiving a burst is prone to misjudgment. Therefore, a low-beam filtering approach using the gradient of the queue length is employed for burst control.

[0148] Specifically, the current low-wavelength filter queue length can be calculated using the following formula:

[0149]

[0150] in, Let t be the length of the i-th low-frequency filter queue with priority p. Then, at time t-1, the length of the i-th low-wave filter queue with priority p is also the historical low-wave filter queue length of the corresponding low-wave filter queue at the previous time point. Let represent the actual length of the i-th queue with priority p at time t; γ is a pre-set fixed parameter. The current low-wave filter queue length at the current time point is obtained by weighted summation of the historical low-wave filter queue lengths and the actual lengths.

[0151] By differentiating the current low-wave filter queue length for each queue, the length growth trend is obtained, expressed as follows:

[0152] Wherein, the third occupancy threshold (i.e., the discard threshold) is denoted by T(t), then the queue length threshold is... K is a fixed parameter. If the actual length of the queue is less than or equal to the corresponding length threshold, no data will be discarded; if the actual length of the queue is greater than the corresponding length threshold, the data in it will be discarded.

[0153] As described above, this application embodiment periodically receives data corresponding to at least one congestion control transmission type transmitted at each time point, and statistically analyzes the buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point. If at the current time point, the target occupancy of data of at least one target congestion control transmission type in the buffer is greater than or equal to a first occupancy threshold, then the data information of the data transmitted at each time point in the current period, and the buffer occupancy information of each time point are obtained. Each data information and each buffer occupancy information are integrated to obtain the current period integrated data corresponding to the current period. The historical period integrated data of the target congestion control transmission type is obtained, and the current period integrated data and the historical period integrated data are input into the data corresponding to the target congestion control transmission type. The system uses a volume prediction model to obtain the predicted target data volume. When the target data volume is greater than or equal to a second data occupancy threshold, it obtains the current total occupancy of the buffer and the current data occupancy of each congestion control transmission type in the buffer, where the second data occupancy threshold is greater than the first data occupancy threshold. Based on the target data volume, the current total occupancy, and the current data occupancy, the system manages the data in the buffer. Compared to related technologies where router buffer management uses the same management scheme for all algorithms, leading to data contention between different congestion control algorithms and resulting in low data transmission efficiency, this application embodiment comprehensively considers the various data information obtained and analyzed to scientifically and rationally manage the data in the buffer, optimize buffer resource utilization, avoid data contention between different congestion control transmission algorithms, and improve data transmission efficiency.

[0154] For details on the implementation of each of the above steps, please refer to the previous examples, which will not be repeated here.

[0155] To facilitate better implementation of the data management method provided in the embodiments of this application, the embodiments of this application also provide an apparatus based on the above-described data management method. The meanings of the terms used are the same as in the data management method described above, and specific implementation details can be found in the descriptions in the method embodiments.

[0156] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a data management method apparatus provided in an embodiment of this application. The data management method apparatus is applied to a computer device as a router. The data management method apparatus may include a statistics unit 601, a first acquisition unit 602, an integration unit 603, an input unit 604, a second acquisition unit 605, and a management unit 606, etc.

[0157] The statistics unit 601 is used to periodically receive data corresponding to at least one congestion control transmission type transmitted at each time point, and to count the buffer occupancy information of the data corresponding to each congestion control transmission type at each time point in the buffer.

[0158] The first acquisition unit 602 is used to acquire data information of data transmitted at each time point in the current period and buffer occupancy information at each time point if the target occupancy of at least one data of the target congestion control transmission type in the buffer is greater than or equal to a first occupancy threshold at the current time point.

[0159] Integration unit 603 is used to integrate each of the data information and each of the buffer occupancy information to obtain the current period integrated data corresponding to the current period;

[0160] Input unit 604 is used to acquire historical periodic integrated data of the target congestion control transmission type, and input the current periodic integrated data and the historical periodic integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume;

[0161] The second acquisition unit 605 is used to acquire the current total occupancy of the buffer and the current data occupancy of each congestion control transmission type in the buffer when the target occupancy is greater than or equal to the second occupancy threshold, wherein the second occupancy threshold is greater than the first occupancy threshold.

[0162] The management unit 606 is used to manage the data in the buffer based on the target data volume, the current total occupancy, and the current data occupancy.

[0163] In some embodiments, the management unit 606 includes:

[0164] The first calculation subunit is used to calculate the ratio of the current total occupancy to the current target data occupancy in the buffer of the target congestion control transmission type when the target occupancy is greater than or equal to the second occupancy threshold and less than the third occupancy threshold, so as to obtain the occupancy multiple.

[0165] The first acquisition subunit is used to acquire the sum of the target data volume and the current data occupancy of other congestion control transmission types in the buffer to obtain the first predicted total occupancy.

[0166] The second calculation subunit is used to calculate the ratio of the target data volume to the first predicted total occupancy to obtain the first predicted occupancy ratio;

[0167] The second acquisition subunit is used to acquire the product of the occupancy multiple and the first predicted occupancy ratio to obtain the calculation result;

[0168] A comparison subunit is used to compare the calculation result with one to obtain a comparison result;

[0169] The management subunit is used to manage the data in the buffer based on the comparison result.

[0170] In some embodiments, the management subunit is configured to:

[0171] When the comparison result is less than one, the amount of data of the target congestion control transmission type in the buffer is reduced.

[0172] When the comparison result is greater than the calculation result, the amount of data transmitted in the other congestion control transmission types in the buffer is reduced.

[0173] In some embodiments, the management unit 606 further includes:

[0174] The third acquisition subunit is used to acquire the sum of the target data volume and the current data occupancy of other congestion control transmission types in the buffer when the target occupancy is greater than or equal to the third occupancy threshold, and obtain the second predicted total occupancy.

[0175] The third calculation subunit is used to calculate the ratio of the target data volume to the second predicted total occupancy to obtain the second predicted occupancy ratio;

[0176] The fourth calculation subunit is used to calculate the ratio of the current target data occupancy in the buffer to the current total occupancy of the data of the target congestion control transmission type, so as to obtain the current occupancy ratio;

[0177] The first calculation subunit is used to perform a weighted summation of the second predicted occupancy ratio and the current occupancy ratio to obtain the expected occupancy ratio;

[0178] The fourth acquisition subunit is used to acquire the remaining storage amount of the buffer;

[0179] A sub-unit is defined for determining, based on the expected occupancy ratio and the remaining storage capacity, the amount of data allowed to enter the buffer for each congestion control transmission type.

[0180] The discard subunit is used to discard data for each of the congestion control transmission types until the data volume of each congestion control transmission type reaches the corresponding allowed data volume.

[0181] In some embodiments, the determining subunit is configured to:

[0182] Based on the expected occupancy ratio and the second predicted total occupancy, the allowable occupancy ratio of the remaining storage is determined;

[0183] Calculate the product of the remaining storage amount and the allowed occupancy ratio to obtain the amount of data allowed to enter the buffer for the target congestion control transmission type;

[0184] Calculate the difference between the remaining storage amount and the allowed amount of data for the target congestion control transmission type entering the buffer, to obtain the allowed amount of data for the other congestion control transmission types entering the buffer.

[0185] In some embodiments, the data management device further includes:

[0186] The third acquisition unit is used to acquire the historical low-wave filter queue length of each queue in the buffer corresponding to the low-wave filter queue in the previous time point, and the actual length of each queue in the buffer in the current time point.

[0187] The first calculation unit is used to perform a weighted summation of the historical low-wave filter queue length and the actual length to obtain the current low-wave filter queue length of each low-wave filter queue at the current time point.

[0188] The second calculation unit is used to differentiate the current low-wave filter queue length of each of the low-wave filter queues to obtain the length growth trend;

[0189] The first determining unit is used to determine the length threshold of each queue based on the length growth trend of each low-wave filter queue and the third occupancy threshold.

[0190] The discard unit is used to discard data in a queue whose actual length exceeds the corresponding length threshold.

[0191] In some embodiments, the data management device further includes:

[0192] The recording unit is used to record the actual amount of data corresponding to the target congestion control transmission type;

[0193] The second determining unit is used to take the current period integrated data and the historical period integrated data as the current sample data, and to take the actual data volume as the label value of the current sample data.

[0194] The training unit is used to train the data volume prediction model corresponding to the target congestion control transmission type using historical sample data and corresponding label values, and the current sample data and corresponding label values.

[0195] The specific implementation of each of the above units can be found in the previous embodiments, and will not be repeated here.

[0196] As described above, in this embodiment, the statistics unit 601 periodically receives data corresponding to at least one congestion control transmission type transmitted at each time point, and counts the buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point; if the target occupancy of at least one target congestion control transmission type in the buffer is greater than or equal to a first occupancy threshold at the current time point, the first acquisition unit 602 acquires the data information of the data transmitted at each time point in the current period, as well as the buffer occupancy information of each time point; the integration unit 603 is used to integrate each data information and each buffer occupancy information to obtain the current period integrated data corresponding to the current period.

[0197] Input unit 604 acquires historical periodic integrated data of the target congestion control transmission type, and inputs the current periodic integrated data and the historical periodic integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume; second acquisition unit 605 acquires the current total occupancy of the buffer and the current data occupancy of each congestion control transmission type in the buffer when the target occupancy is greater than or equal to a second occupancy threshold, wherein the second occupancy threshold is greater than the first occupancy threshold; management unit 606 manages the data in the buffer based on the target data volume, the current total occupancy, and the current data occupancy. Compared with related technologies, where router buffer management uses the same management scheme for all algorithms, leading to data contention between different congestion control algorithms and resulting in low data transmission efficiency, this application embodiment can comprehensively consider the acquired and analyzed data information to scientifically and rationally manage the data in the buffer, optimize buffer resource utilization, avoid data contention between different congestion control transmission algorithms, and improve data transmission efficiency.

[0198] The specific implementation of each of the above units can be found in the previous embodiments, and will not be repeated here.

[0199] Reference Figure 4 , Figure 4 This is a partial structural block diagram of a computer device 1000 serving as a router to implement embodiments of the present disclosure. The computer device 1000 can vary considerably due to different configurations or performance characteristics, and may include one or more central processing units (CPUs) 622 (e.g., one or more processors) and memory 632, and one or more storage media 630 (e.g., one or more mass storage devices) storing application programs 642 or data 644. The memory 632 and storage media 630 may be temporary or persistent storage. The program stored in the storage media 630 may include one or more modules (not shown in the figure), each module including a series of instruction operations on the server 600. Furthermore, the CPU 622 may be configured to communicate with the storage media 630 and execute the series of instruction operations in the storage media 630 on the server 600.

[0200] Computer device 1000 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input / output interfaces 658, and / or one or more operating systems 641, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0201] The central processing unit 622 in the computer device 1000 can be used to execute the data management method of the embodiments of this disclosure, for example:

[0202] Periodically receive data corresponding to at least one congestion control transmission type transmitted at each time point, and count the buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point;

[0203] If at the current time point there is at least one data of the target congestion control transmission type whose target occupancy in the buffer is greater than or equal to the first occupancy threshold, then obtain the data information of the data transmitted at each time point in the current period, as well as the buffer occupancy information at each time point;

[0204] Each piece of data information and each piece of buffer occupancy information are integrated to obtain the current period integrated data corresponding to the current period.

[0205] Obtain historical periodic integrated data of the target congestion control transmission type, and input the current periodic integrated data and the historical periodic integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume;

[0206] When the target occupancy is greater than or equal to the second occupancy threshold, the current total occupancy of the buffer and the current data occupancy of each congestion control transmission type in the buffer are obtained, and the second occupancy threshold is greater than the first occupancy threshold.

[0207] The data in the buffer is managed based on the target data volume, the current total occupancy, and the current data occupancy.

[0208] This disclosure also provides a computer-readable storage medium for storing program code for executing the data management methods of the foregoing embodiments.

[0209] This disclosure also provides a computer program product comprising a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the data management method described above. For example:

[0210] Periodically receive data corresponding to at least one congestion control transmission type transmitted at each time point, and count the buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point;

[0211] If at the current time point there is at least one data of the target congestion control transmission type whose target occupancy in the buffer is greater than or equal to the first occupancy threshold, then obtain the data information of the data transmitted at each time point in the current period, as well as the buffer occupancy information at each time point;

[0212] Each piece of data information and each piece of buffer occupancy information are integrated to obtain the current period integrated data corresponding to the current period.

[0213] Obtain historical periodic integrated data of the target congestion control transmission type, and input the current periodic integrated data and the historical periodic integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume;

[0214] When the target occupancy is greater than or equal to the second occupancy threshold, the current total occupancy of the buffer and the current data occupancy of each congestion control transmission type in the buffer are obtained, and the second occupancy threshold is greater than the first occupancy threshold.

[0215] The data in the buffer is managed based on the target data volume, the current total occupancy, and the current data occupancy.

[0216] Furthermore, the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0217] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0218] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0219] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0220] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0221] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0222] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0223] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0224] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0225] The above is a detailed description of the embodiments of this application. However, this application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A data management method, characterized in that, include: Periodically receive data corresponding to at least one congestion control transmission type transmitted at each time point, and count the buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point; If at the current time point there is at least one data of the target congestion control transmission type whose target occupancy in the buffer is greater than or equal to the first occupancy threshold, then obtain the data information of the data transmitted at each time point in the current period, as well as the buffer occupancy information at each time point; Each piece of data information and each piece of buffer occupancy information are integrated to obtain the current period integrated data corresponding to the current period. Obtain historical periodic integrated data of the target congestion control transmission type, and input the current periodic integrated data and the historical periodic integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume; When the target occupancy is greater than or equal to the second occupancy threshold, the current total occupancy of the buffer and the current data occupancy of each congestion control transmission type in the buffer are obtained, and the second occupancy threshold is greater than the first occupancy threshold. The data in the buffer is managed based on the target data volume, the current total occupancy, and the current data occupancy.

2. The data management method according to claim 1, characterized in that, The management of data in the buffer based on the target data volume, the current total occupancy, and the current data occupancy includes: When the target occupancy is greater than or equal to the second occupancy threshold and less than the third occupancy threshold, the ratio of the current total occupancy to the current target data occupancy in the buffer for the data of the target congestion control transmission type is calculated to obtain the occupancy multiple. The sum of the target data volume and the current data occupancy of other congestion control transmission types in the buffer is obtained to get the first predicted total occupancy; Calculate the ratio of the target data volume to the first predicted total occupancy to obtain the first predicted occupancy ratio; Obtain the product of the occupancy multiple and the first predicted occupancy ratio to get the calculation result; The calculation result is compared with one to obtain the comparison result; Based on the comparison results, the data in the buffer is managed.

3. The data management method according to claim 2, characterized in that, The step of managing the data in the buffer based on the comparison result includes: When the comparison result is less than one, the amount of data of the target congestion control transmission type in the buffer is reduced. When the comparison result is greater than the calculation result, the amount of data transmitted in the other congestion control transmission types in the buffer is reduced.

4. The data management method according to claim 2, characterized in that, The method further includes: When the target occupancy is greater than or equal to the third occupancy threshold, the sum of the target data volume and the current occupancy of other data of other congestion control transmission types in the buffer is obtained to obtain the second predicted total occupancy; Calculate the ratio of the target data volume to the second predicted total occupancy to obtain the second predicted occupancy ratio; Calculate the ratio of the current target data occupancy in the buffer to the current total occupancy for the data of the target congestion control transmission type, and obtain the current occupancy ratio; The expected occupancy ratio is obtained by weighted summing the second predicted occupancy ratio and the current occupancy ratio. Obtain the remaining storage amount of the buffer; Based on the expected occupancy ratio and the remaining storage capacity, determine the amount of data allowed to enter the buffer for each congestion control transmission type; Data for each congestion control transmission type is discarded until the data volume for each congestion control transmission type reaches the corresponding allowed data volume.

5. The data management method according to claim 4, characterized in that, The step of determining the allowable data amount for each congestion control transmission type to enter the buffer based on the expected occupancy ratio and the remaining storage amount includes: Based on the expected occupancy ratio and the second predicted total occupancy, the allowable occupancy ratio of the remaining storage is determined; Calculate the product of the remaining storage amount and the allowed occupancy ratio to obtain the amount of data allowed to enter the buffer for the target congestion control transmission type; Calculate the difference between the remaining storage amount and the allowed amount of data for the target congestion control transmission type entering the buffer, to obtain the allowed amount of data for the other congestion control transmission types entering the buffer.

6. The data management method according to claim 1, characterized in that, The method further includes: Get the historical low-wave filter queue length of each queue in the buffer at the previous time point, and the actual length of each queue in the buffer at the current time point; The weighted sum of the historical low-wave filter queue length and the actual length is used to obtain the current low-wave filter queue length of each low-wave filter queue at the current time point. The derivative of the current low-wavelength filter queue length for each of the aforementioned low-wavelength filter queues is used to obtain the length growth trend; The length threshold of each queue is determined based on the length growth trend of each low-wave filter queue and the third occupancy threshold; Data in queues whose actual length exceeds the corresponding length threshold is discarded.

7. The data management method according to claim 1, characterized in that, After inputting the current periodic integrated data and the historical periodic integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume, the method further includes: Record the actual amount of data corresponding to the target congestion control transmission type; The current period integrated data and the historical period integrated data are used as the current sample data, and the actual data volume is used as the label value of the current sample data; The data volume prediction model corresponding to the target congestion control transmission type is trained using historical sample data and corresponding label values, as well as the current sample data and corresponding label values.

8. A data management device, characterized in that, include: The statistics unit is used to periodically receive data corresponding to at least one congestion control transmission type transmitted at each time point, and to count the buffer occupancy information of the data corresponding to each congestion control transmission type in the buffer at each time point. The first acquisition unit is used to acquire data information of data transmitted at each time point in the current period and buffer occupancy information at each time point if the target occupancy of at least one data of the target congestion control transmission type in the buffer is greater than or equal to a first occupancy threshold at the current time point. An integration unit is used to integrate each of the data information and each of the buffer occupancy information to obtain the current period integrated data corresponding to the current period; The input unit is used to acquire historical periodic integrated data of the target congestion control transmission type, and input the current periodic integrated data and the historical periodic integrated data into the data volume prediction model corresponding to the target congestion control transmission type to obtain the predicted target data volume. The second acquisition unit is used to acquire the current total occupancy of the buffer and the current data occupancy of each congestion control transmission type in the buffer when the target occupancy is greater than or equal to the second occupancy threshold, wherein the second occupancy threshold is greater than the first occupancy threshold. The management unit is used to manage the data in the buffer based on the target data volume, the current total occupancy, and the current data occupancy.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the data management method according to any one of claims 1 to 7.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the data management method according to any one of claims 1 to 7.

Images