Data compensation method and device, storage medium and program product

Abstract

The embodiment of the invention provides a data compensation method and device, a storage medium and a program product, and relates to the technical field of data storage, and the method comprises the steps: obtaining a first feature set of a data packet transmitted in a first time period, the first feature set being used for indicating attribute information of the data packet transmitted in the first time period; inputting the first feature set into a prediction model to obtain a first packet loss rate, the first packet loss rate being used for indicating the packet loss rate of the data packet in the second time period; and determining a target data compensation strategy corresponding to the second time period based on the first packet loss rate and a preset packet loss rate. Through adoption of the scheme, the technical problem of low efficiency of the current network transmission method in the related technology is solved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of data transmission, and in particular, to a data compensation method and device, a storage medium, and a program product. Background Art

[0002] During network transmission, data packets may be lost from the sender to the receiver, which is called network packet loss. Network packet loss affects the performance of network communication. When the packet loss rate is high, it may lead to increased overall latency, decreased throughput, freezes, intermittent connections, decreased application performance, and impaired user experience.

[0003] Therefore, in the case of network packet loss, packet loss compensation technology is needed to resend lost data packets or recover data packets to improve network stability and service quality. However, in the related art, a single network compensation method is usually used, which has poor compensation effect in the face of a changing network environment.

[0004] In view of the technical problem of low efficiency of current network transmission methods in related technologies, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a data compensation method and device, a storage medium, and a program product to at least solve the technical problem of low efficiency of current network transmission methods in the related art.

[0006] According to one embodiment of the present application, a data compensation method is provided, including: obtaining a first feature set of data packets transmitted in a first time period, wherein the first feature set is used to indicate attribute information of the data packets transmitted in the first time period; inputting the first feature set into a prediction model to obtain a first packet loss rate, wherein the first packet loss rate is used to indicate the packet loss rate of data packets in a second time period; and determining a target data compensation strategy corresponding to the second time period based on the first packet loss rate and a preset packet loss rate.

[0007] In an exemplary embodiment, the above-mentioned determination of the data compensation strategy corresponding to the second time period based on the first packet loss rate and the preset packet loss rate includes: when the first packet loss rate is greater than the preset packet loss rate and the data compensation strategy corresponding to the current moment is the first data compensation strategy, the first data compensation strategy is adjusted to the second data compensation strategy, and the second data compensation strategy is determined as the target data compensation strategy; when the first packet loss rate is less than the preset packet loss rate and the data compensation strategy corresponding to the current moment is the second data compensation strategy, the second data compensation strategy is adjusted to the first data compensation strategy, and the first data compensation strategy is determined as the target data compensation strategy.

[0008] In an exemplary embodiment, the above-mentioned determination of the data compensation strategy corresponding to the second time period based on the first packet loss rate and the preset packet loss rate includes: updating the retransmission timeout time of the first data compensation strategy based on the preset time and the first packet loss rate; updating the number of retransmissions of the first data compensation strategy based on the preset number of times and the first packet loss rate.

[0009] In an exemplary embodiment, the determining of the data compensation strategy corresponding to the second time period based on the first packet loss rate and the preset packet loss rate includes: updating the redundancy of the second data compensation strategy based on the first packet loss rate.

[0010] In an exemplary embodiment, the above-mentioned obtaining the first feature set of the data packets transmitted in the first time period includes at least one of the following: obtaining a first information set of the data packets transmitted in the first time period, and processing the first information set to obtain the first feature set; determining the mean, variance and standard deviation of the data packets transmitted in the first time period based on the first information set, and determining the mean, variance and standard deviation as the first feature set; determining the packet loss rate in the first time period, and determining the packet loss rate in the first time period as the first feature set.

[0011] In an exemplary embodiment, the above-mentioned obtaining the first information set of the data packets transmitted in the first time period includes: determining the first information set of the data packets transmitted in the first time period at the network interface; storing the first information set in a log file; and obtaining the first information set from the log file.

[0012] In an exemplary embodiment, the above-mentioned inputting the first feature set into the prediction model to obtain the first packet loss rate includes: inputting the first feature set into the input layer of the prediction model; obtaining the second feature set through the hidden layer of the prediction model; performing a nonlinear transformation on the second feature set to obtain the first packet loss rate.

[0013] In an exemplary embodiment, the training process of the above prediction model includes: determining a target parameter set of the prediction model; determining a prediction model to be trained based on the target parameter set, and training the prediction model to be trained to obtain a prediction model.

[0014] In an exemplary embodiment, the above-mentioned determination of the target parameter set of the prediction model includes: obtaining an initial parameter set of the prediction model; training to obtain an initial prediction model corresponding to the initial parameter set, and determining the fitness of the initial prediction model; iterating the initial parameter set p times: in the case of the first iteration, determining the initial parameter set as the first parameter set, and using the first parameter set to update to obtain the second parameter set; training to obtain a prediction model corresponding to the second parameter set, and determining the fitness of the prediction model corresponding to the second parameter set; determining the parameter set corresponding to the larger value of the fitness of the initial prediction model and the fitness of the prediction model corresponding to the second parameter set as the maximum value; The optimal parameter set is determined; in the case of the qth iteration, the q+1th parameter set is determined based on the optimal parameter set and the qth parameter set, and the fitness of the prediction model corresponding to the q+1th parameter set is determined; the parameter set corresponding to the larger value of the fitness of the initial prediction model and the fitness of the prediction models corresponding to the 2nd parameter set to the q+1th parameter set is updated as the optimal parameter set; when the fitness of the prediction model corresponding to the tth parameter set is greater than the preset fitness threshold or q is equal to p, the optimal parameter set is determined as the target parameter set, wherein p is an integer greater than 0, q is an integer greater than 0 and less than or equal to p, and t is an integer greater than 0 and less than or equal to p.

[0015] According to another embodiment of the present application, a data compensation device is provided, including: a first acquisition module, used to acquire a first feature set of data packets transmitted in a first time period, wherein the first feature set is used to indicate attribute information of the data packets transmitted in the first time period; a prediction module, used to input the first feature set into a prediction model to obtain a first packet loss rate, wherein the first packet loss rate is used to indicate the packet loss rate of data packets in a second time period; and a strategy determination module, used to determine a target data compensation strategy corresponding to the second time period based on the first packet loss rate and a preset packet loss rate.

[0016] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0017] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0018] Through this application, a first feature set of data packets transmitted in a first time period is obtained, wherein the first feature set is used to indicate the attribute information of the data packets transmitted in the first time period; the first feature set is input into a prediction model to obtain a first packet loss rate, wherein the first packet loss rate is used to indicate the packet loss rate of data packets in a second time period; based on the first packet loss rate and the preset packet loss rate, a target data compensation strategy corresponding to the second time period is determined. The packet loss rate of the second time period can be predicted based on the characteristics of the data packets monitored in real time, and a data compensation strategy can be determined based on the packet loss rate to adapt to different network environments and ensure high efficiency and reliability of data transmission. The technical problem of low efficiency of current network transmission methods in related technologies is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a hardware structure block diagram of an optional data compensation method according to an embodiment of the present application;

[0020] Figure 2 is a flow chart of an optional data compensation method according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of an optional data compensation method according to an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of another optional data compensation method according to an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of another optional data compensation method according to an embodiment of the present application;

[0024] Figure 6 It is a structural block diagram of an optional data compensation device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] According to one aspect of an embodiment of the present application, a data compensation method is provided. As an optional implementation, the data compensation method can be applied to, but is not limited to, Figure 1 The data compensation system in the hardware environment shown in the figure. The data compensation system may include but is not limited to a terminal device 102, a network 110, a server 112, and a database 114. The terminal device 102 runs a target client. The terminal device 102 includes a display 108, a processor 106, and a memory 104. The display 108 can be used to provide a human-computer interaction interface to receive human-computer interaction operations. The processor is used to generate an interaction instruction in response to the human-computer interaction operation and send the interaction instruction to the server, and the memory is used to store data.

[0028] In addition, the server 112 includes a processing engine 116, which is used to perform storage or reading operations on the database 114. Specifically, the processing engine 116 reads data and data features from the database 114.

[0029] Assumptions Figure 1 The terminal device 102 in the embodiment runs a client for data interaction. The specific process of this embodiment is as follows: In step S102, the terminal device 102 can transmit data packets to the server 112 through the network 110. Then, step S104 is executed to obtain a first feature set of data packets transmitted in a first period of time, and the server 112 executes step S106 to input the first feature set into the prediction model to obtain a first packet loss rate. And step S108 is executed to determine the target data compensation strategy.

[0030] Optionally, in this embodiment, the terminal device 102 may be a terminal device configured with a target client, which may include but is not limited to at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop, a tablet computer, a PDA, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV, etc. The target client may be a client that supports providing data compensation tasks, such as a video client and a browser client. The network may include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network and a wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that implement wireless communication. The server may be a single server, or a server cluster consisting of multiple servers, or a cloud server. The above is only an example, and no limitation is made to this in this embodiment.

[0031] Through this application, a first feature set of data packets transmitted in a first time period is obtained, wherein the first feature set is used to indicate the attribute information of the data packets transmitted in the first time period; the first feature set is input into a prediction model to obtain a first packet loss rate, wherein the first packet loss rate is used to indicate the packet loss rate of data packets in a second time period; based on the first packet loss rate and the preset packet loss rate, a target data compensation strategy corresponding to the second time period is determined. The packet loss rate of the second time period can be predicted based on the characteristics of the data packets monitored in real time, and a data compensation strategy can be determined based on the packet loss rate to adapt to different network environments and ensure high efficiency and reliability of data transmission. The technical problem of low efficiency of current network transmission methods in related technologies is solved.

[0032] In this embodiment, a data compensation method is provided. Figure 2 is a flow chart of an optional data compensation method according to an embodiment of the present application, such as Figure 2 As shown, the data compensation method includes:

[0033] Step S202, obtaining a first feature set of data packets transmitted in a first time period, wherein the first feature set is used to indicate attribute information of the data packets transmitted in the first time period;

[0034] It should be noted that the first feature set may be features obtained after processing the data packet, or may be information obtained when acquiring the data packet.

[0035] Step S204, inputting the first feature set into the prediction model to obtain a first packet loss rate, wherein the first packet loss rate is used to indicate a packet loss rate of data packets in the second time period;

[0036] It should be noted that the prediction model is a pre-trained model, which can predict the packet loss rate, and its input is the characteristics of the data packet.

[0037] Step S206: determining a target data compensation strategy corresponding to the second time period based on the first packet loss rate and the preset packet loss rate.

[0038] It should be noted that the preset packet loss rate can be set as needed, and the data compensation strategy used in the future can be determined based on the first packet loss rate and the preset packet loss rate. The packet loss rate refers to the ratio of the number of data packets lost in a specific time period to the total number of data packets sent.

[0039] Through this application, a first feature set of data packets transmitted in a first time period is obtained, wherein the first feature set is used to indicate the attribute information of the data packets transmitted in the first time period; the first feature set is input into a prediction model to obtain a first packet loss rate, wherein the first packet loss rate is used to indicate the packet loss rate of data packets in a second time period; based on the first packet loss rate and the preset packet loss rate, a target data compensation strategy corresponding to the second time period is determined. The packet loss rate of the second time period can be predicted based on the characteristics of the data packets monitored in real time, and a data compensation strategy can be determined based on the packet loss rate to adapt to different network environments and ensure high efficiency and reliability of data transmission. The technical problem of low efficiency of current network transmission methods in related technologies is solved.

[0040] In an optional embodiment, the above-mentioned determination of the data compensation strategy corresponding to the second time period based on the first packet loss rate and the preset packet loss rate includes: when the first packet loss rate is greater than the preset packet loss rate and the data compensation strategy corresponding to the current moment is the first data compensation strategy, the first data compensation strategy is adjusted to the second data compensation strategy, and the second data compensation strategy is determined as the target data compensation strategy; when the first packet loss rate is less than the preset packet loss rate and the data compensation strategy corresponding to the current moment is the second data compensation strategy, the second data compensation strategy is adjusted to the first data compensation strategy, and the first data compensation strategy is determined as the target data compensation strategy.

[0041] In an optional implementation, in the initial state of the network, the target data compensation strategy can be set as the first data compensation strategy, and the initial state of the network can be when the network is good, such as low packet loss rate and low delay. The first data compensation strategy can be to use a retransmission mechanism. Automatic repeat request (ARQ) is an error correction protocol used by the data link layer and the transport layer to ensure the reliability of data transmission. If the sender does not receive the confirmation from the receiver within a certain period of time, it will consider that the data packet is lost or damaged during transmission and resend the data packet.

[0042] In an optional implementation, when the network status is good, such as when the packet loss rate is low, the problem can be solved by requesting retransmission. The retransmission mechanism can ensure that the lost data packet can be retransmitted. However, when the network packet loss rate is high or there is a large network delay, using the first data compensation strategy may result in lower performance. Therefore, when the packet loss rate is high, the first data compensation strategy can be converted to the second data compensation strategy.

[0043] In an optional implementation, the second data compensation strategy may be forward error correction, which is a technology used to improve reliability in data transmission. It enables the receiving end to detect and recover lost data packets without resending by sending additional redundant data (coded packets).

[0044] In an optional implementation, forward error correction requires transmission of additional redundant information, thus occupying more bandwidth resources. Therefore, when the packet loss rate of the network is low, the second data compensation strategy can be converted to the first data compensation strategy.

[0045] In an optional implementation, the measurement of the network's packet loss rate can be determined by a pre-set preset packet loss rate. It can be considered that when the network's current packet loss rate is greater than the preset packet loss rate, the network's packet loss rate is large, and when it is less than or equal to the preset packet loss rate, the network's packet loss rate is small.

[0046] Through the above implementation of the present application, the predicted first packet loss rate is compared with the preset packet loss rate to determine the state of the future network environment, and a suitable data compensation strategy is selected according to the predicted network state, so as to dynamically adjust the packet loss recovery strategy according to the real-time monitored network state to adapt to different networks, thereby improving the efficiency and reliability of data transmission.

[0047] In an optional embodiment, the above-mentioned data compensation strategy corresponding to the second time period is determined based on the first packet loss rate and the preset packet loss rate, including: updating the retransmission timeout period of the first data compensation strategy based on the preset time and the first packet loss rate; updating the number of retransmissions of the first data compensation strategy based on the preset number of times and the first packet loss rate.

[0048] In an optional implementation, the determining of the data compensation strategy corresponding to the second time period based on the first packet loss rate and the preset packet loss rate includes: updating the redundancy of the second data compensation strategy based on the first packet loss rate.

[0049] In an optional implementation, when the data compensation strategy does not need to be changed, the parameters of the first data compensation strategy and the second data compensation strategy can be adjusted, such as the parameters of the first data compensation strategy may include a retransmission timeout and a number of retransmissions. The sender starts a timer after sending a data packet. If no confirmation response is received within the specified time, the data packet will be retransmitted. This time is the timeout. The TCP implementation sets a maximum retransmission limit. If this limit is reached, TCP may give up retransmission and report an error. This limit value can be adjusted through configuration parameters to adapt to different network environments and application requirements.

[0050] In an optional implementation, the retransmission timeout period may be adjusted according to the packet loss rate, as shown in the following formula:

[0051] T timeout =T base ×(1+k×P)

[0052] Among them, T timeout is the retransmission timeout. base is the basic timeout. k is an adjustment factor that reflects the impact of packet loss rate on the timeout. P is the first packet loss rate (a value between 0 and 1). The basic timeout and adjustment factor can be set.

[0053] In an optional implementation, the number of retransmissions may be adjusted according to the packet loss rate, as shown in the following formula:

[0054] N retries =max(N min , ceil(α·P))

[0055] Among them, N retries is the maximum number of retransmissions, N min is the minimum number of retransmissions set, α is the adjustment factor related to the packet loss rate, P is the first packet loss rate (a value between 0 and 1), and ceil() is the rounding up function. The minimum number of retransmissions and the adjustment factor can be set.

[0056] In an optional embodiment, the parameters of the second data compensation strategy may have redundancy. The redundancy of FEC (forward error correction) refers to the proportion of additional coded data added to the original data in order to achieve error detection and correction. In practical applications, choosing an appropriate redundancy requires balancing error correction capability and bandwidth utilization. For example, in a bandwidth-constrained environment, it may be necessary to choose a lower redundancy to reduce bandwidth consumption, even if this means sacrificing some error correction capability. On the contrary, in an environment with sufficient bandwidth, a higher redundancy can be selected to improve error correction capability.

[0057] In an optional implementation, the redundancy may be adjusted according to the packet loss rate, as shown in the following formula:

[0058]

[0059] Where: R is the redundancy, P is the first packet loss rate (a value between 0 and 1).

[0060] In an optional implementation, when switching between the first data compensation strategy and the second data compensation strategy, the above parameters may be gradually adjusted to achieve a smooth transition between different strategies.

[0061] Figure 3 It is a schematic diagram of an optional data compensation method according to an embodiment of the present application; in the case of an unknown network state or a reloading method, the data compensation strategy can be initialized, and the first data compensation strategy (retransmission mechanism) is used by default. After obtaining the data in the first time period of the current network, the prediction model can be used to predict the network status of the second time period, such as predicting the packet loss rate of the network to obtain the first packet loss rate. After determining the first packet loss rate, the predicted first packet loss rate and the preset packet loss rate (threshold) can be used for comparison. When the first packet loss rate is greater than the preset packet loss rate, it is determined that the target data compensation strategy should be the second data compensation strategy (forward error correction). When the first packet loss rate is less than the preset packet loss rate, it is determined that the target data compensation strategy should be the first data compensation strategy. Combine the current data compensation strategy to determine whether to switch the data compensation strategy. If the current data compensation strategy is consistent with the target data compensation strategy, there is no need to switch the data compensation strategy, the current strategy can be continued, and the parameters of the current target data compensation strategy can be updated according to the predicted first packet loss rate. If the current data compensation strategy is inconsistent with the target data compensation strategy, the current data compensation strategy needs to be switched to the target data compensation strategy. During the switching process, the parameters of the first data compensation strategy or the second data compensation strategy can be gradually changed to achieve a smooth transition. After determining the target data compensation strategy, it can be used in the current network state.

[0062] Through the above implementation of the present application, the data compensation strategy can be determined according to the predicted first packet loss rate. When the data compensation strategy does not change, its parameters can be updated to better adapt to the current network environment. When the data compensation strategy changes, the parameters can be gradually changed to achieve smooth strategy switching, so as to maintain the stability of the service while improving the data transmission efficiency.

[0063] In an optional embodiment, the above-mentioned obtaining the first feature set of the data packets transmitted in the first time period includes at least one of the following: obtaining a first information set of the data packets transmitted in the first time period, and processing the first information set to obtain the first feature set; determining the mean, variance and standard deviation of the data packets transmitted in the first time period based on the first information set, and determining the mean, variance and standard deviation as the first feature set; determining the packet loss rate in the first time period, and determining the packet loss rate in the first time period as the first feature set.

[0064] In an optional embodiment, the above-mentioned obtaining the first information set of the data packets transmitted in the first time period includes: determining the first information set of the data packets transmitted in the first time period on the network interface; storing the first information set in a log file; and obtaining the first information set from the log file.

[0065] It should be noted that the first information set may be the source IP address, destination IP address, port and size information of the data packet in the first time period.

[0066] In an optional implementation, the extended Berkeley Packet Filter (eBPF) technology can be used to implement kernel-level network traffic monitoring and collection. The role of kernel space is to manage and process the sending and receiving of network data packets. When a data packet arrives at the kernel's network protocol stack, the kernel will try to pass the data packet to the application in user space. The application in user space usually interacts with the network protocol stack in kernel space through a socket interface. When the application sends data, the data is passed to the kernel space, and then the kernel is responsible for sending it over the network. When receiving data, the kernel passes the data packet to the application in user space. User space and kernel space are two main parts in the operating system architecture.

[0067] In an optional embodiment, an eBPF program can be written that may include the following parts: Define maps to store counters, for example, maps that count TCP and UDP packets separately. Code that hooks on the input and output paths, using BPF instructions to capture packets entering and leaving the network interface. Check the packet type, distinguish between TCP and UDP packets, and update the corresponding counters. Extract the source IP address, destination IP address, port, and size information of the packet and process or log it as needed. Packet loss events can be detected by analyzing the packet sequence number and its TCP / UDP fields. For example, if the sequence number of a TCP packet does not match what is expected, or if packets with the same sequence number are received continuously, it may indicate that packet loss has occurred.

[0068] In an alternative implementation, data from the kernel can be transferred to user space using bpf_perf_event_output or bpf_map_update_elem. The source IP address, destination IP address, port, and size information for each packet can be extracted and stored in a map or sent to user space via bpf_perf_event_output(). An eBPF map, a data structure that shares data between kernel space and user space, can be updated using bpf_map_update_elem.

[0069] In an optional embodiment, in user space, data collected by the eBPF program can be accessed through a bpf system call or a libbpf library. The user space can process the data read from the eBPF map, such as analyzing, filtering, or aggregating. The processed data is then written to a log file to obtain a first set of information using standard file I / O operations.

[0070] In an optional embodiment, after obtaining the first information set, data preprocessing can be performed on it. For example, checking and removing duplicate data: a database or data processing tool (such as SQL, Pandas, etc.) can be used to identify and delete duplicate records. Missing values ​​can also be processed. For example, for numerical data, statistical methods such as mean, median, mode or linear interpolation can be used to fill in missing values. If there are many missing values ​​or it is impossible to reasonably interpolate, consider deleting records containing missing values. The data format can also be unified, such as converting timestamps to standard time formats, and regular expressions or string processing functions can be used to format IP addresses and ports. After data preprocessing, the updated first information set can be determined as the first feature set.

[0071] In an optional embodiment, data packets can be grouped according to time windows and network interfaces, and the mean, variance, and standard deviation of the data packet size corresponding to each time window or each network interface can be calculated. A network interface is a connection point between a computer and a network, which allows a computer to send and receive data. In network monitoring, data packets can be captured on a specific network interface to analyze the network traffic passing through the interface. A time window is a time period used to limit the range of data so that analysis can be performed within a specific time period. The mean, variance, and standard deviation of the data packet size corresponding to each time window or each network interface can be used as a first feature set.

[0072] In an optional embodiment, the packet loss rate within a specified time window can be calculated. A time window can be defined: the length of each time window is determined, for example, it can be 5 seconds, 10 seconds or 1 minute, etc. The number of sliding windows N is determined to determine how many time windows the packet loss rate is tracked at the same time. N sliding windows are created, each window is used to track the packet loss situation within a specific time period. A counter can be set for each window to record the number of packet losses and the total number of sent packets within the window. During the data transmission process, each sent data packet is tracked and recorded in the corresponding time window. When the data packet is confirmed to be received, the received packet counter of the corresponding window is updated. If the confirmation of the data packet is not received after the timeout, it is regarded as a packet loss, and the packet loss counter of the corresponding window is updated. Whenever a new data packet arrives, the oldest window is slid forward (that is, the oldest data is discarded), and the new data packet is added to the latest window. The packet loss rate of each window is updated, which can be calculated by the formula: packet loss rate = (number of packet losses / total number of sent packets) × 100%. The packet loss rate corresponding to the first time period can be determined as the first feature.

[0073] Through the above implementation of the present application, the real-time monitoring of network traffic and the accurate counting of data packets are performed through the kernel-level monitoring technology to ensure that data is transmitted to the user space. The data packets in the first period are processed and analyzed, key features are extracted, and a first feature set is formed to provide high-quality input for the model and improve the accuracy of the prediction. In order to better adjust the data compensation strategy.

[0074] In an optional embodiment, the above-mentioned inputting the first feature set into the prediction model to obtain the first packet loss rate includes: inputting the first feature set into the input layer of the prediction model; obtaining the second feature set through the hidden layer of the prediction model; performing a nonlinear transformation on the second feature set to obtain the first packet loss rate.

[0075] Figure 4 It is a schematic diagram of another optional data compensation method according to an embodiment of the present application; in an optional implementation, the first feature set is fed into the input layer of the prediction model, and then passed layer by layer, and the output of each layer becomes the input of the next layer until the last layer (output layer) produces the final output to obtain the first packet loss rate. During the forward propagation process, the input of each layer can be weighted by the weight connected to it, and then a bias term is added, and finally an activation function is used to calculate the output of the layer to obtain a higher feature, that is, the second feature set. The second feature set is processed using an activation function, and the activation function introduces nonlinearity so that the prediction model can learn and simulate nonlinear relationships and complex patterns to obtain the first packet loss rate.

[0076] In an optional implementation, the training process of the above prediction model includes: determining a target parameter set of the prediction model; determining a prediction model to be trained based on the target parameter set, and training the prediction model to be trained to obtain a prediction model.

[0077] In an optional embodiment, the target parameter set can be selected based on experience or by selecting an algorithm. The target parameter set may include parameters such as the number of layers of the prediction model, the number of neurons in each layer, and the learning rate. Specifically, the number of layers is used to indicate the number of hidden layers. The number of neurons is used to indicate the number of neurons in each hidden layer. The learning rate is used to indicate the step size of the weight update during the training process. It may also include the type of activation function used in the hidden layer and the output layer, such as ReLU, Sigmoid, or Tanh. The type of algorithm used for weight update, such as SGD, Adam, RMSprop, etc. The number of samples used in each training iteration, etc.

[0078] In an optional embodiment, after determining the target parameter set, the performance of the network can be evaluated by comparing the output of the prediction model with the true value and calculating a loss function (such as mean square error or cross entropy loss). The gradient of the loss function is back-propagated through the network, and the chain rule is used to calculate the impact of each weight on the final loss. Gradient descent or its variants are used to update the weights and bias terms in the network according to the back-propagated gradient.

[0079] In an optional embodiment, the above-mentioned determination of the target parameter set of the prediction model includes: obtaining an initial parameter set of the prediction model; training an initial prediction model corresponding to the initial parameter set, and determining the fitness of the initial prediction model; iterating the initial parameter set p times: in the case of the first iteration, determining the initial parameter set as the first parameter set, and using the first parameter set to update the second parameter set; training to obtain a prediction model corresponding to the second parameter set, and determining the fitness of the prediction model corresponding to the second parameter set; determining the parameter set corresponding to the larger value of the fitness of the initial prediction model and the fitness of the prediction model corresponding to the second parameter set as the optimal parameter set; in the case of the qth iteration, the q+1th parameter set is determined based on the optimal parameter set and the qth parameter set, and the fitness of the prediction model corresponding to the q+1th parameter set is determined; the parameter set corresponding to the larger value of the fitness of the initial prediction model and the fitness of the prediction models corresponding to the 2nd parameter set to the q+1th parameter set is updated as the optimal parameter set; when the fitness of the prediction model corresponding to the tth parameter set is greater than the preset fitness threshold or q is equal to p, the optimal parameter set is determined as the target parameter set, wherein p is an integer greater than 0, q is an integer greater than 0 and less than or equal to p, and t is an integer greater than 0 and less than or equal to p.

[0080] In an optional embodiment, initialization can be performed, that is, an initial parameter set is randomly generated as the initial position of the individual in the population. Each individual represents a set of hyperparameters, such as learning rate, number of layers, number of neurons in each layer, etc. Then define the search space: for each hyperparameter, define a search range. For example, the learning rate may be between 0.001 and 0.1, the number of layers may be 1 to 5, etc. And use the current hyperparameter combination to train the prediction model, and evaluate its performance on the validation set, usually using the mean square error (MSE) or other indicators as the fitness function. The parameter set can then be updated using a preset update method. After updating the parameter set, it can be detected whether the new parameter set exceeds the defined search space. Train the prediction model again and calculate the corresponding fitness. If the latest fitness performance is better than the fitness corresponding to the current best parameter set, the best parameter set can be updated.

[0081] In an optional implementation, the training set can be obtained by obtaining the log file of the user space. The characteristic information of the data packet of the time window or network interface can be obtained in the user space by using the above-mentioned kernel-level monitoring method. The obtained characteristic information and data packets can be divided into a training set, a validation set and a test set according to a certain ratio, such as 70% as a training set, 15% as a validation set, and 15% as a test set.

[0082] In an optional implementation, the parameter set may be updated using the following formula:

[0083]

[0084] in, is the position of the next iteration of the i-th individual, is the individual's current optimal position, r 1 and r 2 is a random number, x 1 and x 2 is the golden ratio coefficient. The random number increases the randomness of the search, allowing individuals to explore different areas; the golden ratio coefficient controls the scope of the search and determines the movement amplitude of the individual in each iteration.

[0085] Figure 5 is a schematic diagram of another optional data compensation method according to an embodiment of the present application; Figure 5As shown, before training the prediction model, the parameter set of the prediction model can be determined first, and the parameter set can be a hyperparameter of the prediction model. Initialization can be performed first to obtain an initial parameter set, and the initial parameter set can be a set of hyperparameter solutions, and each hyperparameter solution represents a potential hyperparameter set. The prediction model can be constructed using the hyperparameter solution, and training can be performed to obtain the trained initial prediction model, and the fitness can be evaluated using the validation set. After determining the fitness, the initial parameter set can be updated, and the corresponding prediction model and its fitness can be trained again. The updated fitness of each individual can be compared with the individual's optimal fitness, and the individual optimal fitness can be updated when it exceeds the individual's optimal fitness. Then compare whether the individual optimal fitness exceeds the current global optimal fitness. When it exceeds the current global optimal fitness, the global optimal fitness can be updated, and the parameter update step size is determined according to the global optimal fitness and the individual optimal fitness. In the case of not meeting the stop condition, repeat the above process to achieve continuous updating of the parameters. In the case of meeting the stop condition, end the update process and output the parameter set corresponding to the current global optimal fitness. The stopping condition can be that the maximum number of iterations is reached, the fitness is greater than the preset fitness, or the fitness tends to be stable.

[0086] Through the above implementation of the present application, the optimal prediction model parameters can be pre-selected, and a trained prediction model can be obtained according to the obtained target parameter set. The prediction model can be used to predict the collected data to obtain the first packet loss rate of the second time period. Using the prediction model obtained in the above manner, a more accurate prediction of network packet loss can be achieved to improve the accuracy of the adjustment of the data compensation strategy.

[0087] In an optional implementation of the present application, eBPF technology can be used to implement network traffic monitoring and data collection in the kernel, and send it to the user space for easy extraction. The information of the obtained data packet can be processed to obtain a first feature set. And the first feature set and a more accurate prediction model are used to predict the first packet loss rate of the second time period. Combined with the first packet loss rate, the preset packet loss rate and the current data compensation strategy, the target data compensation strategy is adjusted to adapt to the changing network environment. Through the above-mentioned method of the present application, it can be ensured that data is quickly transmitted to the user space to ensure the high efficiency and accuracy of data collection, and the prediction is performed by predicting the prediction model of the predetermined parameter set, which improves the accuracy of the prediction of the first packet loss rate. In the process of adjusting the data compensation strategy, the strategy can be dynamically adjusted according to the first packet loss rate, and smooth switching between different strategies can be achieved to adapt to the dynamically changing network environment. The stability and efficiency of network data transmission are improved, and the user experience is improved.

[0088] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, or by hardware, but in many cases the former is a better implementation method. Based on this understanding, the computer software product is stored in a storage medium (such as ROM / RAM, disk, CD), including several instructions for enabling a terminal device (which can be a computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0089] In the present embodiment, a data compensation device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof have been made no further. As used below, the term "module" may implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and contemplated.

[0090] Figure 6 is a structural block diagram of an optional data compensation device according to an embodiment of the present application, such as Figure 6 As shown, the device comprises:

[0091] A first acquisition module 62, configured to acquire a first feature set of data packets transmitted in a first time period, wherein the first feature set is used to indicate attribute information of the data packets transmitted in the first time period;

[0092] A prediction module 64, configured to input the first feature set into a prediction model to obtain a first packet loss rate, wherein the first packet loss rate is used to indicate a packet loss rate of data packets in a second time period;

[0093] The strategy determination module 66 is used to determine the target data compensation strategy corresponding to the second time period based on the first packet loss rate and the preset packet loss rate.

[0094] Through this application, a first feature set of data packets transmitted in a first time period is obtained, wherein the first feature set is used to indicate the attribute information of the data packets transmitted in the first time period; the first feature set is input into a prediction model to obtain a first packet loss rate, wherein the first packet loss rate is used to indicate the packet loss rate of data packets in a second time period; based on the first packet loss rate and the preset packet loss rate, a target data compensation strategy corresponding to the second time period is determined. The packet loss rate of the second time period can be predicted based on the characteristics of the data packets monitored in real time, and a data compensation strategy can be determined based on the packet loss rate to adapt to different network environments and ensure high efficiency and reliability of data transmission. The technical problem of low efficiency of current network transmission methods in related technologies is solved.

[0095] Optionally, the above-mentioned strategy determination module 66 is also used to adjust the first data compensation strategy to the second data compensation strategy and determine the second data compensation strategy as the target data compensation strategy when the first packet loss rate is greater than the preset packet loss rate and the data compensation strategy corresponding to the current moment is the first data compensation strategy; and to adjust the second data compensation strategy to the first data compensation strategy and determine the first data compensation strategy as the target data compensation strategy when the first packet loss rate is less than the preset packet loss rate and the data compensation strategy corresponding to the current moment is the second data compensation strategy.

[0096] Optionally, the determining the data compensation strategy corresponding to the second time period based on the first packet loss rate and the preset packet loss rate includes: updating the redundancy of the second data compensation strategy based on the first packet loss rate.

[0097] Optionally, the above-mentioned first acquisition module 62 is also used to: obtain a first information set of data packets transmitted in a first time period, and process the first information set to obtain a first feature set; determine the mean, variance and standard deviation of the data packets transmitted in the first time period based on the first information set, and determine the mean, variance and standard deviation as the first feature set; determine the packet loss rate of the first time period, and determine the packet loss rate of the first time period as the first feature set.

[0098] Optionally, the above-mentioned obtaining the first information set of the data packets transmitted in the first time period includes: determining the first information set of the data packets transmitted in the first time period at the network interface; storing the first information set in a log file; and obtaining the first information set from the log file.

[0099] Optionally, the prediction module 64 is further used to: input the first feature set into the input layer of the prediction model; obtain the second feature set through the hidden layer of the prediction model; perform nonlinear transformation on the second feature set and obtain the first packet loss rate.

[0100] Optionally, the training process of the above prediction model includes: determining a target parameter set of the prediction model; determining a prediction model to be trained based on the target parameter set, and training the prediction model to be trained to obtain the prediction model.

[0101] Optionally, the above-mentioned determination of the target parameter set of the prediction model includes: obtaining an initial parameter set of the prediction model; training to obtain an initial prediction model corresponding to the initial parameter set, and determining the fitness of the initial prediction model; iterating the initial parameter set p times: in the case of the first iteration, determining the initial parameter set as the first parameter set, and using the first parameter set to update to obtain the second parameter set; training to obtain a prediction model corresponding to the second parameter set, and determining the fitness of the prediction model corresponding to the second parameter set; determining the parameter set corresponding to the larger value of the fitness of the initial prediction model and the fitness of the prediction model corresponding to the second parameter set as the optimal parameter set combination; in the case of the qth iteration, the q+1th parameter set is determined based on the optimal parameter set and the qth parameter set, and the fitness of the prediction model corresponding to the q+1th parameter set is determined; the parameter set corresponding to the larger value of the fitness of the initial prediction model and the fitness of the prediction models corresponding to the 2nd parameter set to the q+1th parameter set is updated as the optimal parameter set; when the fitness of the prediction model corresponding to the tth parameter set is greater than the preset fitness threshold or q is equal to p, the optimal parameter set is determined as the target parameter set, wherein p is an integer greater than 0, q is an integer greater than 0 and less than or equal to p, and t is an integer greater than 0 and less than or equal to p.

[0102] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0103] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0104] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0105] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0106] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0107] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0108] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0109] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A data compensation method, characterized in that: The method comprises: Acquire a first feature set of data packets transmitted in a first time period, wherein the first feature set is used to indicate attribute information of the data packets transmitted in the first time period; Inputting the first feature set into a prediction model to obtain a first packet loss rate, wherein the first packet loss rate is used to indicate a packet loss rate of data packets in a second time period; Based on the first packet loss rate and a preset packet loss rate, a target data compensation strategy corresponding to the second time period is determined.

2. The method according to claim 1, characterized in that The determining, based on the first packet loss rate and a preset packet loss rate, a data compensation strategy corresponding to the second time period includes: When the first packet loss rate is greater than the preset packet loss rate and the data compensation strategy corresponding to the current moment is the first data compensation strategy, adjusting the first data compensation strategy to a second data compensation strategy, and determining the second data compensation strategy as the target data compensation strategy; When the first packet loss rate is less than the preset packet loss rate and the data compensation strategy corresponding to the current moment is the second data compensation strategy, the second data compensation strategy is adjusted to the first data compensation strategy, and the first data compensation strategy is determined as the target data compensation strategy.

3. The method according to claim 2, characterized in that The determining, based on the first packet loss rate and a preset packet loss rate, a data compensation strategy corresponding to the second time period includes: Updating a retransmission timeout period of the first data compensation strategy based on a preset time and the first packet loss rate; The number of retransmissions of the first data compensation strategy is updated based on a preset number of times and the first packet loss rate.

4. The method according to claim 2, characterized in that: The determining, based on the first packet loss rate and a preset packet loss rate, a data compensation strategy corresponding to the second time period includes: The redundancy of the second data compensation strategy is updated based on the first packet loss rate.

5. The method according to claim 1, characterized in that The obtaining of a first feature set of a data packet transmitted in a first time period includes at least one of the following: Acquire a first information set of data packets transmitted in the first time period, and process the first information set to obtain a first feature set; Determine a mean, variance, and standard deviation of data packets transmitted in the first time period based on the first information set, and determine the mean, variance, and standard deviation as the first feature set; Determine a packet loss rate in the first time period, and determine the packet loss rate in the first time period as the first feature set.

6. The method according to claim 5, characterized in that The obtaining of a first information set of data packets transmitted in the first time period includes: Determining, at the network interface, a first information set of data packets transmitted within the first time period; storing the first information set in a log file; The first information set is obtained from the log file.

7. The method according to claim 1, characterized in that The step of inputting the first feature set into a prediction model to obtain a first packet loss rate includes: Inputting the first feature set into an input layer of the prediction model; Obtaining a second feature set through the hidden layer of the prediction model; The second feature set is subjected to nonlinear transformation to obtain the first packet loss rate.

8. The method according to claim 7, characterized in that The training process of the prediction model includes: Determining a target parameter set of the prediction model; A prediction model to be trained is determined based on the target parameter set, and the prediction model to be trained is trained to obtain the prediction model.

9. The method according to claim 8, characterized in that Determining a target parameter set of the prediction model includes: Obtaining an initial parameter set of the prediction model; Train to obtain an initial prediction model corresponding to the initial parameter set, and determine the fitness of the initial prediction model; The initial parameter set is iterated p times: In the case of the first iteration, the initial parameter set is determined as the first parameter set, and the first parameter set is used to update the second parameter set; Train to obtain a prediction model corresponding to the second parameter set, and determine the fitness of the prediction model corresponding to the second parameter set; Determine the parameter set corresponding to the larger value of the fitness of the initial prediction model and the fitness of the prediction model corresponding to the second parameter set as the optimal parameter set; In the case of the qth iteration, a q+1th parameter set is determined based on the optimal parameter set and the qth parameter set, and the fitness of the prediction model corresponding to the q+1th parameter set is determined; The parameter set corresponding to the larger value among the fitness of the initial prediction model and the fitness of the prediction models corresponding to the second parameter set to the q+1th parameter set is updated as the optimal parameter set; When the fitness of the prediction model corresponding to the tth parameter set is greater than the preset fitness threshold or the q is equal to p, the optimal parameter set is determined as the target parameter set, wherein p is an integer greater than 0, q is an integer greater than 0 and less than or equal to p, and t is an integer greater than 0 and less than or equal to p.

10. A data compensation device, characterized in that: The device comprises: A first acquisition module, used to acquire a first feature set of data packets transmitted in a first time period, wherein the first feature set is used to indicate attribute information of the data packets transmitted in the first time period; A prediction module, configured to input the first feature set into a prediction model to obtain a first packet loss rate, wherein the first packet loss rate is used to indicate a packet loss rate of data packets in a second time period; A strategy determination module is used to determine a target data compensation strategy corresponding to the second time period based on the first packet loss rate and a preset packet loss rate.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method according to any one of claims 1 to 9 when executed by a processor.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images