A path-aware restoration method for information age learning based on multi-armed bandit
Through the information age learning path perception recovery method based on multi-arm slot machine, the information age and variance of the retransmission packet after smoothing is calculated, and the optimal path is selected for retransmission, which solves the problem of slow retransmission path switching perception in streaming video transmission. , improving service quality and user experience.
Patent Information
- Application Number
- CN202411280697.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-09-13
AI Technical Summary
In streaming video transmission with high data freshness requirements, the retransmission path switching perception is slow, resulting in deterioration of service quality of streaming video transmission and degradation of user experience.
Using the information age learning path perception recovery method based on multi-arm slot machine, the information age and variance of the smoothed retransmission packet are calculated through multiple retransmission paths between the sending and receiving ends, and the income of each path is updated. The optimal path that maximizes returns performs packet retransmission, and uses Bayesian trusted boundary adjustment factor to optimize the retransmission strategy.
Ensure the freshness of retransmission data packets, improve the timeliness and accuracy of data transmission, reduce the demand for computing resources, ensure that measures can be taken quickly when path state changes, and improve the stability and reliability of streaming video transmission , improve service quality.
Smart Images

Figure CN118803380B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of transmission control technology, and in particular to an information age learning path perception recovery method based on a multi-armed tiger machine. Background Art
[0002] In the modern network environment, with the continued rapid growth of video traffic and the widespread popularity of emerging applications such as virtual reality, augmented reality and 360-degree video, higher requirements are placed on the carrying capacity and transmission service quality of network technology.
[0003] When using existing technologies for data transmission, the sender usually periodically sends probe data packets to the receiver and sets a timeout threshold to monitor the status of each retransmission path. When the probe data packet fails to be confirmed in time, the probe timeout mechanism is activated, prompting the sender to adopt a fast recovery and fast retransmission strategy to ensure the stability and reliability of data transmission.
[0004] However, since the probe timeout mechanism uses the delay of data packet confirmation to evaluate the availability of the path, for applications such as streaming video that require high data freshness, the retransmission path switching perception is slow, resulting in the degradation of the quality of streaming video transmission service, thereby reducing the user experience. Therefore, it is urgent to provide a solution to improve the above problems. Summary of the invention
[0005] The purpose of the present invention is to provide an information age learning path perception recovery method based on a multi-armed bandit, so as to improve the problem that the retransmission path switching perception is slow in the prior art, resulting in low efficiency of streaming media video transmission service quality.
[0006] The present invention provides an information age learning path perception recovery method based on a multi-armed bandit machine, which adopts the following technical solutions:
[0007] A data transmission graph is established based on a sender and a receiver of streaming media video, wherein the sender sends multiple main data packets to the receiver, and when data transmission fails, the sender sends multiple retransmission data packets to the receiver, wherein there are multiple retransmission paths between the sender and the receiver;
[0008] Obtaining the information age of the retransmitted data packet based on the generation time of the retransmitted data packet and the reception time of the receiving end, and obtaining the smoothed information age of the retransmitted data packet after smoothing the information age;
[0009] The mean and variance of the information age of the smoothed retransmitted data packets are calculated, and the benefit of each retransmission path is updated based on the mean and variance to obtain an optimal path that maximizes the benefit, and the data packets are retransmitted in the optimal path.
[0010] The beneficial effects of the information age learning path-aware recovery method based on a multi-armed bandit provided by the present invention are that, firstly, the present invention proposes the concept of information age of retransmitted data packets, thereby ensuring the freshness of retransmitted data packets, thereby ensuring the timeliness of streaming media data transmission; secondly, the Bayesian trusted boundary adjustment factor is used to reduce the demand for computing resources and improve the data transmission efficiency and accuracy; finally, a data packet retransmission mechanism is designed based on profit maximization to ensure that when the path state changes, measures can be taken quickly to ensure the stability and reliability of data transmission, thereby improving the service quality of streaming media video transmission.
[0011] Optionally, the information age of the retransmitted data packet is expressed as:
[0012] ;
[0013] in, Indicates the information age of the retransmitted data packet, Indicates that the receiving end is on the path The local timestamp of the received retransmitted packet, Indicates the first The time it takes to generate a retransmitted packet.
[0014] Optionally, when the sending end sends multiple main data packets to the receiving end, if one or more of the main data packets are lost, the receiving end sends a retransmission data instruction to the sending end, and the sending end starts uploading the retransmission data packets to the receiving end after receiving the retransmission data instruction.
[0015] Optionally, the process of obtaining the optimal path that maximizes the benefit includes:
[0016] Initialize the number of slot machine arms, the information age of the retransmitted data packet, and the strategy set, wherein each arm corresponds to a retransmission path, the number of arms is the number of retransmission paths, each retransmission path corresponds to a probability distribution of benefits, and the strategy set consists of the retransmitted data packet, the retransmission path set, the loss round set, and the scheduling round set;
[0017] In each time step, a path is randomly selected from the retransmission path set based on the exploration probability to transmit the data packet, and a retransmission path that maximizes the current benefit is selected based on the utilization probability to transmit the data packet;
[0018] If the data transmission is successful, the current benefit is updated based on the Bayesian credible boundary adjustment factor; if the transmission fails, the receiving end adds the data packet to the retransmission data packet list and returns it to the sending end, and the sending end resends the retransmission data packet to the receiving end;
[0019] The frequency of the pull rod currently selected by the multi-armed bandit machine is updated, and the optimal path that maximizes the profit is obtained based on the frequency of the pull rod until the maximum time step is reached.
[0020] Optionally, the Bayesian credible boundary adjustment factor It can be expressed as:
[0021] ;
[0022] in, Indicates the total number of retransmission paths, represents the variance of the smoothed retransmitted packet information age, Indicates the frequency of the currently selected lever of the multi-armed slot machine.
[0023] Optionally, the frequency of the pull rod currently selected by the multi-armed slot machine is expressed as:
[0024] ;
[0025] in, Indicates the frequency of the currently selected lever of the multi-armed slot machine. Indicates the number of retransmission rounds for lost packets.
[0026] Optionally, when updating the current benefit based on the Bayesian credible boundary adjustment factor, the benefit value of each retransmission path is:
[0027] Q = E r ∼ π [ 1 E ( Δ d p ) + ln(| P |) N t ( d p ) · θ ] ;
[0028] in, represents the benefit value of each retransmission path, represents the mean value of the information age of the retransmitted data packets after smoothing, Indicates the total number of retransmission paths, Indicates the frequency of the currently selected lever of the multi-armed slot machine. represents the Bayesian credible boundary adjustment factor.
[0029] Optionally, the information age of the smoothed retransmitted data packets follows a log-normal distribution, satisfying the relationship: ,in:
[0030] ;
[0031] ;
[0032] in, Indicates the information age of the retransmitted data packet after smoothing, represents the mean of the log information age, represents the variance of the log information age, represents the variance of the information age of the retransmitted packets after smoothing, Represents the mean information age of retransmitted packets after smoothing. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 A flow chart showing an information age learning path perception recovery method based on a multi-armed bandit provided by the present invention;
[0034] Figure 2 A system architecture diagram showing an information age learning path perception recovery method based on a multi-armed bandit provided by the present invention;
[0035] Figure 3 A simulation topology diagram showing the streaming media video transmission provided by the present invention;
[0036] Figure 4 A comparison diagram showing the real-time effective throughput provided by the present invention;
[0037] Figure 5 A comparison diagram showing the number of data packets provided by the present invention;
[0038] Figure 6 A peak signal-to-noise ratio comparison diagram provided by the present invention is shown. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.
[0040] In some embodiments, see Figure 1 The present invention provides an information age learning path perception recovery method based on a multi-armed bandit machine, which adopts the following technical solutions:
[0041] S1, a data transmission graph is established based on a sender and a receiver of streaming media video, the sender sends multiple main data packets to the receiver, and when data transmission fails, the sender sends multiple retransmission data packets to the receiver, wherein there are multiple retransmission paths between the sender and the receiver;
[0042] S2, obtaining the information age of the retransmitted data packet based on the generation time of the retransmitted data packet and the reception time of the receiving end, and obtaining the smoothed information age of the retransmitted data packet after smoothing the information age;
[0043] S3. Calculate the mean and variance of the information age of the smoothed retransmitted data packets, and update the benefit of each retransmission path based on the mean and variance to obtain an optimal path that maximizes the benefit, and retransmit the data packets in the optimal path.
[0044] In some embodiments, see Figure 2 There are two transmission paths for data packets, namely retransmission path #1 and retransmission path #2 (retransmission path #1 is equivalent to path #1, and retransmission path #2 is equivalent to path #2). The sender transmits multiple data packets about streaming media videos to the sending buffer on the two transmission paths of retransmission path #1 and retransmission path #2. After receiving the data packets, the sending buffer sends the data packets to multiple network routing nodes. Multiple data packets are transmitted between multiple network routing nodes. Finally, the network routing nodes send all the data packets to the receiving end through the two transmission paths of retransmission path #1 and retransmission path #2. The receiving end then sends all the data packets to the receiving end buffer to complete the data transmission process. If one or more data packets fail during the transmission process, the receiving end sends an information flow instruction for retransmitting the data to the sending end. After receiving the instruction, the sending end will select a path with the greatest benefit from retransmission path #1 and retransmission path #2 to retransmit the lost data packets.
[0045] Specifically, when executing step S1, the information age of the retransmitted data packet is expressed as:
[0046] ;
[0047] in, Indicates the information age of the retransmitted data packet, Indicates that the receiving end is on the path The local timestamp of the received retransmitted packet, Indicates the first The time it takes to generate a retransmitted packet.
[0048] Furthermore, when executing step S1, the sending end sends multiple main data packets to the receiving end, if the main data packet is lost, the receiving end sends a retransmission data instruction to the sending end, and the sending end starts uploading the retransmission data packet to the receiving end after receiving the retransmission data instruction.
[0049] In some embodiments, when the information age of the retransmitted data packet after smoothing is obtained after executing step S2, the information age of the retransmitted data packet after smoothing is:
[0050] ;
[0051] in, Indicates the information age of the retransmitted data packet after smoothing, Indicates that in the current path Previous The information age of the data packet, represents the average information age of retransmitted packets, , , Indicates that in the current path The maximum information age using the same strategy, Indicates that in the current path The minimum information age for the same strategy.
[0052] Furthermore, the average information age of retransmitted packets It can be expressed as:
[0053] ;
[0054] Among them, N represents the current path The number of retransmitted packets.
[0055] In some embodiments, the process of obtaining the optimal path that maximizes the revenue in executing step S3 includes:
[0056] S31, initializing the number of slot machine arms, the information age of the retransmitted data packet and the strategy set, wherein each arm corresponds to a retransmission path, the number of arms is the number of retransmission paths, each retransmission path corresponds to a probability distribution of benefits, and the strategy set consists of the retransmitted data packet, the retransmission path set, the loss round set and the scheduling round set;
[0057] S32. In each time step, a path is randomly selected from the retransmission path set based on the exploration probability to transmit the data packet, and a retransmission path that maximizes the current benefit is selected based on the utilization probability to transmit the data packet;
[0058] S33. If the data transmission is successful, the current benefit is updated based on the Bayesian credible boundary adjustment factor; if the transmission fails, the receiving end adds the data packet to the retransmission data packet list and returns it to the sending end, and the sending end resends the retransmission data packet to the receiving end;
[0059] S34, updating the frequency of the currently selected rod pulling of the multi-armed bandit machine, and obtaining the optimal path that maximizes the profit based on the frequency of the rod pulling, until the maximum time step is reached.
[0060] Specifically, when executing step S31, there are a total of M retransmitted data packets, which are expressed as: , the retransmission path has , expressed as: , the lost round set It can be expressed as: , , Indicates the path The last confirmed round.
[0061] Furthermore, the scheduling round set follows the following principles: first, a path subset is selected from multiple retransmission paths to retransmit the data packet. Second, if the current path has an available window, the retransmission data packet request is scheduled to the current path. At this time, the scheduling round set is expressed as: ,in, Indicates that retransmission packets are assigned to the path superior, Indicates that retransmission packets are assigned to the path On, and so on, Indicates that retransmission packets are assigned to the path superior.
[0062] Furthermore, by combining the retransmission data packets, the retransmission path set, the loss round set, and the scheduling round set, the strategy set can be described as a four-element strategy group: .
[0063] Further, when executing step S32, at each time step Based on the exploration probability From the retransmission path Randomly select a path from Retransmit the data packet, where: , and The utilization probability selection makes the current profit The data packet is retransmitted using a maximized retransmission path.
[0064] Specifically, when executing step S33, when data transmission fails, the receiving end feeds back the data packet that needs to be retransmitted in the form of an ACK message and sends it to the sending end. After receiving the ACK message, the sending end re-uploads the data packet to the receiving end.
[0065] Furthermore, when executing step S33, the Bayesian credible boundary adjustment factor It can be expressed as:
[0066] ;
[0067] in, Indicates the total number of retransmission paths, represents the variance of the smoothed retransmitted packet information age, Indicates the frequency of the currently selected lever of the multi-armed slot machine.
[0068] In fact, the Bayesian credible margin adjustment factor The purpose is to balance path exploration and path utilization among multiple retransmission paths by dynamically adjusting the ratio of exploration and utilization, ensuring that better paths can be gradually discovered during the retransmission path selection process.
[0069] Further, when executing step S34, the frequency of the pull rod currently selected by the multi-armed slot machine is expressed as:
[0070] ;
[0071] in, Indicates the frequency of the currently selected lever of the multi-armed slot machine. Indicates the number of retransmission rounds for lost packets.
[0072] In fact, the frequency of pulling the rod of the multi-armed slot machine starts from 0, that is, the initial The value of is 0, It increases linearly with a tolerance of 1.
[0073] In some embodiments, when executing step S3 to update the current benefit based on the Bayesian credible boundary adjustment factor, the benefit value of each retransmission path is:
[0074] Q = E r ∼ π [ 1 E ( Δ d p ) + ln(| P |) N t ( d p ) · θ ] ;
[0075] in, represents the benefit value of each retransmission path, represents the mean value of the information age of the retransmitted data packets after smoothing, Indicates the total number of retransmission paths, Indicates the frequency of the currently selected lever of the multi-armed slot machine. represents the Bayesian credible boundary adjustment factor.
[0076] In fact, when calculating the benefit value of each retransmission path Then, by selecting the optimal path , to dynamically adjust the retransmission path to improve the performance and user experience of streaming video transmission.
[0077] Furthermore, in heterogeneous network environments, there are uncertain factors such as queuing and transmission delays. These uncertain factors often lead to asymmetric distribution and long-tail effect of transmission time. The lognormal distribution can well describe this characteristic. Variables such as delay and packet loss rate in heterogeneous networks may fluctuate significantly, making the rate of change of information age present a skewed distribution. The lognormal distribution is suitable for describing random variables in this case because it has positive values and can reflect the probability distribution characteristics of large deviations. Therefore, the information age of the retransmitted data packet after smoothing obeys the lognormal distribution, that is, , specifically, satisfying the relationship:
[0078] ;
[0079] ;
[0080] in, Indicates the information age of the retransmitted data packet after smoothing, represents the mean of the log information age, represents the variance of the log information age, represents the variance of the information age of the retransmitted packets after smoothing, Represents the mean information age of retransmitted packets after smoothing.
[0081] In order to verify the effectiveness of the information age learning path-aware recovery method based on a multi-armed bandit provided by the present invention, a simulation experiment of multi-path parallel transmission is carried out in a simulation topology diagram to study the performance of the method proposed by the present invention.
[0082] like Figure 3 As shown, QUIC streams are transmitted between adjacent network routing nodes, and there are two data pipes between the sender and the receiver. Different data indicators are assigned to the two data pipes to achieve streaming video transmission. The bandwidth on retransmission path #1 is 30-50Mbps, the delay is 80-100ms, and the packet loss rate (PLR) is fixed at 1.5%. The packet loss rate of retransmission path #2 is 0.5%, the bandwidth is 100-130Mbps, the delay is 10-40ms, and the receiving end buffer is set to 4MB.
[0083] like Figure 4As shown, an information age learning path-aware recovery method (AoIFR) based on a multi-armed bandit is provided by the present invention. There are two retransmission paths corresponding to this method, namely: AoIFR-path#1 and AoIFR-path#2. The retransmission paths corresponding to the MPQUIC (multi-path QUIC) algorithm are: MPQUIC-path#1 and MPQUIC-path#2. In the simulation experiment, AoIFR-path#1 is compared with MPQUIC-path#1, and AoIFR-path#2 is compared with MPQUIC-path#2.
[0084] Depend on Figure 4 It can be seen that compared with the MPQUIC algorithm, the method provided by the present invention (AoIFR) has considerable performance improvement in the instantaneous effective throughput of AoIFR-path#1 and AoIFR-path#2. The instantaneous effective average throughput on AoIFR-path#1 is 73.21 Kbps, which is 73.1% higher than the 42.29 Kbps of the MPQUIC algorithm, and the throughput performance on AoIFR-path#2 is about 2.7 times higher.
[0085] like Figure 5 As shown, the method (AoIFR) provided by the present invention receives 1042 and 121 more data packets in 90 seconds than the MPQUIC algorithm on AoIFR-path#1 and AoIFR-path#2, respectively.
[0086] In streaming video transmission, Peak Signal-to-Noise Ratio (PSNR) is one of the objective indicators to measure video quality. It quantifies the impact of factors such as data loss, delay, and bit error on video quality during the transmission process. Figure 6 As shown, in the method provided by the present invention, the average peak signal-to-noise ratios of AoIFR-path#1 and AoIFR-path#2 are 36.30 and 33.91, which are improved by 14.1% and 3.9% respectively compared with 31.81 and 32.65 of the MPQUIC algorithm.
[0087] In summary, the information age learning path-aware recovery method based on a multi-armed bandit provided by the present invention can reduce the number of unnecessary retransmissions, thereby optimizing the efficiency of data packet transmission and retransmission.
[0088] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.
Claims
1. A method for information age learning path perception recovery based on multi-armed bandit, characterized in that: include: A data transmission graph is established based on a sender and a receiver of streaming media video, wherein the sender sends multiple main data packets to the receiver, and when data transmission fails, the sender sends multiple retransmission data packets to the receiver, wherein there are multiple retransmission paths between the sender and the receiver; Obtaining the information age of the retransmitted data packet based on the generation time of the retransmitted data packet and the reception time of the receiving end, and obtaining the smoothed information age of the retransmitted data packet after smoothing the information age; Initialize the number of slot machine arms, the information age of the retransmitted data packet, and the strategy set, wherein each arm corresponds to a retransmission path, the number of arms is the number of retransmission paths, each retransmission path corresponds to a probability distribution of benefits, and the strategy set consists of the retransmitted data packet, the retransmission path set, the loss round set, and the scheduling round set; In each time step, a path is randomly selected from the retransmission path set based on the exploration probability to transmit the data packet, and a retransmission path that maximizes the current benefit is selected based on the utilization probability to transmit the data packet; if the data transmission is successful, the current benefit is updated based on the Bayesian credible boundary adjustment factor; if the transmission fails, the receiving end adds the data packet to the retransmission data packet list and returns it to the sending end, and the sending end resends the retransmission data packet to the receiving end, where The Bayesian Credibility Margin Adjustment Factor It can be expressed as: ; in, Indicates the total number of retransmission paths, represents the variance of the smoothed retransmitted packet information age, Indicates the frequency of the currently selected lever of the multi-armed slot machine; The benefit value of each retransmission path is: ; in, represents the benefit value of each retransmission path, represents the mean value of the information age of the retransmitted data packets after smoothing, Indicates the total number of retransmission paths, Indicates the frequency of the currently selected lever of the multi-armed slot machine. represents the Bayesian credible boundary adjustment factor; Update the frequency of the lever currently selected by the multi-armed slot machine, and obtain the optimal path that maximizes the profit based on the frequency of the lever, and retransmit the data packet in the optimal path until the maximum time step is reached, wherein the frequency of the lever currently selected by the multi-armed slot machine is expressed as: ; in, Indicates the frequency of the currently selected lever of the multi-armed slot machine. Indicates the number of retransmission rounds for lost packets.
2. According to claim 1, a method for information age learning path perception recovery based on a multi-armed bandit machine is characterized in that: The information age of the retransmitted data packet is expressed as: ; in, Indicates the information age of the retransmitted data packet, Indicates that the receiving end is on the path The local timestamp of the received retransmitted packet, Indicates the first The time it takes to generate a retransmitted packet.
3. According to claim 1, a method for information age learning path perception recovery based on a multi-armed bandit machine is characterized in that: When the sending end sends multiple main data packets to the receiving end, if one or more of the main data packets are lost, the receiving end sends a retransmission data instruction to the sending end, and the sending end starts uploading the retransmission data packets to the receiving end after receiving the retransmission data instruction.
4. The information age learning path perception recovery method based on a multi-armed bandit according to claim 1 is characterized in that: The information age of the smoothed retransmitted data packets follows a log-normal distribution and satisfies the relationship: ,in: ; ; in, Indicates the information age of the retransmitted data packet after smoothing, represents the mean of the log information age, represents the variance of the log information age, represents the variance of the information age of the retransmitted packets after smoothing, Represents the mean information age of retransmitted packets after smoothing.
