An Adaptive Multipath Real-Time Streaming Media Transmission Method Based on LinUCB

Abstract

This invention discloses an adaptive multipath real-time streaming media transmission method based on LinUCB, comprising: constructing a real-time streaming media multipath transmission system; establishing a two-layer session message handshake and establishing multipaths; loading and initializing a model; transmitting streaming media data; adaptive transmission adjustment; calculating rewards and updating the model; and completing real-time streaming media transmission. The online learning method based on LinUCB in this invention can switch real-time video stream transmission strategies in a lightweight adaptive manner according to the dynamic network environment; the adaptive real-time streaming media transmission platform based on MPQUIC aggregates the bandwidth of multiple transmission paths, which can effectively improve transmission throughput and reduce the initial latency and rebuffering time of real-time streaming media transmission.

Description

Technical Field

[0001] This invention relates to the field of computer network transmission control, and in particular to an adaptive multipath real-time streaming media transmission method based on LinUCB. Background Technology

[0002] In recent years, live streaming applications have become an indispensable part of people's daily lives, accounting for 74% of all mobile traffic. Live streaming encompasses various categories, including live product broadcasts, remote monitoring, and VR / AR. To ensure a good user experience for live streaming applications, high-bandwidth, low-latency network transmission services are required. However, network congestion, latency, jitter, and high-bitrate live streaming transmissions exceeding the current link bandwidth can lead to stuttering and blurry streaming images.

[0003] Current real-time streaming media transmission protocols, taking the mainstream RTMP protocol as an example, are mostly built on top of TCP. However, TCP requires 1-3 RTTs to complete the handshake, making it unsuitable for real-time streaming applications with strict initial latency requirements. Moreover, TCP is implemented in kernel space, and optimizing its functionality requires adjustments to the entire operating system kernel. The QUIC protocol, an extension of UDP, can effectively reduce handshake latency. Furthermore, QUIC's user-space implementation allows for easy support of protocol functionality extensions.

[0004] Multipath transport protocols improve transmission bandwidth and reliability by establishing multiple paths between end-to-end connections. Combining the features of the QUIC protocol, multipath QUIC supports flexible protocol function expansion and reduces initial latency, providing a foundation for optimizing the user experience of real-time streaming applications.

[0005] To further ensure the quality of service and user experience of real-time streaming applications, improvements to the transmission protocol are necessary. However, optimizing the protocol for network scenarios and streaming media characteristics has become a key issue. This invention employs the lightweight LinUCB algorithm, which can switch between four proposed transmission optimization modules, effectively addressing the user experience degradation issues of real-time streaming under conditions of "high packet loss," "insufficient bandwidth," and "weak network." Summary of the Invention

[0006] To address the problems existing in the prior art, the purpose of this invention is to provide an adaptive multipath real-time streaming media transmission method based on LinUCB. This method can not only reduce the rebuffering time of real-time streaming media transmission, but also reduce streaming media latency and initial opening latency.

[0007] To achieve the above objectives, this invention provides an adaptive multipath real-time streaming media transmission method based on LinUCB, the method comprising the following steps:

[0008] Step 1: Construct a real-time streaming media multipath transmission system. This system uses the RTMP protocol for real-time streaming media push and the MPQUIC protocol for adaptive multipath transmission. The transmission system includes a partially reliable retransmission module A, a discard module A, a copy module A, and a stream pointer control module A.

[0009] Step 2: Handshake and establish a two-layer session message, and establish multiple paths;

[0010] Step 3: Model loading and initialization, loading the protocol stack model file Lin, and configuring it for each action. Initialize model parameters and ;in For one A dimensional matrix measures the observed state history information. For one A dimensional matrix measures the observed reward history information. The observed state dimension;

[0011] Step 4: Streaming media data transmission;

[0012] Step 5: Adaptive transmission adjustment; select the switching action according to the switching strategy, and select to start the stream pointer control module A, copy module A, discard module A or partial reliable retransmission module A according to the switching instruction;

[0013] Step 6: Calculate the reward and update the model;

[0014] Step 7: Complete the real-time streaming media transmission.

[0015] Furthermore, the real-time streaming media multipath transmission system consists of a server and a client. The client deploys an RTMP client and an MPQUIC transmission client. The RTMP client includes an RTMP handshake module A, a frame segmentation module A, a frame encapsulation module A, and a frame parsing module A. The MPQUIC transmission client includes a handshake module A and a session A. The session A maintains the following modules: path management module A, data stream module A, sending module A, congestion control module A, scheduler module A, and acknowledgment module A.

[0016] Furthermore, the second step includes the following sub-steps:

[0017] S2.1 The RTMP handshake module B of the RTMP server starts listening and calls the underlying MPQUIC handshake module B;

[0018] The handshake module B of the S2.2 MPQUIC server starts listening;

[0019] S2.3 The RTMP handshake module A of the RTMP client starts the handshake process and calls the underlying MPQUIC handshake module A;

[0020] S2.4 Handshake module A initiates the handshake process, calling the underlying UDP.Dial interface. The interface accepts the server's IP address and port number, and sends a probe packet containing connection parameters, including the client's supported protocol version, the newly created connection ID, and the key. It then waits for a response packet from handshake module B to verify the validity of the server's certificate and key. Upon receiving the packet, server-side handshake module B establishes session B based on the parsed connection ID. After receiving the server's handshake response, the client establishes session A.

[0021] S2.5 To maintain this connection, Session A establishes Path Management Module A, Data Flow Module A, Sending Module A, Congestion Control Module A, Scheduler Module A, and Response Module A; the module establishment process for Session B is the same as that for Session A.

[0022] S2.6 Path management module B polls the network card address of the server and sends an ADDADDRESS frame to announce the multiple addresses of the local machine. Path management module A receives the ADDADDRESS frame sent by the peer and polls the network card address of the local machine to establish multiple paths between the end-to-end address pairs.

[0023] S2.7 Data stream module A establishes a data stream, data stream module B receives the data stream, and both parties send data on the established data stream; the client and server respectively return the established session and data stream connection to the upper-layer RTMP application;

[0024] S2.8 RTMP handshake module A and RTMP handshake module B send handshake messages to each other through the established data stream.

[0025] Furthermore, the fourth step includes:

[0026] An S4.1 RTMP client reads a single frame of video data. Frame encapsulation module A parses the frame data structure and constructs the RTMP message header. Frame segmentation module A segments the data and fills the payload portion of the RTMP message. Then, it calls the Write interface to process the message data. Write data to the data stream;

[0027] S4.2 Scheduler A determines the size of the congestion control window provided by congestion control module A. It will be retrieved from data flow module A The data of the specified size is used to fill data stream frames and encapsulate them into data packets. This is the maximum message size;

[0028] S4.3 Scheduler A collects the RTT of all paths and selects the path with the lowest RTT to schedule the message; the message is sent to the link corresponding to the path.

[0029] S4.4 The server receives the message, parses the message content, extracts the data stream frames and fills them into the stream queue managed by the data stream module B;

[0030] The S4.5 RTMP server sequentially reads the next frame of RTMP messages from data stream module B. If the data that makes up the message has not yet arrived, the reading process is blocked and S4.5 is repeated; if the data has arrived, the frame decapsulation module B reads the data from it.

[0031] Furthermore, the fifth step includes:

[0032] S5.1 Transmission monitoring module A collects path information. The collection time is when new data is written to the transmission buffer, denoted as time [time]. , ,in This represents the total number of paths. ,in, For path exist Time-to-Time (RTT) For path exist Congestion window at any time, For path exist The number of messages sent at any given time. For path exist Packet loss rate at any given moment;

[0033] S5.2 Frame Information Evaluation Module A acquires information from video frames. ,in For frame rate, For real-time bitrate, The current frame size. The current frame type;

[0034] S5.3 Combines the path information and video frame information collected in S5.1 and S5.2 as features;

[0035] S5.4 Select the switching action based on the calculation; the action space includes starting the stream pointer control module A, copying module A, discarding module A, or partially reliable retransmission module A.

[0036] Furthermore, in step S5.4, when the switching action is selected to start the flow pointer control module A, step S5.5 is executed where the flow pointer control module A calculates the spacing that the data in each path should maintain. , For path The round-trip latency of transmission, For path The round-trip latency of transmission, For path The congestion window, based on the data spacing between path pairs, sets a send pointer for each path. Each path is based on Data is sent in parallel.

[0037] Furthermore, in step S5.4, when the switching action is selected as starting copy module A, step S5.6 is executed where copy module A performs copying based on each video frame. Type The system copies packets carrying critical frame data and redundantly schedules these packets to all available paths. Non-critical frame data is not copied.

[0038] Furthermore, in step S5.4, when the switching action is selected as activating the discard module A, step S5.7 is executed whereby the discard module A discards data based on each video frame. Type Messages carrying non-critical frame data will be discarded.

[0039] Furthermore, in step S5.4, when the switching action is selected as starting the partial reliable retransmission module A, step S5.8 is executed whereby the partial reliable retransmission module A performs the switching based on each video frame. Type If a message carrying critical frame data is lost, it will be retransmitted; if non-critical frame data is lost, it will not be retransmitted.

[0040] Furthermore, in step six, the model parameters are updated with the current features and rewards. When new data is rewritten to the send buffer, the process proceeds to step S5.1.

[0041] Beneficial effects:

[0042] The stream pointer control module, copying module, discarding module, and partially reliable retransmission module constructed in this invention can effectively cope with network conditions such as out-of-order packets, high packet loss rate, insufficient bandwidth, and weak network conditions.

[0043] This invention employs an online learning method based on LinUCB, which can switch real-time video stream transmission strategies in a lightweight and adaptive manner according to the dynamic network environment.

[0044] This invention is an adaptive real-time streaming media transmission platform based on MPQUIC. It aggregates the bandwidth of multiple transmission paths, which can effectively improve transmission throughput and reduce the initial latency and re-buffering time of real-time streaming media transmission. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the overall process of the adaptive multipath real-time streaming media transmission method based on LinUCB of the present invention.

[0046] Figure 2 This is a logical structure diagram of the adaptive multipath real-time streaming media transmission system based on LinUCB according to the present invention.

[0047] Figure 3 This is a comparison chart of real-time streaming media rebuffering times for multipath transmission schemes in a real network environment.

[0048] Figure 4 This is a comparison chart of the initial latency of real-time streaming media using a multi-path transmission scheme in a real network environment. Detailed Implementation

[0049] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0051] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0052] The following combination Figures 1-4Specific embodiments of the present invention will be described in detail below. It should be understood that the specific embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the present invention.

[0053] This invention implements an adaptive multipath real-time streaming media transmission method and system based on the LinUCB algorithm. For example... Figure 1 As shown, the present invention includes the following steps:

[0054] Step 1: Construct a real-time streaming media multipath transmission system. This system uses the RTMP protocol for real-time streaming media push and the MPQUIC protocol for adaptive multipath transmission. The specific method is as follows:

[0055] The real-time streaming media multipath transmission system consists of a server and a client. The client deploys an RTMP client and an MPQUIC transmission client. The system's structure and the connections between modules are as follows: Figure 2 As shown. The RTMP client includes an RTMP handshake module A, a frame segmentation module A, a frame encapsulation module A, and a frame parsing module A; the MPQUIC transmission client includes a handshake module A and a session A, which maintains a path management module A, a data stream module A, a sending module A, a congestion control module A, a scheduler module A, and an acknowledgment module A. To achieve adaptive transmission, this invention implements a LinUCB module A, a partially reliable retransmission module A, a discard module A, a copy module A, a stream pointer control module A, a transmission monitoring module A, and a frame information evaluation module A in the MPQUIC protocol stack. The server deploys an RTMP server and an MPQUIC transmission server, where the RTMP server also includes a frame aggregation module B and a frame decapsulation module B; the MPQUIC transmission server session also maintains a receiving module B and a feedback module B.

[0056] The LinUCB module A receives path status information and video frame information collected by the transmission monitoring module A and the frame information evaluation module A as input status. It then calculates the output action using a model and selects one of the following modules based on the action: Partial Reliable Retransmission Module A, Drop Module A, Copy Module A, or Stream Pointer Control Module A. After each output action, the feedback module B feeds back the inter-frame interval of the video frames to the sending end's LinUCB module A to calculate the reward and updates the parameters according to the reward model.

[0057] Step 2: Handshake and establish a two-layer session message, establishing multiple paths. Details are as follows:

[0058] S2.1 The RTMP handshake module B of the RTMP server starts listening and calls the underlying MPQUIC handshake module B;

[0059] S2.2 The handshake module B of the MPQUIC server starts listening; this step includes S2.2.1:

[0060] S2.2.1 Handshake module B calls the UDP.Listen interface to start UDP listening;

[0061] S2.3 The RTMP handshake module A of the RTMP client starts the handshake process and calls the underlying MPQUIC handshake module A;

[0062] S2.4 Handshake module A initiates the handshake process, calling the underlying UDP.Dial interface. The interface accepts the server's IP address and port number, and sends a probe packet containing connection parameters, including the client's supported protocol version, the newly created connection ID, and the key. It then waits for a response packet from handshake module B to verify the validity of the server's certificate and key. Upon receiving the packet, server-side handshake module B establishes session B based on the parsed connection ID. After receiving the server's handshake response, the client establishes session A.

[0063] S2.5 To maintain this connection, Session A establishes Path Management Module A, Data Flow Module A, Sending Module A, Congestion Control Module A, Scheduler Module A, and Response Module A. The module establishment process for Session B is the same as that for Session A.

[0064] S2.6 Path management module B polls the network card address of the server and sends an ADDADDRESS frame to announce the multiple addresses of the local machine. Path management module A receives the ADDADDRESS frame sent by the peer and polls the network card address of the local machine to establish multiple paths between the end-to-end address pairs.

[0065] S2.7 Data stream module A establishes a data stream, data stream module B receives the data stream, and both parties send data on the established data stream. The client and server respectively return the established session and data stream connection to the upper-layer RTMP application;

[0066] In S2.8, RTMP handshake module A and RTMP handshake module B exchange handshake messages through the established data stream. First, RTMP handshake module A sends C0 and C1. C0 contains the RTMP version number, and C1 contains a timestamp and a random number. The timestamp occupies the first 4 bytes, followed by 1528 bytes of random number. The server responds with S0, S1, and S2. S0 contains the RTMP version number, confirming the version supported by the server. S1 contains the timestamp and random number sent by the client, as well as the server's own timestamp and random number. S2 contains a random number generated by the server to complete the handshake process. The client sends C2, which it generates after receiving S1. C2 typically contains the client's timestamp, the server's timestamp, and the server's random number.

[0067] Step 3: Model loading and initialization, loading the protocol stack model file Lin. This file stores the model parameters trained using the LinUCB algorithm. During transmission startup, LinUCB module A inputs the model parameters and loads them into the model. For each action... Initialize model parameters and ,in For one A dimensional matrix is ​​used to measure the observed state history information. For one A dimensional matrix is ​​used to measure observed reward history information. In this embodiment, the observed state dimension is... Take 8. For details of the status, see S5.1.

[0068] The model's output is an action. , , The action space specifically includes four actions: initiating the stream pointer control module A, initiating the copy module A, initiating the discard module A, and initiating the partially reliable retransmission module A. During the initial model training, then... Initialize to , for 3D identity matrix Initialize to , for Zero-dimensional matrix.

[0069] Step 4: Streaming media data transmission, as detailed below:

[0070] An S4.1 RTMP client reads a single frame of video data. Frame encapsulation module A parses the frame data structure and constructs the RTMP message header. Frame segmentation module A segments the data and fills the payload portion of the RTMP message. Then, it calls the Write interface to process the message data. Write to the data stream.

[0071] S4.2 Scheduler A determines the size of the congestion control window provided by congestion control module A. It will be retrieved from data flow module A The data of the specified size is used to fill data stream frames and encapsulate them into data packets. This is the maximum message size.

[0072] S4.3 Scheduler A, taking MinRTT as an example, collects the RTT of all paths and selects the path with the lowest RTT to schedule the message. The message is then sent to the link corresponding to that path.

[0073] S4.4 The server receives the message, parses the message content, extracts the data stream frames and fills them into the stream queue managed by the data stream module B.

[0074] The S4.5 RTMP server sequentially reads the next frame of RTMP messages from data stream module B. If the data that makes up the message has not yet arrived, the reading process is blocked and 4.5 is repeated; if the data has arrived, the frame decapsulation module B reads the data from it.

[0075] Step 5: Adaptive transmission adjustment, including:

[0076] S5.1 Transmission monitoring module A collects path information. The acquisition time is when new data is written to the transmit buffer, denoted as time 1. ,in This represents the total number of paths. ,in, For path exist The round-trip time (RTT) of transmission at any given moment. For path exist Congestion window at any time, For path exist The amount of data being sent at any given moment. For path exist Packet loss rate at any given moment.

[0077] S5.2 Frame Information Evaluation Module A acquires information from video frames. ,in For frame rate, For real-time bitrate, The current frame size. The current frame type.

[0078] S5.3 Combines the path information and video frame information collected in S5.1 and S5.2 as features; The characteristics of time are denoted as The features are input into LinUCB module A. For example, in this embodiment, in Two transmission paths are used. Path 1 has a round-trip time of 50 milliseconds, a congestion window size of 2000, 500 data items being transmitted, and a packet loss rate of 0. Path 2 has a round-trip time of 70 milliseconds, a congestion window size of 1000, 600 data items being transmitted, and a packet loss rate of 0.1%. The frame rate is 30 fps, the real-time bitrate is 500 Kbps, the frame size is 100 KB, and the frame type is keyframe. , , , .

[0079] S5.4 According to parameters Features of the input Calculate ,in, Used to measure observed selection actions Status history information, Used to measure observed selection actions Reward history information, To The inverse matrix, These are the parameters in the ridge regression algorithm. Measured in Select action in the state The expected reward. To increase the model's adaptive exploration capability, in The calculation of the upper confidence interval has been added. ,in These are hyperparameters. According to... The user selects a switching action, choosing the one that maximizes the expected reward. The action space includes starting Stream Pointer Control Module A (starting S5.5), Copy Module A (starting S5.6), Drop Module A (starting S5.7), or Partial Reliable Retransmission Module A (starting S5.8). Stream Pointer Control Module A calculates the spacing between data packets to address out-of-order issues; this module is started by default. Copy Module A copies keyframe data packets and starts when the packet loss rate exceeds a set first threshold. Drop Module A starts when bandwidth is below a set second threshold, dropping packets carrying non-keyframe data. Partial Reliable Retransmission Module A starts in weak network scenarios, performing differentiated retransmissions. A weak network scenario refers to a network data transmission rate below a set third threshold.

[0080] S5.5 Stream pointer control module A calculates the spacing that the data should maintain for each path. , For path The round-trip latency of transmission, For path The round-trip latency of transmission, For path The congestion window, based on the data spacing between path pairs, sets a send pointer for each path. Each path is based on Parallel data transmission. Unlike traditional methods that schedule data sequentially across multiple paths, this module calculates the difference in the amount of data that should be transmitted between paths using latency and congestion windows. It then divides the data transmission according to the arrival time at the receiver, reducing out-of-order packets and ensuring that packets arrive at the receiver in the correct order.

[0081] S5.6 Copy module A copies each video frame. Type This module replicates packets carrying keyframe data and redundantly schedules these packets across all available paths. Non-keyframe data is not replicated. This module addresses the issue of significant video quality degradation due to keyframe data loss in high-packet-loss network environments by redundantly replicating high-priority data, effectively protecting it.

[0082] S5.7 Discard Module A based on each video frame Type This module discards packets carrying non-critical frame data. It addresses network scenarios with insufficient bandwidth by proactively discarding non-critical data to reserve more bandwidth for higher-priority critical frame data, thus providing better streaming media transmission quality.

[0083] S5.8 Partial Reliable Retransmission Module A, based on each video frame... Type The module retransmits lost packets carrying critical frame data, but does not retransmit lost packets carrying non-critical frame data. This module addresses scenarios where bandwidth and priority are insufficient. By implementing differentiated retransmission settings for critical and non-critical frame data, it improves the reliability of high-priority data without incurring significant bandwidth overhead.

[0084] Step 6: Calculate the reward and update the model. Based on... ,as well as Calculate the penalty items separately. and rewards , For video frames Number of bytes of data received For video frames Total number of data bytes. The threshold for determining video frame timeout is set to 200, based on the general video frame timeout setting. To receive video frames The time interval with the previous frame is obtained by subtracting the arrival interval timestamps between video frames from the time interval timestamps obtained by the feedback module B. The feedback module B encapsulates this time interval into a response frame and sends it back to the sending end. For video frames The types are divided into keyframes and non-keyframes. Weights are assigned to different types of video frames, with keyframes set to 1 and non-keyframes set to 0.7. Each video frame... The reward is Since the actions taken have a lasting impact on the network state, this invention calculates a persistent reward. From video frames Initially, the last three video frames were calculated, resulting in the video frame count. The accumulated reward is used as the current video frame. The reward. After calculating the reward at each time step, use the features of the current time step. and rewards For each action Model parameters and Update , When new data is rewritten to the send buffer, proceed to S5.1.

[0085] Step 7: Complete the real-time streaming media transmission.

[0086] The following technical effects can be achieved by using this invention:

[0087] (1) The stream pointer control module, copy module, discard module and partially reliable retransmission module constructed in the fifth step of the present invention can effectively deal with network conditions such as out-of-order packets, high packet loss rate, insufficient bandwidth and weak network.

[0088] (2) The fifth and sixth steps of the present invention adopt the online learning method based on LinUCB, which provides a lightweight adaptive way to switch the real-time video stream transmission strategy according to the dynamic network environment.

[0089] The MPQUIC-based adaptive real-time streaming media transmission platform implemented in this invention aggregates the bandwidth of multiple transmission paths, which can effectively improve transmission throughput and reduce the initial latency and re-buffering time of real-time streaming media transmission.

[0090] Figure 3The invention demonstrates that, in real-world network scenarios employing both 4G and WiFi transmission paths, when streaming video in real-time using Sintel, the rebuffering time is reduced by 97.6%, 74.1%, 95.0%, and 73.8% compared to MPTCP, MPQUIC, Peekaboo, and DQN*, respectively. When streaming video in real-time using SS, the rebuffering time is reduced by 89.0%, 26.8%, 62.5%, and 85.0% compared to MPTCP, MPQUIC, Peekaboo, and DQN*, respectively. MPTCP and MPQUIC are two commonly used multipath transmission schemes, Peekaboo is a multipath transmission scheme based on online learning-based path switching, and DQN* is a multipath transmission scheme based on the deep reinforcement learning algorithm DQN.

[0091] Figure 4 The invention demonstrates that, in real-world network scenarios employing both 4G and WiFi transmission paths, when streaming video in real-time using Sintel, the initial latency of the proposed invention is reduced by 47.7%, 41.1%, 18.5%, and 27.1% compared to MPTCP, MPQUIC, Peekaboo, and DQN*. When streaming video in real-time using SS, the initial latency of the proposed invention is reduced by 66.1%, 44.7%, 37.3%, and 47.0% compared to MPTCP, MPQUIC, Peekaboo, and DQN*. MPTCP and MPQUIC are two commonly used multipath transmission schemes, Peekaboo is a multipath transmission scheme based on online learning-based path switching, and DQN* is a multipath transmission scheme based on the deep reinforcement learning algorithm DQN.

[0092] Any process or method described in the flowcharts of this invention or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, achievable on any computer-readable medium for use by an instruction execution system, apparatus, or device. The computer-readable medium can be any medium containing a program for storage, communication, propagation, or transmission for use by an execution system, apparatus, or device, including read-only memory, magnetic disks, or optical disks.

[0093] In the description of this specification, references to terms such as "embodiment," "example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, those skilled in the art can combine or combine the different embodiments or examples described in this specification and the features therein without causing contradiction.

[0094] While embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and alterations to the above embodiments within the scope of the present invention.

Claims

1. An adaptive multipath real-time streaming media transmission method based on LinUCB, characterized in that, The method includes the following steps: Step 1: Construct a real-time streaming media multipath transmission system. This system uses the RTMP protocol for real-time streaming media push and the MPQUIC protocol for adaptive multipath transmission. The transmission system includes a partially reliable retransmission module A, a discard module A, a copy module A, and a stream pointer control module A. Step 2: Establish two-layer session messages and create multiple paths; Step 3: Model loading and initialization, loading the protocol stack model file Lin, and configuring it for each action. Initialize model parameters and ;in For one A dimensional matrix is ​​used to measure the observed state history information. For one A dimensional matrix is ​​used to measure observed reward history information. The observed state dimension; Step 4: Streaming media data transmission; Step 5: Adaptive transmission adjustment; select the switching action according to the switching strategy, and select to start the stream pointer control module A, copy module A, discard module A or partial reliable retransmission module A according to the switching instruction; Step 6: Calculate the reward and update the model; Step 7: Complete real-time streaming media transmission; In the second step, the two-layer session messages include: a session establishment message between the RTMP client and the RTMP server, and a session establishment message between the MPQUIC transport client and the MPQUIC transport server; and the establishment of a multipath between the MPQUIC transport client and the MPQUIC transport server. The fifth step includes: S5.1 Transmission monitoring module A collects path information. The collection time is when new data is written to the transmission buffer, denoted as time [time]. , ,in This represents the total number of paths. ,in, For path exist The round-trip time of transmission at any moment, For path exist Congestion window at any time, For path exist The number of messages sent at any given time. For path exist Packet loss rate at any given moment; S5.2 Frame Information Evaluation Module A acquires information from video frames. ,in For frame rate, For real-time bitrate, The current frame size. The current frame type; S5.3 The path information and video frame information collected in S5.1 and S5.2 are combined as features; the features are input into LinUCB module A, and LinUCB module A outputs a decision based on maximizing the expected reward. S5.4 Select the switching action based on the decision output by LinUCB module A; the action space includes starting the stream pointer control module A, copying module A, discarding module A, or partially reliable retransmission module A; Step six includes: according to ,as well as Calculate the penalty items separately. and rewards , For video frames Number of bytes of data received For video frames Total number of data bytes To determine the threshold for video frame timeout, To receive video frames The time interval between the previous frame, For video frames type Weights are assigned to different types of video frames; rewards are used. For each action Model parameters and Update; This is the current video frame.

2. The adaptive multipath real-time streaming media transmission method based on LinUCB according to claim 1, characterized in that, The real-time streaming media multipath transmission system consists of a server and a client. The client deploys an RTMP client and an MPQUIC transmission client. The RTMP client includes RTMP handshake module A, frame segmentation module A, frame encapsulation module A, and frame parsing module A; the MPQUIC transport client includes MPQUIC handshake module A and session A, which maintains: path management module A, data stream module A, sending module A, congestion control module A, scheduler module A, and acknowledgment module A.

3. The adaptive multipath real-time streaming media transmission method based on LinUCB according to claim 1, characterized in that, The second step includes the following sub-steps: S2.1 The RTMP handshake module B of the RTMP server starts listening and calls the underlying MPQUIC handshake module B; S2.2 MPQUIC handshake module B enables listening; S2.3 The RTMP handshake module A of the RTMP client starts the handshake process and calls the underlying MPQUIC handshake module A; S2.4 MPQUIC handshake module A begins the handshake process, calling the underlying UDP.Dial interface. The interface takes the server's IP address and port number as input, sends a probe packet containing connection parameters, including the protocol version supported by the client, the newly established connection ID, and the key. It then waits for the response packet from MPQUIC handshake module B to verify the validity of the server's certificate and key. After receiving the message, MPQUIC handshake module B establishes session B based on the parsed connection ID; after receiving the server's handshake response, the client establishes session A. S2.5 To maintain this connection, Session A establishes Path Management Module A, Data Stream Module A, Sending Module A, Congestion Control Module A, Scheduler Module A, and Acknowledgment Module A; To maintain this connection, Session B establishes Path Management Module B, Data Stream Module B, Receiving Module B, and Acknowledgment Module B. S2.6 Path management module B polls the network card address of the server and sends an ADDADDRESS frame to announce the multiple addresses of the local machine. Path management module A receives the ADDADDRESS frame sent by the peer and polls the network card address of the local machine to establish multiple paths between the end-to-end address pairs. S2.7 Data stream module A establishes a data stream, data stream module B receives the data stream, and both parties send data on the established data stream; The client and server will each establish a data stream and return it to the upper-layer RTMP application; S2.8 RTMP handshake module A and RTMP handshake module B send handshake messages to each other through the established data stream.

4. The adaptive multipath real-time streaming media transmission method based on LinUCB according to claim 1, characterized in that, The fourth step includes: An S4.1 RTMP client reads a single frame of video data. Frame encapsulation module A parses the frame data structure and constructs the RTMP message header. Frame segmentation module A segments the data and fills the payload portion of the RTMP message. Then, it calls the Write interface to send the message data. Write data to the data stream; S4.2 Scheduler A determines the size of the congestion control window provided by congestion control module A. It will be retrieved from data flow module A The data of the specified size is used to fill data stream frames and encapsulate them into data packets. This is the maximum message size; S4.3 Scheduler A collects the round-trip time (RTT) of all paths and selects the path with the lowest RTT to schedule the message; the message is sent to the link corresponding to the path. S4.4 The server receives the message, parses the message content, extracts the data stream frames and fills them into the stream queue managed by the data stream module B; The S4.5 RTMP server sequentially reads the next frame of RTMP messages from data stream module B. When the data that makes up the message has not yet arrived, the reading process is blocked and S4.5 is repeated; when the data arrives, the frame decapsulation module B reads the data from it.

5. The adaptive multipath real-time streaming media transmission method based on LinUCB according to claim 1, characterized in that, When the switching action selected in step S5.4 is to start the flow pointer control module A, step S5.5 is executed: the flow pointer control module A calculates the spacing that the data in each path should maintain. , For path The round-trip latency of transmission, For path The round-trip latency of transmission, For path The congestion window, based on the data spacing between path pairs, sets a send pointer for each path. Each path is based on Data is sent in parallel.

6. The adaptive multipath real-time streaming media transmission method based on LinUCB according to claim 1, characterized in that, When the switching action selected in step S5.4 is to start copy module A, step S5.6 is executed: Copy module A copies each video frame... Type The system copies packets carrying critical frame data and redundantly schedules these packets to all available paths. Non-critical frame data is not copied.

7. The adaptive multipath real-time streaming media transmission method based on LinUCB according to claim 1, characterized in that, When the switching action selected in step S5.4 is to activate the discard module A, step S5.7 is executed: the discard module A, based on each video frame... Type Messages carrying non-critical frame data will be discarded.

8. The adaptive multipath real-time streaming media transmission method based on LinUCB according to claim 1, characterized in that, When the switching action selected in step S5.4 is to activate the partially reliable retransmission module A, step S5.8 is executed: the partially reliable retransmission module A performs a certain number of video frames. Type If a message carrying critical frame data is lost, it will be retransmitted; if non-critical frame data is lost, it will not be retransmitted.

Images