Adaptive multi-path real-time streaming media transmission method based on LinUCB

Abstract

The invention discloses an adaptive multi-path real-time streaming media transmission method based on LinUCB. The method comprises the following steps: constructing a real-time streaming media multi-path transmission system; handshake is carried out, two layers of session messages are established, and multiple paths are established; loading and initializing the model; streaming media data transmission; self-adaptive transmission adjustment is carried out; calculating rewards and updating the model; and real-time streaming media transmission is completed. According to the on-line learning method based on LinUCB, real-time video stream transmission strategies can be switched in a lightweight self-adaptive mode according to a dynamic network environment; according to the adaptive real-time streaming media transmission platform based on MPQUIC, the bandwidths of a plurality of transmission paths are aggregated, so that the transmission throughput can be effectively improved, and the first opening time delay and the re-buffering time of real-time streaming media transmission are reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer network transmission control, in particular to a LinUCB-based adaptive multi-path real-time streaming media transmission method. Background Art

[0002] In recent years, live streaming applications have become an integral part of daily life, accounting for 74% of all mobile traffic. Live streaming services encompass diverse categories, including live product broadcasts, remote monitoring, and VR / AR. To ensure a positive user experience for live streaming applications, high-bandwidth, low-latency network transmission services are essential. However, network congestion, latency, and jitter, as well as high-bitrate live streaming that exceeds the current link bandwidth, can cause streaming to become lag-prone and blurry.

[0003] Today's real-time streaming protocols, such as the mainstream RTMP protocol, are largely based on TCP. However, TCP requires 1-3 RTTs to complete the handshake, making it unsuitable for real-time streaming applications that require high latency. Furthermore, TCP is implemented in kernel mode, and optimizing protocol functionality requires adjustments to the entire operating system kernel. The QUIC protocol, an extension of UDP, effectively reduces handshake latency. Furthermore, QUIC's user-mode implementation facilitates the expansion of protocol functionality.

[0004] The multipath transport protocol improves transmission bandwidth and reliability by establishing multiple paths between end-to-end connections. Combined with the characteristics of the QUIC protocol, the multipath QUIC protocol can support flexible protocol feature expansion and reduce initial latency, providing a foundation for optimizing the user experience of real-time streaming applications.

[0005] To further ensure the quality of service and user experience of real-time streaming applications, transmission protocols need to be improved. However, optimizing the protocols for network scenarios and streaming characteristics has become a key issue. This paper uses the lightweight LinUCB algorithm, which can switch between four proposed transmission optimization modules. It can effectively address the degradation of user experience in real-time streaming under conditions of "high packet loss", "insufficient bandwidth", and "weak network conditions". Summary of the Invention

[0006] In view of the problems existing in the prior art, the purpose of the present invention is to provide an adaptive multi-path real-time streaming media transmission method based on LinUCB, which can not only reduce the rebuffering time of real-time streaming media transmission, but also reduce streaming media delay and initial delay.

[0007] To achieve the above object, the present invention provides an adaptive multi-path real-time streaming media transmission method based on LinUCB, the method comprising the following steps: Step 1: Build a real-time streaming media multi-path transmission system. This system uses the RTMP protocol for real-time streaming media push and the MPQUIC protocol for adaptive multi-path transmission. The transmission system includes a partial reliable retransmission module A, a discard module A, a replication module A, and a stream pointer control module A. Step 2: Handshake and establish two-layer session messages to establish multipath; Step 3: Model loading and initialization, load the protocol stack model file Lin, for each action , initialize the model parameters and ;in For one dimensional matrix, which measures the observed state history information, For one dimensional matrix, which measures the observed reward history information, is the observed state dimension; Step 4: Streaming data transmission; Step 5: Adaptive transmission adjustment; select the switching action according to the switching strategy, and select the start of the flow pointer control module A, the copy module A, the discard module A or the partially reliable retransmission module A according to the switching instruction; Step 6: Calculate rewards and update the model; Step 7: Complete the live streaming transmission.

[0008] Furthermore, the real-time streaming media multi-path transmission system consists of a server and a client, and the client deploys an RTMP client and an MPQUIC transmission client; the RTMP client includes an RTMP handshake module A, a frame segmentation module A, a frame encapsulation module A, and a frame parsing module A; the MPQUIC transmission client includes: a handshake module A and a session A, and the session A maintains: a path management module A, a data stream module A, a sending module A, a congestion control module A, a scheduler module A, and a response module A.

[0009] Furthermore, the second step includes the following sub-steps: S2.1 The RTMP handshake module B of the RTMP server starts listening and calls the handshake module B of the underlying MPQUIC; S2.2 The handshake module B of the MPQUIC server starts listening; S2.3 The RTMP handshake module A of the RTMP client starts the handshake process and calls the handshake module A of the underlying MPQUIC; S2.4 Handshake module A begins the handshake process by calling the underlying UDP.Dial interface, passing in the server's IP address and port number, and sending a probe message containing connection parameters, including the client's supported protocol version, the newly created connection ID, and the key. It then waits for a response message from handshake module B to verify the validity of the server's certificate and key. After receiving the message, handshake module B on the server parses the connection ID and establishes session B. After receiving the server's handshake response, the client establishes session A. S2.5 To maintain this connection, Session A establishes Path Management Module A, Data Flow Module A, Send Module A, Congestion Control Module A, Scheduler Module A, and Response Module A. The module establishment process for Session B is the same as for Session A. S2.6 Path Management Module B polls the server's network card address and sends an ADDADDRESS frame to announce its multiple addresses. Path Management Module A receives the ADDADDRESS frame and polls its own network card address, establishing multiple paths between the end-to-end address pairs. S2.7 Data stream module A establishes a data stream, data stream module B receives the data stream, and both parties send data on the established data stream. The client and server each return the established session and data stream connection to the upper-layer RTMP application. S2.8 RTMP handshake module A and RTMP handshake module B send handshake messages to each other through the established data stream.

[0010] Furthermore, the fourth step includes: S4.1 RTMP client reads a frame of video data , the frame encapsulation module A parses the frame data structure and forms the RTMP message header, the frame segmentation module A segments the data and fills the payload part of the RTMP message, and then calls the Write interface to write the message data Write data stream; S4.2 Scheduler A determines the congestion control window size provided by congestion control module A. , will be taken out from data flow module A The data of the size is filled in the data stream frame and encapsulated into a data message. is the maximum message size; S4.3 Scheduler A collects the RTTs of all paths and selects the path with the lowest RTT to schedule the message. The message is then sent to the link corresponding to the path. S4.4 The server link receives the message, parses the message content, extracts the data stream frames, and places them into the stream queue managed by data stream module B. S4.5 The RTMP server reads the next frame of RTMP message from data stream module B in sequence If the data constituting the message has not arrived yet, the reading process is blocked and S4.5 is repeated; if the data has arrived, the frame decapsulation module B reads the data from it.

[0011] Furthermore, the fifth step includes: S5.1 Transmission monitoring module A collects path information. The collection time is when new data is written into the send buffer, which is recorded as time , ,in is the total number of paths, ,in, For path exist RTT at the moment, For path exist The congestion window at that moment, For path exist The number of messages being sent at the moment, For path exist Packet loss rate at the moment; S5.2 Frame Information Evaluation Module A obtains information about video frames ,in is the frame rate, is the real-time bit rate, is the current frame size, is the current frame type; S5.3 combines the path information and video frame information collected by S5.1 and S5.2 as features; S5.4 Select a switching action based on the calculation; the action space includes starting the flow pointer control module A, copying the module A, discarding the module A, or partially reliable retransmission module A.

[0012] Furthermore, when the switching action selected in step S5.4 is to start the flow pointer control module A, step S5.5 is executed to calculate the spacing that the data of each path should maintain. , For path The round-trip transmission delay, For path The round-trip transmission delay, For path The congestion window is based on the data spacing between the path pairs, and a sending pointer is set for each path. , each path is based on Send data in parallel.

[0013] Furthermore, when the switching action selected in step S5.4 is to start the copy module A, step S5.6 is executed to copy the module A according to each video frame. Type , copy the messages carrying key frame data and redundantly schedule these messages to all available paths. For non-key frame data, no copy is performed.

[0014] Further, when the switching action selected in step S5.4 is to start the discard module A, step S5.7 is executed to discard the module A according to each video frame. Type , discarding the message carrying non-key frame data.

[0015] Further, when the switching action selected in step S5.4 is to start the partially reliable retransmission module A, step S5.8 is executed in which the partially reliable retransmission module A starts the partially reliable retransmission module A according to each video frame. Type , the message carrying key frame data is retransmitted after loss, and non-key frame data is not retransmitted after loss.

[0016] Furthermore, in the sixth step, the model parameters are updated using the current features and rewards. When the new data is rewritten into the send buffer, go to S5.1.

[0017] Beneficial effects: The flow pointer control module, replication module, discard module and partially reliable retransmission module constructed in the present invention can effectively deal with network conditions such as message disorder, high packet loss rate, insufficient bandwidth and weak network.

[0018] The present invention adopts an online learning method based on LinUCB, which can switch the real-time video stream transmission strategy in a lightweight and adaptive manner according to the dynamic network environment.

[0019] The present invention is based on the adaptive real-time streaming media transmission platform of MPQUIC, which aggregates the bandwidth of multiple transmission paths, can effectively improve the transmission throughput, and reduce the initial delay and rebuffering time of real-time streaming media transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of the overall flow of the adaptive multi-path real-time streaming media transmission method based on LinUCB of the present invention; Figure 2 This is a logical structure diagram of the adaptive multi-path real-time streaming media transmission system based on LinUCB of the present invention; Figure 3 This is a comparison chart of the real-time streaming media rebuffering time of multi-path transmission solutions in a real network environment; Figure 4This is a comparison chart of the first-open delay of real-time streaming media in a real network environment using multi-path transmission solutions. DETAILED DESCRIPTION

[0021] The following will clearly and completely describe the technical solution of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0022] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0023] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0024] The following combination Figure 1-Figure 4 The specific embodiments of the present invention are described in detail. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0025] The present invention implements an adaptive multi-path real-time streaming media transmission method and system based on the LinUCB algorithm. Figure 1 As shown, the present invention includes the following steps: Step 1: Build a real-time streaming multi-path transmission system. This system uses the RTMP protocol for real-time streaming and the MPQUIC protocol for adaptive multi-path transmission. The specific method is: The real-time streaming multi-path transmission system consists of a server and a client. The client deploys an RTMP client and an MPQUIC transmission client. The system composition and the connection relationship between the modules are as follows: Figure 2As shown. The RTMP client includes an RTMP handshake module A, a frame segmentation module A, a frame encapsulation module A, and a frame parsing module A; the MPQUIC transmission client includes a handshake module A and a session A, wherein session A maintains a path management module A, a data stream module A, a sending module A, a congestion control module A, a scheduler module A, and a response module A. To achieve adaptive transmission, the present invention implements a LinUCB module A, a partially reliable retransmission module A, a discarding module A, a copying module A, a stream pointer control module A, a transmission monitoring module A, and a frame information evaluation module A in the MPQUIC protocol stack. The server deploys an RTMP server and an MPQUIC transmission server, wherein the RTMP server also includes a frame aggregation module B and a frame decapsulation module B. The MPQUIC transmission server session also maintains a receiving module B and a feedback module B.

[0026] LinUCB Module A receives path state information and video frame information collected by Transmission Monitoring Module A and Frame Information Evaluation Module A as input. It calculates output actions using the model and then selects one of the following: Partial Reliable Retransmission Module A, Discard Module A, Copy Module A, or Stream Pointer Control Module A. After each output action, Feedback Module B feeds the interframe interval of the video frame back to the sending LinUCB Module A to calculate a reward and update its parameters based on the reward model.

[0027] Step 2: Handshake and establish two-layer session messages to establish multi-path. The details are as follows: S2.1 The RTMP handshake module B of the RTMP server starts listening and calls the handshake module B of the underlying MPQUIC; S2.2 The MPQUIC server's handshake module B starts listening. This step includes S2.2.1: S2.2.1 Handshake module B calls the UDP.Listen interface to start UDP listening; S2.3 The RTMP handshake module A of the RTMP client starts the handshake process and calls the handshake module A of the underlying MPQUIC; S2.4 Handshake Module A begins the handshake process by calling the underlying UDP.Dial interface, passing in the server's IP address and port number, and sending a probe message containing connection parameters, including the client's supported protocol version, the newly created connection ID, and the key. It then waits for a response message from Handshake Module B to verify the validity of the server's certificate and key. After receiving the message, Handshake Module B on the server parses the connection ID and establishes Session B. After receiving the server's handshake response, the client establishes Session A. S2.5 To maintain this connection, Session A establishes Path Management Module A, Data Flow Module A, Send Module A, Congestion Control Module A, Scheduler Module A, and Response Module A. The module establishment process for Session B is the same as that for Session A. S2.6 Path Management Module B polls the server's network card address and sends an ADDADDRESS frame to announce its multiple addresses. Path Management Module A receives the ADDADDRESS frame and polls its own network card address, establishing multiple paths between the end-to-end address pairs. S2.7 Data stream module A establishes a data stream, data stream module B receives the data stream, and both parties send data on the established data stream. The client and server each return the established session and data stream connection to the upper-layer RTMP application. S2.8 RTMP Handshake Modules A and B exchange handshake messages over the established data stream. RTMP Handshake Module A first sends C0 and C1. C0 contains the RTMP version number, and C1 contains a timestamp and a random number. The timestamp occupies the first 4 bytes, followed by a 1528-byte random number. The server responds with S0, S1, and S2. S0 contains the RTMP version number, confirming the version supported by the server. S1 contains the timestamp and random number sent by the client, as well as the server's own timestamp and random number. S2 contains a random number generated by the server, completing the handshake. The client sends C2, which is generated after receiving S1. C2 typically contains the client's timestamp, the server's timestamp, and the server's random number.

[0028] Step 3: Model loading and initialization, load the protocol stack model file Lin. The file stores the model parameters trained based on the LinUCB algorithm and is stored in the Lin file. When the transmission starts, the LinUCB module A inputs the model parameters and loads them into the model. For each action , initialize the model parameters and ,in For one dimensional matrix, used to measure the observed state history information, For one dimensional matrix, used to measure the observed reward history information, is the observed state dimension. In this embodiment Select 8. For details on the status, see S5.1.

[0029] The output of the model is action , , is the action space, which includes 4 actions, namely starting the flow pointer control module A, starting the copy module A, starting the discard module A and starting the partial reliable retransmission module A. When the model is first trained, Initialized to , for dimensional identity matrix, Initialized to , for dimensional zero matrix.

[0030] Step 4: Streaming data transmission, as follows: S4.1 RTMP client reads a frame of video data , the frame encapsulation module A parses the frame data structure and forms the RTMP message header, the frame segmentation module A segments the data and fills the payload part of the RTMP message, and then calls the Write interface to write the message data Write to the data stream.

[0031] S4.2 Scheduler A determines the congestion control window size provided by congestion control module A. , will be taken out from data flow module A The data of the size is filled in the data stream frame and encapsulated into a data message. The maximum message size.

[0032] S4.3 Scheduler A, taking MinRTT as an example, collects the RTTs of all paths and selects the path with the lowest RTT to schedule the message. The message is then sent to the link corresponding to the path.

[0033] S4.4 The server link receives the message, parses the message content, extracts the data flow frame and fills it into the flow queue managed by the data flow module B.

[0034] S4.5 The RTMP server reads the next frame of RTMP message from data stream module B in sequence If the data constituting the message has not arrived yet, the reading process is blocked and 4.5 is repeated; if the data has arrived, the frame decapsulation module B reads the data from it.

[0035] Step 5: Adaptive transmission adjustment, including: S5.1 Transmission monitoring module A collects path information The acquisition time is when new data is written into the send buffer, recorded as time ,in is the total number of paths, ,in, For path exist The round-trip time (RTT) of the transmission at the time, For path exist The congestion window at that moment, For path exist The amount of data being sent at the moment, For path exist Packet loss rate at a given moment.

[0036] S5.2 Frame Information Evaluation Module A obtains information about video frames ,in is the frame rate, is the real-time bit rate, is the current frame size, The current frame type.

[0037] S5.3 combines the path information and video frame information collected by S5.1 and S5.2 as features; The characteristic of the moment is recorded as , input the features into LinUCB module A. For example, in this embodiment, Two paths are used for transmission at all times. The round-trip delay of path 1 is 50 milliseconds, the congestion window size is 2000, the amount of data being sent is 500, and the packet loss rate is 0; the round-trip delay of path 2 is 70 milliseconds, the congestion window size is 1000, the amount of data being sent is 600, and the packet loss rate is 0.1; the frame rate is 30fps, the real-time bit rate is 500Kbps, the frame size is 100KB, and the frame type is key frame. , , , .

[0038] S5.4 According to the parameters With the input features Calculate ,in, Used to measure the observed choice action Status history information, Used to measure the observed choice action Reward history information, For The inverse matrix of is a parameter in the ridge regression algorithm, Measured in In the state, select action In order to increase the adaptive exploration ability of the model, Added calculation of upper confidence interval in ,in is a hyperparameter. , select a switching action, and the selected action is the action that maximizes the expected reward. The action space includes starting the flow pointer control module A (starting S5.5), the copy module A (starting S5.6), the discard module A (starting S5.7), or the partially reliable retransmission module A (starting S5.8). Among them, the function of the flow pointer control module A is to calculate the spacing that should be maintained between data to solve the disorder problem. This module is started by default. The copy module A is used to copy key frame data packets. This module is started when the packet loss rate exceeds the set first threshold. The discard module A is started in a scenario where the bandwidth is lower than the set second threshold, and the payload non-key frame data packets are lost. The partially reliable retransmission module A is started in a weak network scenario and performs differentiated retransmission. A weak network scenario refers to a scenario where the network data transmission speed is lower than the set third threshold.

[0039] S5.5 Stream Pointer Control Module A calculates the spacing that the data on each path should maintain , For path The round-trip transmission delay, For path The round-trip transmission delay, For path The congestion window is based on the data spacing between the path pairs, and a sending pointer is set for each path. , each path is based on Parallel data transmission. Unlike traditional methods that schedule data sequentially across multiple paths, this module uses latency and congestion windows to calculate the difference in the amount of data that should be sent between paths. It then divides the data transmission based on the time it takes to arrive at the receiving end, reducing packet out-of-order and ensuring that packets arrive at the receiving end in order.

[0040] S5.6 Copy module A according to each video frame Type , copies packets carrying key frame data and redundantly schedules these packets to all available paths. Non-key frame data is not copied. This module addresses the problem that key frame data loss can cause significant loss of video quality in high packet loss network conditions. By redundantly copying high-priority data, it effectively protects high-priority data.

[0041] S5.7 Discard module A based on each video frame Type , discarding packets carrying non-critical frame data. This module responds to network scenarios with insufficient bandwidth. When bandwidth is insufficient, it actively discards non-critical data to reserve more bandwidth for higher-priority critical frame data, providing better streaming media transmission quality.

[0042] S5.8 Partially reliable retransmission module A based on each video frame Type , messages carrying key frame data are retransmitted after loss, while non-key frame data is not retransmitted after loss. This module can be used in scenarios where both bandwidth and priority are insufficient. By setting differentiated retransmission settings for key frame and non-key frame data, the reliability of high-priority data is improved without incurring large bandwidth overhead.

[0043] Step 6: Calculate the reward and update the model. ,as well as , calculate the penalty terms respectively and rewards , For video frames The number of data bytes received, For video frames The total number of data bytes. The threshold for determining video frame timeout is set to 200, which is based on the general video frame timeout setting. To receive video frames The time interval with the previous frame is obtained by subtracting the arrival interval timestamps between video frames by feedback module B. Feedback module B encapsulates the time interval into a response frame and sends it back to the transmitter. For video frames The type is divided into keyframes and non-keyframes. The weights assigned to different types of video frames are set to 1 for key frames and 0.7 for non-key frames. The reward is Since the actions taken will have a lasting impact on the network state, the present invention calculates the lasting reward , will be drawn from the video frame At the beginning, the last three video frames are calculated to get the video frame The reward accumulation of the current video frame Rewards. After calculating the reward at the moment, use the current moment’s features and rewards For each action Model parameters and To update, , When new data is rewritten into the send buffer, go to S5.1.

[0044] Step 7: Complete the live streaming transmission.

[0045] The present invention can achieve the following technical effects: (1) The flow pointer control module, replication module, discard module, and partially reliable retransmission module constructed in the fifth step of the present invention can effectively deal with network conditions such as message disorder, high packet loss rate, insufficient bandwidth, and weak network.

[0046] (2) The fifth and sixth steps of the present invention adopt an online learning method based on LinUCB, which provides a lightweight adaptive way to switch the real-time video stream transmission strategy according to the dynamic network environment.

[0047] The MPQUIC-based adaptive real-time streaming media transmission platform implemented by the present invention aggregates the bandwidth of multiple transmission paths, which can effectively improve the transmission throughput and reduce the initial delay and rebuffering time of real-time streaming media transmission.

[0048] Figure 3 The study demonstrated that, in a real-world network scenario using both 4G and WiFi transmission paths, the rebuffering time of the present invention decreased by 97.6%, 74.1%, 95.0%, and 73.8% when streaming real-time sintel video, compared to MPTCP, MPQUIC, Peekaboo, and DQN*. When streaming real-time ss video, the rebuffering time of the present invention decreased by 89.0%, 26.8%, 62.5%, and 85.0% compared to MPTCP, MPQUIC, Peekaboo, and DQN*. MPTCP and MPQUIC are two commonly used multipath transmission schemes, Peekaboo is a multipath transmission scheme based on online learning path switching, and DQN* is a multipath transmission scheme based on the deep reinforcement learning algorithm DQN.

[0049] Figure 4 The demonstration demonstrates that, in a real-world network scenario using both 4G and WiFi transmission paths, the present invention reduces the initial latency by 47.7%, 41.1%, 18.5%, and 27.1% when streaming real-time video sintel, compared to MPTCP, MPQUIC, Peekaboo, and DQN*. When streaming real-time video ss, the present invention reduces the initial latency by 66.1%, 44.7%, 37.3%, and 47.0% when streaming real-time video ss. MPTCP and MPQUIC are two commonly used multipath transmission schemes, Peekaboo is a multipath transmission scheme based on online learning path switching, and DQN* is a multipath transmission scheme based on the deep reinforcement learning algorithm DQN.

[0050] Any process or method described in the flowchart of the present invention or in other ways herein can be understood as representing a module, segment or portion of code including one or more executable instructions for implementing specific logical functions or process steps, which can be implemented in any computer-readable medium for use by an instruction execution system, device or apparatus. The computer-readable medium can be any medium that stores, communicates, propagates or transmits a program for use by an execution system, device or apparatus, including read-only memory, magnetic disk or optical disk, etc.

[0051] Throughout this specification, reference to terms such as "embodiment" and "example" indicates that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, those skilled in the art may combine or integrate different embodiments or examples described in this specification, as well as features therein, without creating any inconsistency.

[0052] Although the above content has shown and described the embodiments of the present invention, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present invention. Ordinary technicians in this field can perform update operations such as changes, modifications, replacements and variations on the above embodiments within the scope of the present invention.

Claims

1. An adaptive multi-path real-time streaming media transmission method based on LinUCB, characterized in that: The method comprises the following steps: Step 1: Build a real-time streaming multi-path transmission system. This system uses the RTMP protocol for real-time streaming push and the MPQUIC protocol for adaptive multi-path transmission. The transmission system includes a partial reliable retransmission module A, a discard module A, a replication module A, and a stream pointer control module A. Step 2: Handshake and establish two-layer session messages to establish multipath; Step 3: Model loading and initialization, load the protocol stack model file Lin, for each action , initialize the model parameters and ;in For one dimensional matrix, used to measure the observed state history information, For one dimensional matrix, used to measure the observed reward history information, is the observed state dimension; Step 4: Streaming data transmission; Step 5: Adaptive transmission adjustment; select the switching action according to the switching strategy, and select the start of the flow pointer control module A, the copy module A, the discard module A or the partially reliable retransmission module A according to the switching instruction; Step 6: Calculate rewards and update the model; Step 7: Complete the live streaming transmission.

2. The method for transmitting adaptive multi-path real-time streaming media based on LinUCB according to claim 1, characterized in that: The real-time streaming multi-path transmission system consists of a server and a client. The client deploys an RTMP client and an MPQUIC transmission client. The RTMP client includes the RTMP handshake module A, the frame segmentation module A, the frame encapsulation module A, and the frame parsing module A; the MPQUIC transmission client includes: the handshake module A and the session A, and the session A maintains: the path management module A, the data stream module A, the sending module A, the congestion control module A, the scheduler module A, and the response module A.

3. The method for transmitting adaptive multi-path real-time streaming media based on LinUCB according to claim 1, characterized in that: The second step includes the following sub-steps: S2.1 The RTMP handshake module B of the RTMP server starts listening and calls the handshake module B of the underlying MPQUIC; S2.2 The handshake module B of the MPQUIC server starts listening; S2.3 The RTMP handshake module A of the RTMP client starts the handshake process and calls the handshake module A of the underlying MPQUIC; S2.4 Handshake module A begins the handshake process by calling the underlying UDP.Dial interface, passing in the server's IP address and port number, and sending a probe message containing connection parameters, including the client's supported protocol version, the newly created connection ID, and the key. It then waits for a response message from handshake module B to verify the validity of the server's certificate and key. After receiving the message, the server's handshake module B establishes session B based on the parsed connection ID; after receiving the server's handshake response, the client establishes session A; S2.5 To maintain this connection, Session A establishes Path Management Module A, Data Flow Module A, Send Module A, Congestion Control Module A, Scheduler Module A, and Response Module A. The module establishment process for Session B is the same as for Session A. S2.6 Path Management Module B polls the server's network card address and sends an ADDADDRESS frame to announce its multiple addresses. Path Management Module A receives the ADDADDRESS frame and polls its own network card address, establishing multiple paths between the end-to-end address pairs. S2.7 Data stream module A establishes a data stream, data stream module B receives the data stream, and both parties send data on the established data stream; The client and server return the established session and data stream connection to the upper-layer RTMP application respectively; S2.8 RTMP handshake module A and RTMP handshake module B send handshake messages to each other through the established data stream.

4. The method for transmitting adaptive multi-path real-time streaming media based on LinUCB according to claim 1, characterized in that: The fourth step includes: S4.1 RTMP client reads a frame of video data , the frame encapsulation module A parses the frame data structure and forms the RTMP message header, the frame segmentation module A segments the data and fills the payload part of the RTMP message, and then calls the Write interface to write the message data Write data stream; S4.2 Scheduler A determines the congestion control window size provided by congestion control module A. , will be taken out from data flow module A The data of the size is filled in the data stream frame and encapsulated into a data message. is the maximum message size; S4.3 Scheduler A collects the round-trip delay (RTT) of all paths and selects the path with the lowest RTT to schedule the message. The message is then sent to the link corresponding to the path. S4.4 The server link receives the message, parses the message content, extracts the data stream frames, and places them into the stream queue managed by data stream module B. S4.5 The RTMP server reads the next frame of RTMP message from data stream module B in sequence When the data constituting the message have not arrived yet, the reading process is blocked and S4.5 is repeated; when the data arrives, the frame decapsulation module B reads the data from it.

5. The method for transmitting adaptive multi-path real-time streaming media based on LinUCB according to claim 1, characterized in that: The fifth step includes: S5.1 Transmission monitoring module A collects path information. The collection time is when new data is written into the send buffer, which is recorded as time , ,in is the total number of paths, ,in, For path exist The round-trip transmission delay at the time, For path exist The congestion window at that moment, For path exist The number of messages being sent at the moment, For path exist Packet loss rate at the moment; S5.2 Frame Information Evaluation Module A obtains information about video frames ,in is the frame rate, is the real-time bit rate, is the current frame size, is the current frame type; S5.3 combines the path information and video frame information collected by S5.1 and S5.2 as features; S5.4 Select a switching action based on the calculation; the action space includes starting the flow pointer control module A, copying the module A, discarding the module A, or partially reliable retransmission module A.

6. The method for transmitting adaptive multi-path real-time streaming media based on LinUCB according to claim 5, characterized in that: When the switching action selected in step S5.4 is to start the flow pointer control module A, step S5.5 is executed: the flow pointer control module A calculates the spacing that the data of each path should maintain. , For path The round-trip transmission delay, For path The round-trip transmission delay, For path The congestion window is based on the data spacing between the path pairs, and a sending pointer is set for each path. , each path is based on Send data in parallel.

7. The method for transmitting adaptive multi-path real-time streaming media based on LinUCB according to claim 5, characterized in that: When the switching action selected in step S5.4 is to start the copy module A, step S5.6 is executed: the copy module A starts the copy module A according to each video frame. Type , copy the messages carrying key frame data, and redundantly schedule these messages to all available paths. For non-key frame data, no copy is performed.

8. The method for transmitting adaptive multi-path real-time streaming media based on LinUCB according to claim 5, characterized in that: When the switching action selected in step S5.4 is to start the discard module A, step S5.7 is executed: the discard module A starts the discard module A according to each video frame. Type , discarding the message carrying non-key frame data.

9. The method for transmitting adaptive multi-path real-time streaming media based on LinUCB according to claim 5, characterized in that: When the switching action selected in step S5.4 is to start the partially reliable retransmission module A, step S5.8 is executed: the partially reliable retransmission module A starts the transmission of the video frame according to the video frame. Type , the message carrying key frame data is retransmitted after loss, and non-key frame data is not retransmitted after loss.

10. The method for adaptive multi-path real-time streaming media transmission based on LinUCB according to any one of claims 5 to 9, characterized in that: In the sixth step, the model parameters are updated with the current features and rewards. When the new data is rewritten into the send buffer, go to S5.1.

Images