An interactive transmission method and system for digitized live data

Abstract

The application is particularly a kind of interactive transmission method and system of digital live data, relating to streaming media transmission technical field, comprising: sliding window segmentation is carried out to dynamic interaction environment characteristic vector of continuous multiple collection periods to obtain time sequence characteristic sequence, after standardization processing is carried out to the time sequence characteristic sequence, input lightweight time sequence prediction model, output link state prediction confidence and predicted deterioration duration.The lightweight time sequence prediction model is deployed in the edge computing node in the application to predict the link deterioration trend, and when the prediction confidence reaches the trigger threshold, the forward error correction code redundancy upgrade and cache pre-population compensation operation are carried out in advance, which solves the technical problem that the existing passive retransmission strategy introduces additional round trip delay waiting and causes high video stuttering frequency under weak network conditions.

Description

Technical Field

[0001] This invention relates to the field of streaming media transmission technology, and in particular to an interactive transmission method and system for digital live streaming data. Background Technology

[0002] With the rapid development of online live streaming, live streaming scenarios that include real-time interactive features such as bullet comments, virtual gifts, live chat, and gesture-triggered effects place increasingly higher demands on the quality of the transmission link. Against this backdrop, streaming media transmission systems need to simultaneously ensure the continuous playback quality of the basic media stream and the low-latency delivery of interactive metadata.

[0003] Existing streaming media transmission solutions typically employ automatic retransmission request strategies to handle packet loss in the link and transmit interactive metadata and the underlying media stream together through the same transmission channel.

[0004] The aforementioned existing technical solutions have the following drawbacks:

[0005] The automatic retransmission request strategy only initiates the retransmission process after packet loss is detected, which introduces an additional recovery waiting time of at least one round-trip delay. During the continuous period of weak network, the accumulation of retransmission requests will cause the sending queue to be blocked, resulting in the superposition and amplification of video stuttering and interactive response delay.

[0006] Interactive metadata and basic media streams share the same transmission channel. When the network is congested, the two compete for limited bandwidth, which means that interactive metadata, which is very small in size but sensitive to latency, cannot be delivered first. This causes a synchronous interruption in the interactive layer experience, and in turn, under weak network conditions, users face the technical problem of video stuttering and loss of interactive layer responsiveness at the same time. Summary of the Invention

[0007] The purpose of this invention is to provide an interactive transmission method and system for digital live streaming data in order to solve the above-mentioned problems.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A method for interactive transmission of digital live streaming data, comprising:

[0010] Edge computing nodes periodically collect multi-dimensional network state parameters of the current session through lightweight monitoring probes, and then align the parameters of each dimension according to the timestamp to generate a dynamic interactive environment feature vector.

[0011] A time-series feature sequence is obtained by sliding window segmentation of the dynamic interactive environment feature vectors of multiple consecutive acquisition cycles. After standardization processing of the time-series feature sequence, it is input into a lightweight time-series prediction model, and the link state prediction confidence and predicted degradation duration are output.

[0012] The prediction confidence is compared with a preset trigger threshold. When the prediction confidence reaches the preset trigger threshold, a forward error correction code redundancy dynamic adjustment instruction is generated for the uplink based on the prediction confidence and the prediction degradation duration and sent to the streamer's push end to perform redundant data insertion. For the downlink, a cache pre-fill instruction is generated and a low-latency keyframe slice is pushed to the viewer's playback end.

[0013] The data stream of the current session is classified, and independent transmission tunnels are established for the basic media stream and the interactive metadata stream. A basic reliable transmission channel is allocated for the basic media stream, and a high-priority lightweight transmission tunnel is established for the interactive metadata stream, using a transmission strategy that combines fixed rate and redundant transmission.

[0014] Preferably, the multi-dimensional network state parameters are collected from three dimensions: uplink, downlink, and inter-edge link. The uplink is the link between the broadcaster's streaming end and the edge computing node, and the collected parameters include uplink packet loss rate, real-time round-trip latency, transmission bit rate, and signal strength fluctuation rate. The downlink is the link between the edge computing node and the viewer's playback end, and the collected parameters include downlink bandwidth, buffer fill level, and video stuttering frequency. The inter-edge link is the link between the edge computing node and the central cloud platform or other edge computing nodes, and the collected parameters include backbone network transmission jitter and concurrent stream throughput.

[0015] Preferably, the edge computing node further parses the business attribute parameters of the interactive data stream and integrates the business attribute parameters into the dynamic interactive environment feature vector; the business attribute parameters include the current live stream type identifier and the interaction density index; wherein, the current live stream type identifier is non-numerical classification data, and the current live stream type identifier is converted into a numerical encoded value through one-hot encoding; the interaction density index is obtained by calculating the number of concurrent bullet comments and the number of touch screen interactions per unit time.

[0016] Preferably, the lightweight time-series prediction model is a Long Short-Term Memory (LSTM) network model deployed on edge computing nodes after model pruning optimization. The LSM network model includes an attention layer before its output layer. This attention layer receives the hidden output sequence of the LSM network model, obtains the attention score for each time step by performing an inner product operation between the hidden state vector and the learnable query vector, normalizes the attention scores at each time step using Softmax to obtain attention weights, and uses these attention weights to perform a weighted summation of the hidden output sequence to obtain a context vector. The context vector outputs the prediction confidence and the prediction degradation duration through two independent output branches of a fully connected layer. The first output branch maps the output value to a closed interval between zero and one using a Sigmoid activation function, and the second output branch constrains the output value to a non-negative real number using a ReLU activation function.

[0017] Preferably, the forward error correction code redundancy dynamic adjustment instruction carries a forward error correction code redundancy adjustment ratio. The forward error correction code redundancy adjustment ratio is calculated by multiplying the base redundancy ratio by the value of an exponential function, where the exponent of the exponential function is the product of a scaling factor and the prediction confidence. The base redundancy ratio is the default proportion of redundant packets to the total original data packets maintained when the link state is stable. The scaling factor controls the growth rate of the forward error correction code redundancy adjustment ratio as the prediction confidence increases. When the predicted degradation duration exceeds a preset duration upper limit, the forward error correction code redundancy adjustment ratio takes the preset redundancy ratio upper limit value.

[0018] Preferably, the step of generating a cache pre-filling instruction for the downlink and pushing low-latency keyframe slices to the viewer's playback terminal includes: the edge computing node performing transcoding acceleration processing on the currently pushed video stream based on the predicted trend of the buffer fill level at the viewer's playback terminal, generating low-latency keyframe slices; embedding the low-latency keyframe slices into the regularly pushed image group data stream, and pushing them to the viewer's playback terminal; wherein, the transcoding acceleration processing refers to the edge computing node temporarily increasing the processing priority of the encoding thread to complete the encoding output of subsequent frame data in advance.

[0019] Preferably, when the link is in a degraded state, the edge computing node performs active frame dropping or spatial resolution reduction operations on the bitstream of non-interest regions in the video frames of the basic media stream, while preserving the complete bitstream of the region of interest; wherein, the region of interest includes the anchor portrait region, the subtitle region, and the interactive effect overlay region; the anchor portrait region obtains the bounding box coordinates by running a lightweight object detection model on the video frames, the subtitle region obtains the overlay position coordinates by parsing the subtitle track metadata in the video stream, and the coordinates of the interactive effect overlay region are provided by the gift animation coordinates and skeletal binding data in the interactive metadata. The complete range of the region of interest is determined by taking the union of the coordinates of the three types of regions.

[0020] Preferably, the high-priority lightweight transmission tunnel adopts the stateless reset mode of the fast UDP network connection protocol. When the network switches at the viewer's playback end, the high-priority lightweight transmission tunnel skips the complete connection handshake process and directly resumes data transmission on the new network path. The edge computing node marks the interactive metadata packets as the highest quality of service level, disables the congestion control algorithm in the high-priority lightweight transmission tunnel, and continuously sends the same interactive instruction packet a preset number of times within a preset time window.

[0021] Preferably, the method further includes:

[0022] After a weak network event ends, the edge computing nodes collect compensation effect log data and compare the actual link degradation level with the prediction confidence and the predicted degradation duration to calculate the prediction deviation value. When the prediction deviation value exceeds a preset deviation threshold, the corrected deviation data, along with the corresponding time-series feature sequence, is uploaded to the central cloud platform as incremental training samples. The central cloud platform aggregates the incremental training samples uploaded by each edge computing node using a federated learning aggregation method to update and train the parameters of the lightweight time-series prediction model. Each edge computing node only uploads the gradient update amount of the model parameters. The central cloud platform performs a weighted average aggregation of each gradient update amount to obtain the updated global model parameters and distributes them to each edge computing node.

[0023] A digital live streaming data interactive transmission system, comprising:

[0024] The network status acquisition module, deployed on edge computing nodes, is used to periodically collect multi-dimensional network status parameters of the current session through lightweight monitoring probes, and then concatenate the parameters of each dimension according to timestamps to generate a dynamic interactive environment feature vector.

[0025] The link degradation prediction module is deployed on an edge computing node. It is used to perform sliding window segmentation on the feature vector of the dynamic interactive environment in multiple consecutive acquisition cycles to obtain a time-series feature sequence. After performing standardization processing on the time-series feature sequence, it is input into a lightweight time-series prediction model and outputs the link state prediction confidence and the predicted degradation duration.

[0026] The bidirectional compensation execution module, deployed on the edge computing node, is used to compare the prediction confidence with the preset trigger threshold. When the prediction confidence reaches the preset trigger threshold, it generates a forward error correction code redundancy dynamic adjustment instruction for the uplink and sends it to the broadcaster's push end to perform redundant data insertion. It also generates a cache pre-fill instruction for the downlink and pushes a low-latency keyframe slice to the viewer's playback end.

[0027] The differentiated transmission tunnel module, deployed on edge computing nodes, is used to classify the data stream of the current session, establish independent transmission tunnels for the basic media stream and the interactive metadata stream, allocate a basic reliable transmission channel for the basic media stream, and establish a high-priority lightweight transmission tunnel for the interactive metadata stream, adopting a transmission strategy that combines fixed rate and redundant transmission.

[0028] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0029] 1. This invention predicts link degradation trends by deploying a lightweight timing prediction model on edge computing nodes, and performs forward error correction code redundancy adjustment and cache prefilling compensation operations in advance when the prediction confidence reaches the trigger threshold. This solves the technical problem that the existing passive retransmission strategy introduces additional round-trip delay waiting, resulting in a high frequency of video stuttering under weak network conditions.

[0030] 2. This invention solves the technical problem of the two types of data competing for bandwidth and the inability to deliver interactive metadata first when sharing a transmission channel, by establishing independent and differentiated transmission tunnels for the basic media stream and the interactive metadata stream. It achieves the technical effect that even if the basic media stream is degraded or stuttered, the interactive metadata can still be continuously delivered to the receiving end through a high-priority lightweight transmission tunnel. Attached Figure Description

[0031] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0032] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0033] Several embodiments of this application will now be described in more detail with reference to the accompanying drawings to enable those skilled in the art to implement this application. This application may be embodied in many different forms and for various purposes and should not be limited to the embodiments set forth herein. These embodiments are provided to make this application thorough and complete, and to fully convey the scope of this application to those skilled in the art. The embodiments described do not limit this application.

[0034] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having a meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0035] Example 1

[0036] Its specific implementation method is combined with the appendix Figure 1 Please provide a detailed explanation.

[0037] In this embodiment, it includes:

[0038] Step 1: Collect multi-dimensional network state parameters and generate dynamic interactive environment feature vectors.

[0039] Edge computing nodes periodically collect multi-dimensional network state parameters involved in the current session through deployed lightweight monitoring probes, and generate dynamic interactive environment feature vectors based on the collected multi-dimensional network state parameters.

[0040] Specifically, edge computing nodes collect network status parameters from three dimensions: uplink, downlink, and inter-edge links. The uplink refers to the link between the broadcaster's streaming end and the edge computing node, and the collected parameters include uplink packet loss rate, real-time round-trip latency, transmission bit rate, and signal strength fluctuation rate. The downlink refers to the link between the edge computing node and the viewer's playback end, and the collected parameters include downlink bandwidth, buffer fill level, and video stuttering frequency. The inter-edge link refers to the link between the edge computing node and the central cloud platform or other edge computing nodes, and the collected parameters include backbone network transmission jitter and concurrent stream throughput.

[0041] Edge computing nodes align the parameters collected from the above dimensions according to timestamps and then concatenate them to generate a dynamic interactive environment feature vector. , is represented as:

[0042] ;

[0043] in, This is the current data collection time; This refers to the uplink packet loss rate; Real-time round-trip latency, in milliseconds; The transmission rate is expressed in bits per second. Signal strength fluctuation rate; This refers to the downlink bandwidth, measured in bits per second. Fill the buffer with water level; This refers to the frequency of video stuttering, measured in times per second. This refers to backbone network transmission jitter, measured in milliseconds. This represents concurrent stream throughput, measured in bits per second.

[0044] It should be noted that the parameters in the aforementioned dynamic interactive environment feature vector refer to the real-time values ​​obtained by the edge computing node from each link endpoint using lightweight monitoring probes within a preset collection period. In other words, the dynamic interactive environment feature vector... Each component in the dataset corresponds to an instantaneous statistical value or the mean of a sliding window within a collection period.

[0045] In this embodiment, to enable the subsequent prediction process to perceive the business characteristics of the current session, the edge computing node also simultaneously parses the business attribute parameters of the interaction data stream and integrates these parameters into the dynamic interaction environment feature vector. The business attribute parameters include the current live stream type identifier (show, e-commerce, game, or live chat) and an interaction density index. The live stream type identifier is non-numerical classification data and requires encoding and classification data processing before integration, converting the live stream type identifier into a numerical encoded value. .

[0046] Furthermore, the above-mentioned data processing for coding classification is implemented using one-hot coding, which maps the four live streaming types—live streaming, e-commerce, gaming, and interactive live streaming—to an activation bit in a four-dimensional binary vector. For example, live streaming corresponds to... E-commerce correspondence This process continues, converting non-numerical category information into numerical encoded values ​​that can be directly used in model calculations. .

[0047] The interaction density metric is obtained by calculating the concurrent number of bullet comments and the number of touch interactions per unit time. The dynamic interaction environment feature vector, after incorporating business attribute parameters, is represented as follows:

[0048] ;

[0049] in, The encoded value that identifies the current live stream type; This is an interaction density indicator. By integrating business attribute parameters, the subsequent prediction process can distinguish the link quality change patterns under different business scenarios.

[0050] Step 2: Based on a lightweight time-series prediction model, predict the degradation trend of the link status and output the prediction confidence level.

[0051] Edge computing nodes perform sliding window segmentation on the dynamic interactive environment feature vectors acquired in multiple consecutive acquisition cycles to obtain a fixed-length temporal feature sequence.

[0052] Specifically, edge computing nodes are based on the current time. As the endpoint, take the past. The dynamic interactive environment feature vectors of each acquisition cycle form a time-series feature sequence. ,in The value is the length of the sliding window. Subsequently, the edge computing nodes perform Z-score normalization on each component in the temporal feature sequence, mapping the numerical range of each component to a unified interval, thereby eliminating the impact of the dimensional differences of each component in step 1 on subsequent model calculations.

[0053] Edge computing nodes input standardized temporal feature sequences into a pre-deployed lightweight temporal prediction model, and the lightweight temporal prediction model outputs... Link state prediction confidence at time 1 and the predicted duration of degradation. .in, The prediction time window represents the period from the current moment. The predicted time span is projected forward, and its unit is consistent with the time unit of the data collection period. The range of values ​​is The larger the value, the higher the link is. The higher the likelihood of a degradation event occurring at any given time; The predicted duration of degradation indicates the predicted length of time the link degradation will last, and the unit is consistent with the time unit of the data collection period.

[0054] It should be noted that the aforementioned lightweight time series prediction model is a Long Short-Term Memory (LSTM) network model deployed on edge computing nodes after model pruning and optimization. The LSTM network model is a conventional time series prediction model, and its input is a standardized time series feature sequence. The output of the Long Short-Term Memory (LSTM) network model is the prediction confidence score. and predicted duration of degradation .

[0055] Furthermore, the output layer of the Long Short-Term Memory (LSTM) network model includes an attention layer. The attention layer receives the hidden layer output sequence of the LTM network model. For the first in the sequence Attention weights at each time step The calculation is as follows:

[0056] ;

[0057] ;

[0058] in, A learnable query vector; For the first The hidden state vector at each time step; For the first The attention score at the nth time step represents the attention score at the nth time step. The degree of correlation between the hidden state vector and the query vector at each time step; Represents an exponential function; This is the summation index variable used to iterate through the hidden layer output sequence at each time step during the summation process; Let be the attention weights after Softmax normalization, representing the th . The contribution ratio of each time step to the final prediction result, and satisfying the following conditions: ; This represents the transpose. The attention layer performs a weighted summation on the hidden layer output sequence to obtain the context vector. :

[0059] ;

[0060] in, A context vector that incorporates information from multiple time scales. Dimensions and hidden state vectors They have the same dimensions. Context vector After mapping through the fully connected layer, the prediction confidence is output respectively. and predicted duration of degradation Specifically, the fully connected layer contains two independent output branches: the first output branch passes through a single-neuron fully connected layer and is then mapped to a Sigmoid activation function. The interval is used to obtain the prediction confidence level. The second output branch passes through a single-neuron fully connected layer and is then activated by a ReLU function to constrain the output value to a non-negative real number, thus obtaining the predicted duration of degradation. Predict the duration of degradation The unit is consistent with the time unit of the data collection period.

[0061] In addition, the input features of the aforementioned lightweight time-series prediction model also include the slope of the current online user count fluctuation. This slope is obtained by performing linear regression on the online user count over multiple consecutive collection periods, reflecting the potential impact of the dynamic changes in live stream traffic on network load. The online user count fluctuation slope is also Z-score standardized before being input into the lightweight time-series prediction model to maintain dimensional consistency with other input components.

[0062] The training of the Long Short-Term Memory (LSTM) network model was completed on the central cloud platform. The training samples consisted of historical time-series feature sequences and corresponding link degradation labels, where each label included a degradation event marker and the actual duration of degradation. Supervised learning was employed during training, with the loss function being a weighted sum of the binary cross-entropy loss for predicted confidence and the mean squared error loss for predicted degradation duration. The Adam optimization algorithm was used for optimization. After training, the LTM network model parameters were distributed to each edge computing node for online inference.

[0063] Edge computing nodes will predict confidence levels With preset trigger threshold Comparison, among which :

[0064] like If the current link status is stable, the edge computing node will maintain the distribution logic of the live stream using the normal real-time transmission protocol or hypertext transmission protocol, and will not trigger compensation operations.

[0065] like If a weak network event is predicted to occur, the edge computing node will trigger the proactive compensation strategy in step 3.

[0066] Step 3: Generate and execute bidirectional link quality dynamic compensation commands based on the prediction results.

[0067] Predicting confidence Once the trigger threshold is reached, the edge computing nodes generate corresponding compensation instructions for the uplink and downlink respectively, and send them to the corresponding terminals for execution. Step 3 includes two sub-steps: uplink redundancy pre-compensation and downlink buffer pre-filling.

[0068] Step 301: Generate a forward error correction code redundancy dynamic adjustment instruction for the uplink, and have the streamer insert redundant data.

[0069] Edge computing nodes based on prediction confidence and predicted duration of degradation Calculate the redundancy increase ratio of the forward error correction code (FEC). And generate a redundancy boost ratio carrying forward error correction codes. The redundancy level is dynamically increased by the edge computing node. The edge computing node sends the redundancy level dynamic increase command to the broadcaster's streaming terminal software development kit.

[0070] After receiving the redundancy dynamic adjustment instruction, the broadcaster's streaming terminal software development kit adjusts the redundancy according to the forward error correction code ratio when encoding and sending the next segment of video data. The proportion of keyframe redundant data packets is actively inserted into the original data packet sequence. These keyframe redundant data packets do not change the content of the original video frame. The purpose of the keyframe redundant data packets is that, within the predicted weak network window period, when some original data packets are lost during transmission, the edge computing node receiving side can directly recover the complete original data using the keyframe redundant data packets, and can complete the decoding without waiting for packet loss retransmission.

[0071] The aforementioned forward error correction code redundancy increase ratio The calculation method is as follows:

[0072] ;

[0073] in, The basic redundancy ratio represents the proportion of redundant packets maintained by default to the total number of original data packets when the link is stable. It is a dimensionless proportion. The scaling factor is an exponentially increasing factor used to control the increase ratio of forward error correction code redundancy. With prediction confidence The growth rate when the value increases is a dimensionless scaling factor; Represents an exponential function; To predict the confidence level, the value range is: This is a dimensionless value. Due to the basic redundancy ratio... Scaling factor and prediction confidence All values ​​are dimensionless, therefore the redundancy of the forward error correction code is increased by a certain percentage. The calculated result is a dimensionless redundancy ratio. Employing an exponential, non-linear growth method, it can rapidly increase redundancy when prediction confidence is high, while maintaining relatively controlled redundancy overhead when prediction confidence is low, thus achieving a balance between bandwidth utilization efficiency and data recovery capability.

[0074] Furthermore, the redundancy ratio of the forward error correction code is increased. Also affected by the predicted duration of degradation Constraints: When predicting the duration of degradation When the preset duration limit is exceeded, the forward error correction code redundancy ratio is increased. A preset upper limit for the redundancy ratio is set to prevent the redundancy overhead from growing indefinitely and exhausting the available bandwidth in scenarios where the predicted degradation duration is too long.

[0075] Step 302: Generate a cache pre-fill instruction for the downlink, and have the edge computing node perform the advance push of key frame slices.

[0076] Edge computing nodes predict the filling trend of the buffer at the viewer's playback device (i.e., predict when the viewer's playback device will...). After a period of time, due to a decrease in downlink bandwidth, it enters a low-water level state. It performs transcoding acceleration processing on the currently pushed video stream to generate additional low-latency keyframe slices.

[0077] Edge computing nodes embed generated low-latency keyframe slices (e.g., 1 to 2) into the regularly pushed Group of Pictures (GOP) data stream, and push the embedded data stream to the viewer's playback device. Upon receiving the embedded low-latency keyframe slices, the viewer's playback player pre-increases its buffer fill level. When subsequent data download rates decrease due to a drop in actual downlink bandwidth, the viewer's playback player can maintain continuous playback using the pre-stored data in the buffer, thus smoothly navigating the period of bandwidth reduction.

[0078] It should be noted that the aforementioned low-latency keyframe slices refer to compressed data segments obtained by edge computing nodes after rapidly transcoding the immediately following frames in the current image group. Transcoding acceleration refers to the edge computing nodes temporarily increasing the processing priority of the encoding threads on top of the regular encoding process, thereby completing the encoding and output of subsequent frame data ahead of schedule at the cost of microsecond-level additional latency.

[0079] Step 4: Classify the data streams and establish differentiated hierarchical transmission tunnels for each stream, then output the routing transmission configuration.

[0080] Edge computing nodes logically classify the data streams in the current session into two categories: basic media streams and interactive metadata streams. They then establish independent transmission tunnels for the basic media streams and interactive metadata streams respectively, and generate corresponding routing transmission configurations.

[0081] Step 401: Establish a basic reliable transmission channel for the basic media stream and execute an adaptive degradation strategy based on the link status.

[0082] Edge computing nodes allocate reliable transport channels based on Transmission Control Protocol (TCP) or Reliable User Datagram Protocol (UDP) for underlying media streams (video and audio streams).

[0083] When the link is in a degraded state, the edge computing node performs active frame dropping or spatial resolution reduction operations on the bitstream of non-interest regions in the video frame during the process of pushing the basic media stream, and only retains the complete bitstream of the region of interest, so as to maintain the minimum continuity of the basic reliable transmission channel and prevent the basic reliable transmission channel from being blocked due to data backlog.

[0084] It should be noted that the aforementioned non-region of interest refers to the background area in the video frame excluding the area of ​​the anchor's image, subtitles, and interactive effects. Edge computing nodes obtain the location coordinates of the region of interest by performing region detection on the video frame, and perform degradation operations on the bitstream data outside the region of interest.

[0085] Furthermore, the specific implementation of the aforementioned region detection is as follows: the edge computing node runs a lightweight object detection model on the video frame and outputs the bounding box coordinates of the anchor's portrait region; the subtitle region obtains the overlay position coordinates of the subtitle region by parsing the subtitle track metadata in the video stream; the coordinates of the interactive effect overlay region are directly provided by the gift animation coordinates and skeletal binding data in the interactive metadata collected in step 1. The complete range of the region of interest is determined by taking the union of the coordinates of the three types of regions, and the bitstream data corresponding to the pixel blocks outside the region of interest is the target for performing the degradation operation.

[0086] Step 402: Establish a high-priority lightweight transmission tunnel for the interactive metadata stream and execute a fixed-rate redundant transmission strategy.

[0087] Edge computing nodes establish a separate high-priority lightweight transmission tunnel based on the User Datagram Protocol (UDP) for interactive metadata streams (bullet screen text, like commands, virtual gift coordinates, skeleton binding data, etc.). The high-priority lightweight transmission tunnel and the basic reliable transmission channel of the basic media stream are physically independent of each other.

[0088] Edge computing nodes mark interactive metadata packets as having the highest Quality of Service (DSCPEF) and disable complex congestion control algorithms in high-priority lightweight transmission tunnels, instead employing a transmission strategy that combines fixed-rate transmission with redundant transmission. Specifically, edge computing nodes continuously transmit the same interactive instruction packet a preset number of times within a preset time window to ensure that at least one copy reaches the receiving end in the event of partial packet loss.

[0089] It should be noted that the aforementioned fixed-rate transmission refers to edge computing nodes injecting data packets into the high-priority lightweight transmission tunnel at a pre-configured fixed transmission rate, without dynamically reducing the transmission rate based on network congestion signals. Since the single packet size of interactive metadata is extremely small (typically tens to hundreds of bytes), fixed-rate transmission will not significantly increase network bandwidth usage.

[0090] In this embodiment, to further reduce the connection recovery overhead of interactive metadata streams during link migration scenarios, the aforementioned high-priority lightweight transmission tunnel adopts a stateless reset mode of the Fast UDP Network Connection (QUIC) protocol. In stateless reset mode, when the viewer's playback end experiences a network switch (e.g., from a wireless LAN to a mobile cellular network), the high-priority lightweight transmission tunnel can skip the complete connection handshake process and directly resume data transmission on the new network path, thereby reducing the handshake overhead during connection migration.

[0091] In this embodiment of the application, in addition to steps 1 to 4, the following steps are also included to continuously optimize the lightweight time series prediction model and improve the accuracy of subsequent predictions.

[0092] Step 5: Collect the compensation effect logs and upload them to the central cloud platform. Update the parameters of the lightweight time series prediction model through federated learning.

[0093] After the weak network event ends, the edge computing nodes will collect the compensation effect log data for that period, including the actual retransmission rate, the success rate of redundant data packet repair, and the interactive response latency statistics of the viewer's playback terminal.

[0094] The edge computing nodes compare the actual link degradation level with the predicted value output in step 2 to calculate the prediction deviation. If the prediction deviation exceeds a preset deviation threshold, the edge computing nodes upload the corrected deviation data along with the corresponding time-series feature sequence as incremental training samples to the model training cluster on the central cloud platform.

[0095] The central cloud platform utilizes federated learning to aggregate incremental training samples uploaded by edge computing nodes across the entire network, updating and training the parameters of the lightweight time-series prediction model. During federated learning, each edge computing node only uploads the gradient updates of the lightweight time-series prediction model parameters, not the original network state data. The central cloud platform performs a weighted average aggregation of the gradient updates uploaded by each edge computing node to obtain the updated global model parameters, and then distributes the updated global model parameters to each edge computing node, replacing the lightweight time-series prediction model parameters deployed in step 2.

[0096] Example 2

[0097] A digital live streaming data interactive transmission system includes the following components:

[0098] The network status acquisition module, deployed on edge computing nodes, is used to periodically collect multi-dimensional network status parameters of the current session through lightweight monitoring probes, and then concatenate the parameters of each dimension according to timestamps to generate a dynamic interactive environment feature vector.

[0099] The link degradation prediction module is deployed on edge computing nodes. It is used to obtain a time-series feature sequence by sliding window segmentation of the dynamic interactive environment feature vectors of multiple consecutive acquisition cycles. After standardization processing of the time-series feature sequence, it is input into a lightweight time-series prediction model and outputs the link state prediction confidence and the predicted degradation duration.

[0100] The bidirectional compensation execution module, deployed on the edge computing node, is used to compare the prediction confidence with the preset trigger threshold. When the prediction confidence reaches the preset trigger threshold, it generates a forward error correction code redundancy dynamic adjustment instruction for the uplink and sends it to the broadcaster's push stream end to perform redundant data insertion. For the downlink, it generates a cache pre-fill instruction and pushes low-latency keyframe slices to the viewer's playback end.

[0101] The differentiated transmission tunnel module, deployed on edge computing nodes, is used to classify the data stream of the current session, establish independent transmission tunnels for the basic media stream and the interactive metadata stream, allocate basic reliable transmission channels for the basic media stream, and establish high-priority lightweight transmission tunnels for the interactive metadata stream, adopting a transmission strategy that combines fixed rate and redundant transmission.

[0102] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0103] It should be noted that, in this document, the use of relational terms such as "first" and "second" is merely for distinguishing one entity or operation from another, and does not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0104] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0105] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0106] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for interactive transmission of digital live streaming data, characterized in that, include: Edge computing nodes periodically collect multi-dimensional network state parameters of the current session through lightweight monitoring probes, and then align the parameters of each dimension according to the timestamp to generate a dynamic interactive environment feature vector. A time-series feature sequence is obtained by sliding window segmentation of the dynamic interactive environment feature vectors of multiple consecutive acquisition cycles. After standardization processing of the time-series feature sequence, it is input into a lightweight time-series prediction model, and the link state prediction confidence and predicted degradation duration are output. The prediction confidence is compared with a preset trigger threshold. When the prediction confidence reaches the preset trigger threshold, a forward error correction code redundancy dynamic adjustment instruction is generated for the uplink based on the prediction confidence and the prediction degradation duration and sent to the streamer's push end to perform redundant data insertion. For the downlink, a cache pre-fill instruction is generated and a low-latency keyframe slice is pushed to the viewer's playback end. The data stream of the current session is classified, and independent transmission tunnels are established for the basic media stream and the interactive metadata stream. A basic reliable transmission channel is allocated for the basic media stream, and a high-priority lightweight transmission tunnel is established for the interactive metadata stream, using a transmission strategy that combines fixed rate and redundant transmission.

2. The interactive transmission method for digital live streaming data according to claim 1, characterized in that, Multidimensional network state parameters are collected from three dimensions: uplink, downlink, and inter-edge links. The uplink is the link between the broadcaster's streaming end and the edge computing node, and the collected parameters include uplink packet loss rate, real-time round-trip latency, transmission bit rate, and signal strength fluctuation rate. The downlink is the link between the edge computing node and the viewer's playback end, and the collected parameters include downlink bandwidth, buffer fill level, and video stuttering frequency. The inter-edge links are the links between the edge computing node and the central cloud platform or other edge computing nodes, and the collected parameters include backbone network transmission jitter and concurrent stream throughput.

3. The interactive transmission method for digital live streaming data according to claim 1, characterized in that, The edge computing node also parses the business attribute parameters of the interactive data stream and integrates the business attribute parameters into the feature vector of the dynamic interactive environment. The business attribute parameters include the current live stream type identifier and the interaction density index. The current live stream type identifier is non-numerical classification data, which is converted into a numerical encoded value through one-hot encoding. The interaction density index is obtained by calculating the number of concurrent bullet comments and touch screen interactions per unit time.

4. The interactive transmission method for digital live streaming data according to claim 1, characterized in that, The lightweight time-series prediction model is a Long Short-Term Memory (LSTM) network model deployed on edge computing nodes after model pruning optimization. An attention layer precedes the output layer of the LSM network model. This attention layer receives the hidden output sequence of the LSM network model and obtains the attention score for each time step by performing an inner product operation between the hidden state vector and the learnable query vector. The attention scores for each time step are then normalized using Softmax to obtain attention weights. These attention weights are then used to perform a weighted summation of the hidden output sequence to obtain a context vector. The context vector is then processed by two independent output branches of a fully connected layer to output the prediction confidence and the prediction degradation duration, respectively. The first output branch uses a Sigmoid activation function to map the output value to a closed interval between zero and one, while the second output branch uses a ReLU activation function to constrain the output value to a non-negative real number.

5. The interactive transmission method for digital live streaming data according to claim 1, characterized in that, The forward error correction code redundancy dynamic adjustment instruction carries the forward error correction code redundancy adjustment ratio. The forward error correction code redundancy adjustment ratio is calculated by multiplying the base redundancy ratio by the value of an exponential function, where the exponent of the exponential function is the product of a scaling factor and the prediction confidence. The base redundancy ratio is the default proportion of redundant packets to the total original data packets when the link is stable. The scaling factor is used to control the growth rate of the forward error correction code redundancy adjustment ratio as the prediction confidence increases. When the predicted degradation duration exceeds a preset duration upper limit, the forward error correction code redundancy adjustment ratio takes the preset redundancy ratio upper limit value.

6. The interactive transmission method for digital live streaming data according to claim 1, characterized in that, The process of generating a buffer pre-fill instruction for the downlink and pushing low-latency keyframe slices to the viewer's playback terminal includes: the edge computing node performing transcoding acceleration processing on the currently pushed video stream based on the predicted trend of the buffer fill level at the viewer's playback terminal, generating low-latency keyframe slices; embedding the low-latency keyframe slices into the regularly pushed image group data stream and pushing them to the viewer's playback terminal; wherein, the transcoding acceleration processing refers to the edge computing node temporarily increasing the processing priority of the encoding thread to complete the encoding output of subsequent frame data in advance.

7. The interactive transmission method for digital live streaming data according to claim 1, characterized in that, When the link is in a degraded state, the edge computing node performs active frame dropping or spatial resolution reduction operations on the bitstream of non-interest regions in the video frames of the basic media stream, while retaining the complete bitstream of the region of interest. The region of interest includes the anchor portrait region, the subtitle region, and the interactive effect overlay region. The anchor portrait region obtains the bounding box coordinates by running a lightweight object detection model on the video frames. The subtitle region obtains the overlay position coordinates by parsing the subtitle track metadata in the video stream. The coordinates of the interactive effect overlay region are provided by the gift animation coordinates and skeleton binding data in the interactive metadata. The complete range of the region of interest is determined by taking the union of the coordinates of the three types of regions.

8. The interactive transmission method for digital live streaming data according to claim 1, characterized in that, The high-priority lightweight transmission tunnel adopts the stateless reset mode of the fast UDP network connection protocol. When the network switches at the viewer's playback end, the high-priority lightweight transmission tunnel skips the complete connection handshake process and directly resumes data transmission on the new network path. The edge computing node marks the interactive metadata packets as the highest quality of service level, disables the congestion control algorithm in the high-priority lightweight transmission tunnel, and continuously sends the same interactive instruction packet a preset number of times within a preset time window.

9. The interactive transmission method for digital live streaming data according to claim 1, characterized in that, Also includes: After a weak network event ends, the edge computing nodes collect compensation effect log data and compare the actual link degradation level with the prediction confidence and the predicted degradation duration to calculate the prediction deviation value. When the prediction deviation value exceeds a preset deviation threshold, the corrected deviation data, along with the corresponding time-series feature sequence, is uploaded to the central cloud platform as incremental training samples. The central cloud platform aggregates the incremental training samples uploaded by each edge computing node using a federated learning aggregation method to update and train the parameters of the lightweight time-series prediction model. Each edge computing node only uploads the gradient update amount of the model parameters. The central cloud platform performs a weighted average aggregation of each gradient update amount to obtain the updated global model parameters and distributes them to each edge computing node.

10. An interactive transmission system for digital live streaming data, comprising an interactive transmission method for digital live streaming data according to any one of claims 1-9, characterized in that, include: The network status acquisition module, deployed on edge computing nodes, is used to periodically collect multi-dimensional network status parameters of the current session through lightweight monitoring probes, and then concatenate the parameters of each dimension according to timestamps to generate a dynamic interactive environment feature vector. The link degradation prediction module is deployed on an edge computing node. It is used to perform sliding window segmentation on the feature vector of the dynamic interactive environment in multiple consecutive acquisition cycles to obtain a time-series feature sequence. After performing standardization processing on the time-series feature sequence, it is input into a lightweight time-series prediction model and outputs the link state prediction confidence and the predicted degradation duration. The bidirectional compensation execution module, deployed on the edge computing node, is used to compare the prediction confidence with the preset trigger threshold. When the prediction confidence reaches the preset trigger threshold, it generates a forward error correction code redundancy dynamic adjustment instruction for the uplink and sends it to the broadcaster's push end to perform redundant data insertion. It also generates a cache pre-fill instruction for the downlink and pushes a low-latency keyframe slice to the viewer's playback end. The differentiated transmission tunnel module, deployed on edge computing nodes, is used to classify the data stream of the current session, establish independent transmission tunnels for the basic media stream and the interactive metadata stream, allocate a basic reliable transmission channel for the basic media stream, and establish a high-priority lightweight transmission tunnel for the interactive metadata stream, adopting a transmission strategy that combines fixed rate and redundant transmission.

Images