A network live broadcast system supporting multi-format output

The network live broadcast system addresses frame rate and encoding adjustments under fluctuating network conditions by using frame analysis, neural networks, and decision tree algorithms to optimize bandwidth and encoding, enhancing video quality and continuity.

CN118694995BActive Publication Date: 2024-12-03FUSHI ZHITONG ELECTRONIC TECH (JINAN) CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411178268.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-12-03
Estimated Expiration
2044-08-27

AI Technical Summary

Technical Problem

Existing live broadcast systems fail to adjust frame rates and encoding standards in a timely manner under fluctuating network conditions, leading to reduced video quality, interruptions, and limited flexibility in bandwidth utilization and output format adjustment, which affects audience experience and limits audience expansion.

Method used

A network live broadcast system that utilizes a video frame analysis module to separate frames, calculate visual features, and evaluate dynamic characteristics, combined with a recurrent neural network for frame rate adjustment and bandwidth optimization, and a coding standard decision module using a decision tree algorithm to select encoding standards based on device and network capabilities, while a network status monitoring module adjusts transmission parameters for multiple output formats.

Benefits of technology

The system provides accurate frame rate management, optimizes bandwidth usage, and dynamically adjusts encoding standards to ensure video quality and continuity, improving live broadcast efficiency and audience satisfaction by adapting to various playback conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118694995B_ABST
    Figure CN118694995B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of network live broadcast technology, specifically to a network live broadcast system supporting multi-format output, the system comprising a video frame analysis module, a frame rate control module, a coding standard decision module and a network status monitoring module. The present invention provides accurate frame rate management capabilities for the live broadcast system by adopting a recurrent neural network to analyze the dynamic changes between frames in the live video stream, and adjusting the frame rate by a change index, allowing the system to automatically optimize bandwidth usage according to the dynamic changes of the scene, reducing unnecessary bandwidth occupancy by reducing the frame rate of static scenes, and increasing the frame rate in dynamic scenes to maintain video clarity. The decision tree algorithm enables the selection of coding standards to be dynamically adjusted according to real-time data, responding to changes in network and device capabilities, ensuring that the video output always adapts to various playback conditions, and by adjusting video transmission parameters in real time, the video stream quality and continuity of the live broadcast system are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of network live broadcasting, and in particular to a network live broadcasting system supporting multi-format output. Background Art

[0002] Live streaming is a technology and method for transmitting audio and video content in real time over the Internet. This field uses encoding and compression technology to optimize the transmission of video and audio data, ensuring a smooth viewing experience even under suboptimal network conditions. In addition, live streaming technology also includes the use of content distribution networks (CDNs), which help reduce latency and increase access speeds by caching content on globally distributed servers. With the development of technology, live streaming has also begun to support multiple output formats to adapt to different playback devices and viewing needs, including different resolutions, file formats, and transmission protocols.

[0003] Among them, the online live broadcast system that supports multi-format output refers to a system that can automatically adjust and output the most suitable video format according to different viewing devices and network environments. Its main purpose is to provide live broadcast practitioners with a flexible and powerful tool to optimize the transmission and display of their content, ensuring that no matter what device or network the audience uses, they can get the best viewing effect. It not only improves the audience satisfaction, but also helps live broadcast practitioners to more effectively manage and distribute their live content, adapt to various broadcast needs, thereby expanding the audience base and improving live broadcast efficiency.

[0004] Under changeable network conditions, existing live broadcast systems often fail to adjust frame rates and encoding standards in a timely manner, resulting in reduced video quality. In particular, when network quality fluctuates greatly, image delays and freezes occur more frequently. Existing technologies are also limited in flexibility in bandwidth utilization and output format adjustment, and fail to effectively match the specific needs of different viewers. For example, in a low-bandwidth environment, the lack of an effective frame rate and encoding adjustment mechanism will lead to live broadcast interruptions, affecting the audience experience, thereby limiting the coverage of live content and the expansion of the audience base. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a network live broadcast system that supports multi-format output.

[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a network live broadcast system supporting multi-format output, the system comprising:

[0007] The video frame analysis module separates the video frames one by one based on the live video stream, calculates the visual features of each frame, and evaluates the dynamic characteristics within the frame based on the visual feature data to generate dynamic characteristic description information;

[0008] The frame rate control module uses a recurrent neural network to analyze the dynamic changes between frames based on the dynamic characteristic description information, performs frame rate adjustment according to the change index, determines the increase or decrease of the frame rate, and optimizes the bandwidth usage according to the adjustment feedback, matches the difference output format requirements, and obtains the adjusted frame rate data;

[0009] The coding standard decision module uses a decision tree algorithm based on the adjusted frame rate data to collect device coding capabilities and current network status information, selects a matching video coding standard in combination with video content characteristics and output format, adjusts the video compression ratio, and obtains a coding decision output record;

[0010] The network status monitoring module monitors the real-time network bandwidth usage and packet loss rate based on the encoding decision output record, calculates the network stability, and adjusts the video transmission parameters for multiple output formats to obtain network adjustment results.

[0011] The present invention is improved in that the step of obtaining the dynamic characteristic description information is specifically as follows:

[0012] Based on the live video stream, the video frames are parsed and the pixel intensity and color distribution of each frame are calculated using the formula:

[0013]

[0014] Get the visual feature vector ,in, is the time index of the frame, is the total number of pixels, It's in time Pixels The gray value of It is the weight coefficient obtained according to the pixel color and brightness frequency;

[0015] The visual feature vectors of consecutive frames and For comparison, use the formula:

[0016]

[0017] Get visual change ,in, and is the visual feature vector of two consecutive frames, is the total number of pixels, is the absolute visual difference between two frames;

[0018] The visual change degree is used to determine the frame dynamics through a dynamic threshold to evaluate whether each frame is a dynamic frame. When the visual change degree is greater than the dynamic threshold, it indicates that the frame is For the previous frame A visual change occurs. When the visual change is less than or equal to the dynamic threshold, it means that the frame and the previous frame There is no visual change in between.

[0019] The present invention is improved in that the step of performing frame rate adjustment according to the change index is specifically:

[0020] Based on the dynamic characteristic description information, a recurrent neural network is used and the formula is adopted:

[0021]

[0022] Calculate the dynamic change index ,in, It's history The dynamic characteristics of the frame description information, is the weighting coefficient, is an exponent used to increase the nonlinear effect, is the time window size;

[0023] Based on the dynamic change index, the new frame rate is calculated using the formula:

[0024]

[0025] Get the adjusted frame rate ,in, is the original frame rate, is the adjustment factor based on network performance evaluation, is the dynamic change index;

[0026] According to the adjusted frame rate, the video stream output settings are updated to match network changes and content dynamics, thereby obtaining a frame rate setting result.

[0027] The present invention is improved in that the step of obtaining the frame rate adjustment data is specifically as follows:

[0028] According to the adjusted frame rate, combined with the original bandwidth usage, the formula is adopted:

[0029]

[0030] Get the bandwidth adjustment factor ,in, is the adjusted frame rate, is the original frame rate, is the sensitivity coefficient of bandwidth adjustment;

[0031] Based on the bandwidth adjustment factor and the original bandwidth, the new bandwidth usage is calculated using the formula:

[0032]

[0033] Get optimized bandwidth usage data ,in, is the raw bandwidth usage, is the bandwidth adjustment factor, is the adjustment factor used to smooth bandwidth changes;

[0034] According to the optimized bandwidth usage data, the output format is adjusted to match the different playback requirements and network conditions, and frame rate adjustment information is obtained.

[0035] The present invention is improved in that the steps of collecting the device encoding capability and current network status information are specifically as follows:

[0036] Based on the adjusted frame rate data, the CPU performance and memory capacity data are obtained using the formula:

[0037]

[0038] Get a device's coding ability score ,in, and is a weight factor used to adjust the impact of CPU and memory on the score. is the CPU speed of the device, is the memory capacity of the device;

[0039] Based on the encoding capability score of the device in question, a network monitoring tool was used to assess bandwidth and latency using the formula:

[0040]

[0041] Get the current network status score ,in, and is a weighting factor used to adjust the impact of bandwidth and latency in the score, is the network bandwidth, It is the inverse of network latency;

[0042] Based on the encoding ability score of the device and the current network status score, the formula is used:

[0043]

[0044] Get integrated device capability and network status scores ,in, is the encoding ability score of the device, is the current network status score.

[0045] The present invention is improved in that the step of obtaining the encoding decision output record is specifically as follows:

[0046] Analyze the key frames of the video stream and extract key parameters, including frame rate and resolution, using the formula:

[0047]

[0048] Get video content feature data ,in, is the video frame rate, is the video resolution, and is a weight factor used to adjust the influence of frame rate and resolution;

[0049] Based on the video content characteristic data, according to the device and network information score, combined with the video content characteristics, the formula is adopted:

[0050]

[0051] Get the output format decision data ,in, It is the video content characteristic data, is the device and network information score, and is a weighting factor used to balance the impact of content characteristics and device network conditions;

[0052] According to the format decision data, a matching video encoding standard is selected. When the format decision data is greater than a threshold, H.265 is selected; when the format decision data is less than or equal to the threshold, H.264 is selected.

[0053] The present invention is improved in that the monitoring steps of the network bandwidth usage rate and packet loss rate are specifically as follows:

[0054] Based on the encoding decision output record, the network bandwidth usage is monitored in real time, and the current bandwidth usage rate is calculated using the formula:

[0055]

[0056] Get bandwidth usage results ,in, is the current network bandwidth usage, is the maximum available bandwidth;

[0057] Based on the bandwidth utilization results, combined with the packet loss rate, the formula is used:

[0058]

[0059] Get network performance indicators ,in, and is a weight factor used to adjust the impact of bandwidth utilization and packet loss rate. is the bandwidth utilization result, It is the packet loss rate result.

[0060] The present invention is improved in that the step of obtaining the network adjustment result is specifically as follows:

[0061] Based on the network stability, the network fitness score is calculated using the formula:

[0062]

[0063] Get the network fitness evaluation results ,in, It is a network performance indicator;

[0064] Based on the network fitness evaluation result, the adjustment of the video transmission parameters is determined using the formula:

[0065]

[0066] Get the adjustment decision results ,in, is the network fitness evaluation result;

[0067] According to the adjustment decision result, video transmission parameters are adjusted for multiple output formats to obtain network adjustment results.

[0068] Compared with the prior art, the advantages and positive effects of the present invention are:

[0069] In the present invention, a recurrent neural network is used to analyze the dynamic changes between frames in the live video stream, and the frame rate is adjusted by a change index, thereby providing an accurate frame rate management capability for the live broadcast system, allowing the system to automatically optimize bandwidth usage according to the dynamic changes of the scene, reducing unnecessary bandwidth occupancy by lowering the frame rate of static scenes, and increasing the frame rate in dynamic scenes to maintain video clarity. The decision tree algorithm enables the selection of encoding standards to be dynamically adjusted according to real-time data, responding to changes in network and device capabilities, ensuring that video output always adapts to various playback conditions, and by adjusting video transmission parameters in real time, the video stream quality and continuity of the live broadcast system are improved, effectively improving the live broadcast efficiency and audience satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 is a system flow chart of the present invention;

[0071] Figure 2 A flowchart for obtaining dynamic characteristic description information in the present invention;

[0072] Figure 3 A flow chart of performing frame rate adjustment according to a change index in the present invention;

[0073] Figure 4 This is a flow chart for obtaining frame rate data adjusted in the present invention;

[0074] Figure 5 It is a flow chart for collecting device encoding capability and current network status information in the present invention;

[0075] Figure 6 A flowchart for obtaining the encoding decision output record in the present invention;

[0076] Figure 7 The monitoring flow chart of network bandwidth utilization and packet loss rate in the present invention;

[0077] Figure 8 This is a flow chart for obtaining network adjustment results in the present invention. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0079] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating positions or positional relationships, are based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0080] Example

[0081] See also Figure 1 The present invention provides a technical solution: a network live broadcast system supporting multi-format output includes:

[0082] The video frame analysis module separates the video frames one by one based on the live video stream, calculates the visual features of each frame, and evaluates the dynamic characteristics within the frame based on the visual feature data to generate dynamic characteristic description information;

[0083] The frame rate control module uses a recurrent neural network based on dynamic feature description information to analyze dynamic changes between frames, performs frame rate adjustment according to the change index, determines whether to increase or decrease the frame rate, and optimizes bandwidth usage based on adjustment feedback to match the difference output format requirements and obtain the adjusted frame rate data;

[0084] The coding standard decision module uses the decision tree algorithm to collect information about the device coding capability and current network status based on the adjusted frame rate data, and selects a matching video coding standard in combination with the video content characteristics and output format, adjusts the video compression ratio, and obtains the coding decision output record;

[0085] The network status monitoring module monitors the real-time network bandwidth usage and packet loss rate based on the encoding decision output records, calculates the network stability, and adjusts the video transmission parameters for various output formats to obtain the network adjustment results.

[0086] Dynamic characteristic description information includes dynamic range, change frequency, and active area. Adjusted frame rate data includes rate adjustment value, frame interval, and output stability. Coding decision output records include selected coding format, expected compression effect, and target output quality. Network adjustment results include adjusted bandwidth quota, packet loss improvement rate, and connection stability rating.

[0087] See also Figure 2 ,The specific steps for obtaining dynamic feature description information are:

[0088] Based on the live video stream, the video frames are parsed and the pixel intensity and color distribution of each frame are calculated using the formula:

[0089]

[0090] Get the visual feature vector ,in, is the time index of the frame, is the total number of pixels, It's in time Pixels The gray value of It is the weight coefficient obtained according to the pixel color and brightness frequency;

[0091] Visual feature vectors for consecutive frames and For comparison, use the formula:

[0092]

[0093] Get visual change ,in, and is the visual feature vector of two consecutive frames, is the total number of pixels, is the absolute visual difference between two frames;

[0094] The visual change degree is used to judge the dynamic nature of the frame through the dynamic threshold, and each frame is evaluated to see whether it is a dynamic frame. When the visual change degree is greater than the dynamic threshold, it means that the frame is For the previous frame A visual change occurs. When the visual change is less than or equal to the dynamic threshold, it means that the frame and the previous frame There is no visual change in between.

[0095] In a specific time frame In the monitoring, three pixels are detected, and their gray values They are 100, 150 and 200 respectively;

[0096] By analyzing the frequency of occurrence of pixels in previous video frames, the corresponding 0.1, 0.2 and 0.3;

[0097] For example, 1920x1080 (1080p video).

[0098] So The calculation is:

[0099]

[0100] Represents the sum of visually weighted features of the frame.

[0101] set up is 100, and the previous frame is 90, the visual change The calculation is:

[0102]

[0103] This small value indicates that the frame-to-frame changes are very small and can be ignored.

[0104] If the dynamic threshold is set to 0.00001, then 0.00000482≤0.00001, which means that no obvious dynamic features are shown.

[0105] See also Figure 3 , the steps of performing frame rate adjustment according to the change index are as follows:

[0106] Based on the dynamic characteristics description information, using the recurrent neural network, the formula is adopted:

[0107]

[0108] Calculate the dynamic change index , used for frame rate adjustment, where It's history The dynamic characteristics of the frame description information, is the weighting coefficient, is an exponent used to increase the nonlinear effect, is the time window size;

[0109] Based on the dynamic change index, the new frame rate is calculated using the formula:

[0110]

[0111] Get the adjusted frame rate ,in, is the original frame rate, is the adjustment factor based on network performance evaluation, is the dynamic change index;

[0112] According to the adjusted frame rate, the video stream output settings are updated to match the network changes and content dynamics, and the frame rate setting result is obtained.

[0113] Collected in real-time monitoring of the system 0.5, and ;

[0114] By analyzing the data, we can obtain ;

[0115] By analyzing historical data, we can obtain , .

[0116] Calculation process:

[0117]

[0118]

[0119]

[0120]

[0121] Represents the dynamic change index of the current frame relative to the previous three frames, which is used for the subsequent frame rate adjustment.

[0122] collection fps;

[0123] Set to 0.1.

[0124] Calculation process:

[0125]

[0126]

[0127]

[0128]

[0129] New frame rate FPS means that based on the analysis of dynamic changes, the system adjusts the frame rate from 30fps to 31.08fps to optimize the smoothness of video playback.

[0130] See also Figure 4 , the specific steps for obtaining the frame rate data are as follows:

[0131] Based on the adjusted frame rate, combined with the original bandwidth usage, the formula is used:

[0132]

[0133] Get the bandwidth adjustment factor ,in, is the adjusted frame rate, is the original frame rate, is the sensitivity coefficient of bandwidth adjustment;

[0134] Based on the bandwidth adjustment factor and the original bandwidth, the new bandwidth usage is calculated using the formula:

[0135]

[0136] Get optimized bandwidth usage data ,in, is the raw bandwidth usage, is the bandwidth adjustment factor, is the adjustment factor used to smooth bandwidth changes;

[0137] According to the optimized bandwidth usage data, the output format is adjusted to match the different playback requirements and network conditions, and the frame rate adjustment information is obtained.

[0138] The following data are collected:

[0139] 32fps;

[0140] 30fps;

[0141] is 1.2.

[0142] Calculation process:

[0143]

[0144]

[0145] result This means that the bandwidth needs to be increased by about 8% relative to the original bandwidth to adapt to the new frame rate adjustment.

[0146] The following data is obtained:

[0147] Mbps;

[0148] for .

[0149] Calculation process:

[0150]

[0151]

[0152] result Mbps shows that the system bandwidth requirement has increased significantly due to the increase in frame rate.

[0153] See also Figure 5 ,The specific steps for collecting device encoding capabilities and current network status information are:

[0154] Based on the adjusted frame rate data, the CPU performance and memory capacity data are obtained using the formula:

[0155]

[0156] Get a device's coding ability score ,in, and is a weight factor used to adjust the impact of CPU and memory on the score. is the CPU speed of the device, is the memory capacity of the device;

[0157] Based on the encoding ability score of the device, use a network monitoring tool to evaluate bandwidth and latency using the formula:

[0158]

[0159] Get the current network status score ,in, and is a weighting factor used to adjust the impact of bandwidth and latency in the score, is the network bandwidth, It is the inverse of network latency;

[0160] Based on the device's coding ability score and the current network status score, the formula is used:

[0161]

[0162] Get integrated device capability and network status scores ,in, is the encoding ability score of the device, is the current network status score.

[0163] From a device monitoring with 3GHz CPU speed and 16GB memory, and according to the device performance impact analysis, and .

[0164] Perform the calculation:

[0165]

[0166]

[0167] score Reflects the encoding capability of the device.

[0168] The acquisition network bandwidth is 100Mbps, the delay is 20ms, and the weight factor is .

[0169] Calculation process:

[0170]

[0171]

[0172] Score Indicates that the current network condition is good and suitable for high-quality video encoding.

[0173] Combined with the previous device coding ability score of 9.5 and network condition score of 85;

[0174] Calculation process:

[0175]

[0176]

[0177]

[0178] Score Provides a comprehensive evaluation that reflects the combined impact of device and network conditions on video encoding performance.

[0179] See also Figure 6 , the steps for obtaining the encoding decision output record are as follows:

[0180] Analyze the key frames of the video stream and extract key parameters, including frame rate and resolution, using the formula:

[0181]

[0182] Get video content feature data ,in, is the video frame rate, is the video resolution, and is a weight factor used to adjust the influence of frame rate and resolution;

[0183] Based on the video content feature data, according to the device and network information score, combined with the video content features, the formula is used:

[0184]

[0185] Get the output format decision data ,in, It is the video content characteristic data, is the device and network information score, and is a weighting factor used to balance the impact of content characteristics and device network conditions;

[0186] According to the format decision data, a matching video coding standard is selected. When the format decision data is greater than a threshold, H.265 is selected; when the format decision data is less than or equal to the threshold, H.264 is selected.

[0187] The following data is collected:

[0188] 30fps;

[0189] 1920x1080;

[0190] set up and .

[0191] Calculation process:

[0192]

[0193]

[0194]

[0195] The video content characteristics score is 3.897, which is used to evaluate the complexity of video content and encoding requirements.

[0196] The following data is collected:

[0197] is 80;

[0198] set up and .

[0199] Calculation process:

[0200]

[0201]

[0202]

[0203] The output format decision score is 5135 and is used to determine the best encoding and compression settings for the video.

[0204] Known The value is 5135, and the threshold is 5000. Then 5135 is greater than 5000, indicating that the H.265 encoding standard is selected.

[0205] See also Figure 7 , the specific steps for monitoring network bandwidth usage and packet loss rate are:

[0206] Based on the encoding decision output record, the network bandwidth usage is monitored in real time, and the current bandwidth usage rate is calculated using the formula:

[0207]

[0208] Get bandwidth usage results ,in, is the current network bandwidth usage, is the maximum available bandwidth;

[0209] Based on the bandwidth utilization results, combined with the packet loss rate, the formula is used:

[0210]

[0211] Get network performance indicators ,in, and is a weight factor used to adjust the impact of bandwidth utilization and packet loss rate. is the bandwidth utilization result, It is the packet loss rate result.

[0212] Collect the currently monitored network bandwidth usage The maximum bandwidth is 200Mbps 1000Mbps;

[0213] Calculation process:

[0214]

[0215] result Indicates that the current network bandwidth usage is 20%.

[0216] According to previous data , collect packet loss rate ,set up is 0.8, is 0.2;

[0217] Calculation process:

[0218]

[0219] result It indicates that the network performance index is 17, and a lower value indicates better network performance.

[0220] See also Figure 8 , the specific steps for obtaining the network adjustment results are:

[0221] According to the network stability, the network fitness score is calculated using the formula:

[0222]

[0223] Get the network fitness evaluation results ,in, It is a network performance indicator;

[0224] Based on the network fitness evaluation results, the adjustment of video transmission parameters is determined using the formula:

[0225]

[0226] Get the adjustment decision results ,in, is the network fitness evaluation result;

[0227] According to the adjustment decision results, video transmission parameters are adjusted for multiple output formats to obtain network adjustment results.

[0228] Obtained from the monitoring system is 40, calculated as follows:

[0229]

[0230] result It indicates that the network fitness is 60%, indicating that the network conditions are suitable for video transmission.

[0231] Using the above obtained , make a judgment. Since 0.6 is greater than 0.5, it means that the network conditions are good. You can choose to increase the bit rate to improve the video quality.

[0232] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A network live broadcast system supporting multi-format output, characterized in that: The system comprises: The video frame analysis module separates the video frames one by one based on the live video stream, calculates the visual features of each frame, and evaluates the dynamic characteristics within the frame based on the visual feature data to generate dynamic characteristic description information; The frame rate control module uses a recurrent neural network to analyze the dynamic changes between frames based on the dynamic characteristic description information, performs frame rate adjustment according to the change index, determines the increase or decrease of the frame rate, and optimizes the bandwidth usage according to the adjustment feedback, matches the difference output format requirements, and obtains the adjusted frame rate data; The coding standard decision module uses a decision tree algorithm based on the adjusted frame rate data to collect device coding capabilities and current network status information, selects a matching video coding standard in combination with video content characteristics and output format, adjusts the video compression ratio, and obtains a coding decision output record; The network status monitoring module monitors the real-time network bandwidth usage and packet loss rate based on the encoding decision output record, calculates the network stability, and adjusts the video transmission parameters for multiple output formats to obtain the network adjustment results; The steps of obtaining the dynamic characteristic description information are specifically as follows: Based on the live video stream, the video frames are parsed and the pixel intensity and color distribution of each frame are calculated using the formula: Get the visual feature vector ,in, is the time index of the frame, is the total number of pixels, It's in time Pixels The gray value of It is the weight coefficient obtained according to the pixel color and brightness frequency; The visual feature vectors of consecutive frames and For comparison, use the formula: Get visual change ,in, and is the visual feature vector of two consecutive frames, is the total number of pixels, is the absolute visual difference between two frames; The visual change degree is used to determine the frame dynamics through a dynamic threshold to evaluate whether each frame is a dynamic frame. When the visual change degree is greater than the dynamic threshold, it indicates that the frame is For the previous frame A visual change occurs. When the visual change is less than or equal to the dynamic threshold, it means that the frame and the previous frame There is no visual change in between; The step of performing frame rate adjustment according to the change index is specifically as follows: Based on the dynamic characteristic description information, a recurrent neural network is used and the formula is adopted: Calculate the dynamic change index ,in, It's history The dynamic characteristics of the frame description information, is the weighting coefficient, is an exponent used to increase the nonlinear effect, is the time window size; Based on the dynamic change index, the new frame rate is calculated using the formula: Get the adjusted frame rate ,in, is the original frame rate, is the adjustment factor based on network performance evaluation, is the dynamic change index; According to the adjusted frame rate, the video stream output settings are updated to match network changes and content dynamics, and a frame rate setting result is obtained.

2. The network live broadcast system supporting multi-format output according to claim 1, characterized in that: The steps of obtaining the frame rate adjustment data are specifically as follows: According to the adjusted frame rate, combined with the original bandwidth usage, the formula is adopted: Get the bandwidth adjustment factor ,in, is the adjusted frame rate, is the original frame rate, is the sensitivity coefficient of bandwidth adjustment; Based on the bandwidth adjustment factor and the original bandwidth, the new bandwidth usage is calculated using the formula: Get optimized bandwidth usage data ,in, is the raw bandwidth usage, is the bandwidth adjustment factor, is the adjustment factor used to smooth bandwidth changes; According to the optimized bandwidth usage data, the output format is adjusted to match the different playback requirements and network conditions, and frame rate adjustment information is obtained.

3. The network live broadcast system supporting multi-format output according to claim 1, characterized in that: The steps for collecting the device encoding capability and current network status information are specifically as follows: Based on the adjusted frame rate data, the CPU performance and memory capacity data are obtained using the formula: Get a device's coding ability score ,in, and is a weight factor used to adjust the impact of CPU and memory on the score. is the CPU speed of the device, is the memory capacity of the device; Based on the encoding capability score of the device in question, a network monitoring tool was used to assess bandwidth and latency using the formula: Get the current network status score ,in, and is a weighting factor used to adjust the impact of bandwidth and latency in the score, is the network bandwidth, It is the inverse of network latency; Based on the encoding ability score of the device and the current network status score, the formula is used: Get integrated device capability and network status scores ,in, is the device's coding ability score, is the current network status score.

4. The network live broadcast system supporting multi-format output according to claim 1, characterized in that: The steps for obtaining the encoding decision output record are specifically as follows: Analyze the key frames of the video stream and extract key parameters, including frame rate and resolution, using the formula: Get video content feature data ,in, is the video frame rate, is the video resolution, and is a weight factor used to adjust the influence of frame rate and resolution; Based on the video content characteristic data, according to the device and network information score, combined with the video content characteristics, the formula is adopted: Get the output format decision data ,in, It is the video content characteristic data. is the device and network information score, and is a weighting factor used to balance the impact of content characteristics and device network conditions; According to the format decision data, a matching video encoding standard is selected. When the format decision data is greater than a threshold, H.265 is selected; when the format decision data is less than or equal to the threshold, H.264 is selected.

5. The network live broadcast system supporting multi-format output according to claim 1, characterized in that: The monitoring steps of the network bandwidth usage and packet loss rate are specifically as follows: Based on the encoding decision output record, the network bandwidth usage is monitored in real time, and the current bandwidth usage rate is calculated using the formula: Get bandwidth usage results ,in, is the current network bandwidth usage, is the maximum available bandwidth; Based on the bandwidth utilization results, combined with the packet loss rate, the formula is used: Get network performance indicators ,in, and is a weight factor used to adjust the impact of bandwidth utilization and packet loss rate. is the bandwidth utilization result, It is the packet loss rate result.

6. The network live broadcast system supporting multi-format output according to claim 1, characterized in that: The steps for obtaining the network adjustment result are specifically as follows: Based on the network stability, the network fitness score is calculated using the formula: Get the network fitness evaluation results ,in, It is a network performance indicator; Based on the network fitness evaluation result, the adjustment of the video transmission parameters is determined using the formula: Get the adjustment decision results ,in, is the network fitness evaluation result; According to the adjustment decision result, video transmission parameters are adjusted for multiple output formats to obtain network adjustment results.

Citation Information

Patent Citations

  • Image processing apparatus,image processing method and computer readable medium

    CN102325257A

  • A method and system for resisting weak network for audio and video transmission

    CN118101941A

  • Method and Device for Adjusting Frame Rate of Video Recording

    US20160249012A1

  • Sensory information providing apparatus, video analysis engine, and method thereof

    US20180232606A1

  • Video quality evaluation method and system based on network health index

    WO2023024316A1