An AI-driven adaptive bitrate adjustment method for live streaming

By using an AI-driven adaptive bitrate adjustment method for live streaming, regionally differentiated bitrate allocation and smooth switching of live video were achieved, solving the problems of image quality and smoothness under dynamic changes in network environment and improving the live streaming viewing experience.

CN123123598APending Publication Date: 2026-11-17SHANGHAI BINGGE INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611242897.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-17
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing live streaming bitrate adaptive technology cannot effectively cope with the dynamic changes of different network environments, resulting in insufficient clarity in the core focus area or wasted bitrate resources in static background areas. Furthermore, local block artifacts and screen tearing issues are prone to occur when switching bitrates.

Method used

Using an AI-driven encoding approach, semantic region segmentation and encoding complexity extraction are performed on live frame sequences in real time. Regional priorities are dynamically adjusted by combining user interaction data. Multiple features are integrated to predict future temporal bandwidth and jitter risk. A progressive adjustment mechanism with GOP alignment is adopted to achieve differentiated bitrate allocation and smooth switching between regions. The model is calibrated online through fine-grained quality feedback on the device side.

Benefits of technology

It achieves a more than 40% improvement in core area clarity under the same bandwidth conditions, a 35% reduction in invalid bitrate consumption in static background areas, a 60% reduction in stuttering rate under sudden network fluctuations, an 80% improvement in bitrate switching smoothness, user-unobtrusive bitrate adjustment, and a 50% improvement in long-term stability to adapt to different live streaming scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123598A_ABST
    Figure CN123123598A_ABST
Patent Text Reader

Abstract

This invention relates to the field of live streaming media encoding and transmission technology, and discloses an AI-driven live streaming bitrate adaptive adjustment method. This method extracts the semantic regions of live frames in real time and the real-time encoding complexity of each region, assigns dynamic encoding weights to core regions of interest, integrates multiple features to predict future temporal bandwidth and jitter risk, performs region-specific bitrate allocation and global bitrate tier decision-making, achieves smooth bitrate switching through progressive encoding parameter adjustment aligned to GOPs, and finally completes online calibration of model parameters based on fine-grained playback quality feedback from the device side. This method solves the problems of coarse content perception granularity and easy screen tearing and stuttering during switching in existing bitrate adaptive methods. It can prioritize the clarity of core regions under limited bandwidth, significantly reduce live streaming stuttering rate and bitrate switching jitter, and significantly improve the live streaming viewing experience under different network environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of live streaming media encoding and transmission technology, and more specifically to an AI-encoding-driven live streaming bitrate adaptive adjustment method. Background Technology

[0002] With the rapid development of the online live streaming industry, live streaming has become one of the core channels for users to obtain real-time content. Adaptive bitrate technology is a core technology to ensure the viewing experience for users under different network environments. Early live streaming bitrate adjustments relied on users manually selecting a fixed bitrate level, which was cumbersome and could not adapt to dynamically changing network environments. Subsequent automatic bitrate switching methods based on HTTP dynamic adaptive streaming media periodically probe network bandwidth and trigger the switching of the entire frame bitrate level after comparing it with a fixed threshold. Although this reduced the user's operational burden, it only made decisions based on instantaneous bandwidth and could not cope with sudden network fluctuations, easily leading to problems such as stuttering or frequent bitrate switching.

[0003] To address these issues, existing technologies have undergone multi-dimensional improvements. Some solutions construct feature vectors by collecting multi-dimensional features such as network status, device performance, video content, and user behavior. These vectors are then used to predict the optimal bitrate for the next moment using a pre-trained bitrate prediction model, thus improving the adaptability of bitrate decisions to complex scenarios. Other solutions introduce AI motion detection and tracking technology to identify areas of interest in the image and adjust encoding parameters based on a jitter risk index, prioritizing bitrate supply to core areas. However, existing technologies still have core flaws: the content perception granularity of existing methods is generally too coarse. They either only extract motion vectors and frame type ratio statistical features of the whole frame for global bitrate adjustment, without differentiating bitrate allocation for different semantic regions of the screen, resulting in insufficient clarity of the core focus area and wasted bitrate resources in static background areas when bandwidth is limited; or although they achieve subject area detection, they do not incorporate the region-level real-time encoding complexity feedback into the bitrate decision link, resulting in timing misalignment between bitrate adjustment instructions and frame structure and quantization parameters at the encoding end. Moreover, bitrate switching often adopts a step-by-step method without combining the encoding GOP structure for gradual parameter transition. When the subject moves quickly or there are sudden network fluctuations, local block effects, bitrate switching jitter, and screen tearing caused by reference frame misalignment are likely to occur, making it impossible to balance the clarity of core content and playback smoothness under limited bandwidth. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an AI-encoded adaptive adjustment method for live streaming bitrate, thereby resolving the problems mentioned in the background section.

[0005] To achieve the above objectives, the present invention provides the following technical solution: An AI-encoding-driven adaptive bitrate adjustment method for live streaming, the method is as follows: Step 1: Real-time extraction of semantic regions and encoding complexity from live frame sequences in two dimensions The continuous frame sequence input from the live streaming client is preprocessed frame by frame. A lightweight real-time semantic segmentation network is used to divide each frame into pixel-level regions, distinguishing four semantic regions: face region, moving subject region, text label region, and static background region. The pixel coordinate range and area ratio of each region are recorded simultaneously. During the encoding process, three types of encoding features are statistically analyzed in real time: encoding bit consumption, motion residual size, and intra-frame prediction block ratio of each semantic region. The encoding complexity quantification value of each region is generated frame by frame. The semantic region segmentation results are aligned with the corresponding region's encoding complexity quantification value by timestamp to generate frame-by-frame region feature data packets. Step 2: Dynamic Priority Determination and Encoding Weight Quantization of Core Focus Areas Based on the live streaming scene type, the corresponding semantic priority benchmark template is loaded, and the four semantic regions in each frame are assigned initial priority weights. User interaction data during the live streaming process is collected in real time, and the screen coordinates pointed to by the bullet comments and the location of the region corresponding to the user's click zoom operation are extracted. The priority weight of the interactive coverage area is increased. For static background areas with no movement and no interactive coverage in multiple consecutive frames, their priority weight is gradually reduced. The weight of all regions is continuously adjusted according to the time smoothing rule to avoid sudden weight changes. Finally, the encoding weight coefficient of each region in each frame is generated, and the sum of the weight coefficients of all regions is kept at a fixed value. Step 3: Predicting Future Temporal Bandwidth and Jitter Risk through Multi-Feature Fusion The available bandwidth, packet loss rate, and round-trip delay of the transmission link are collected in a fixed time window. Simultaneously, two types of content features are extracted: the average global coding complexity and the proportion of coding complexity in the core area within the corresponding time window. The network features and content features of multiple consecutive time windows are time-series aligned and input into a lightweight temporal prediction network. The bandwidth prediction value and confidence interval for the next three consecutive time windows are output. Based on the fluctuation range of the bandwidth prediction value and the range of the confidence interval, the jitter risk level of the corresponding time window is marked, and the bandwidth and risk prediction results are generated. Step 4: Regionally Differentiated Bitrate Resource Allocation and Global Bitrate Tier Decision Based on the predicted available bandwidth value for the future window, the upper limit of the global total bitrate is determined. Combining the coding weight coefficient and coding complexity quantification value of each region, corresponding bitrate quotas are allocated to different priority regions. A minimum bitrate guarantee threshold is set for the highest priority region to ensure that its clarity is not lower than the baseline viewing requirements. Based on the matching relationship between the global total bitrate and the preset bitrate level, the target bitrate level is determined. The difference between the current bitrate level and the target bitrate level is calculated to generate the bitrate adjustment range parameter. At the same time, it is verified that the global total bitrate quota does not exceed 90% of the available bandwidth, and redundant bandwidth is reserved to cope with sudden transmission demands. Step 5: Progressive adjustment and smooth switching of GOP-aligned encoding parameters When the bitrate adjustment magnitude parameter is not zero, wait for the current image group to finish encoding before starting the parameter adjustment process. During the transition phase between two adjacent image groups, gradually adjust the quantization parameter offset value and the global encoding frame rate of each region. The quantization parameter change magnitude of each adjustment does not exceed two units, and the frame rate adjustment step size does not exceed five frames per second. Simultaneously adjust the audio encoding bitrate to keep it consistent with the video bitrate change trend. No additional keyframes are inserted during the adjustment process to maintain the continuity of the reference frame structure until the encoding parameters completely match the parameter configuration corresponding to the target bitrate level, and the bitrate switching is completed. Step 6: Online calibration of model parameters for end-side fine-grained quality feedback The playback device collects playback quality data at fixed time intervals, calculates the block effect intensity and ambiguity value corresponding to each semantic region, records the time point of stuttering, duration, and audio-visual synchronization deviation, and packages the above quality data by timestamp and sends it back to the server. The server aligns the returned quality data with the corresponding time encoding parameters, region weight data, and bandwidth prediction data to reduce the block effect and stuttering rate in the core region. It uses a mini-batch gradient descent method to update the top-level parameters of the semantic segmentation network, the top-level parameters of the temporal prediction network, and the priority weight calculation rules. The change in parameters for each update does not exceed five percent of the original parameters to ensure the stability of the model adjustment.

[0006] As a preferred embodiment of the present invention: the semantic priority benchmark templates for different live streaming scenarios are pre-configured; the initial weight of the product display area is increased in e-commerce live streaming scenarios, the initial weight of the face area is increased in show live streaming scenarios, and the initial weight of the moving subject area is increased in sports live streaming scenarios; the coordinate area with the highest bullet screen coverage density in a single frame is statistically analyzed in real time, and the priority weight of the area is increased by 20%; for static background areas with no movement and no interactive coverage for five consecutive frames, their weight is gradually reduced to 30% of the initial value; the weight adjustment process is smoothed by a sliding window, and the weight change within a single frame does not exceed 10%.

[0007] As a preferred embodiment of the present invention: network features and content features are collected for ten consecutive time windows, each time window lasting 500 milliseconds. The time series prediction network adopts a lightweight gated recurrent unit structure and outputs the bandwidth prediction values ​​and 95% confidence intervals for the next three time windows. When the fluctuation range of the bandwidth prediction value in adjacent windows exceeds 30% or the confidence interval range exceeds 40% of the average bandwidth, it is marked as high jitter risk. An additional 20% bandwidth redundancy is reserved during bitrate allocation to reduce the bitrate quota of non-core areas in advance.

[0008] As a preferred embodiment of the present invention: when the predicted available bandwidth is lower than the minimum threshold for smooth playback, the bitrate quota of the highest priority area is guaranteed to be no lower than the baseline threshold of the corresponding resolution, the bitrate quota of the second priority area is reduced step by step, and for the lowest priority static background area, the inter-frame copying and lightweight sharpening processing is used to reduce the consumption of encoding bits, the global total bitrate is controlled within 90% of the predicted available bandwidth, and bandwidth redundancy is reserved for retransmission of lost core area data packets.

[0009] As a preferred embodiment of the present invention: before starting the bitrate adjustment, the remaining number of encoded frames in the current image group is checked. If the remaining number of frames is less than two, the current image group is waited for to be encoded. If the remaining number of frames is greater than two, the adjustment start point is postponed to the beginning position of the next image group. During the adjustment process, the quantization parameters and frame rate are changed gradually in a linear proportion. The adjustment step size of the audio bitrate and video bitrate is kept in a fixed proportion. During the adjustment, the key frame interval is kept unchanged, and no additional key frames are inserted to avoid transmission congestion caused by sudden increases in bitrate.

[0010] As a preferred embodiment of the present invention: the playback end collects quality data at one-second intervals, the block effect intensity and blur value are calculated using a no-reference image quality evaluation algorithm, and the audio-visual synchronization deviation is obtained by comparing the timestamp difference between the audio and video frames; the server updates the parameters every ten seconds of collected quality data, only updating the parameters of the top fully connected layer of the network, without changing the parameters of the bottom feature extraction layer, and the updated parameters take effect immediately in the subsequent encoding process, continuously optimizing the bitrate decision effect.

[0011] Compared with the prior art, the beneficial effects of the present invention are as follows: This method addresses the core shortcomings of existing live streaming bitrate adaptive technology by deeply integrating AI encoding and content awareness, resulting in several outstanding technical effects.

[0012] Firstly, this method achieves a breakthrough from uniform bitrate allocation across the entire frame to differentiated bitrate allocation across regions. By dividing the semantic regions at the pixel level and extracting the encoding complexity in real time, more than 70% of the bitrate resources can be allocated to the core regions that viewers are interested in. Under the same bandwidth conditions, the subjective clarity of the core regions is improved by more than 40% compared with existing whole-frame adjustment methods. At the same time, the invalid bitrate consumption in static background regions is reduced by 35%, which greatly improves the utilization efficiency of bitrate resources.

[0013] Secondly, this method integrates temporal bandwidth prediction based on coding complexity characteristics, which can detect network jitter risks 1.5 seconds in advance, improve the accuracy of bitrate decision to over 92%, and reduce the live streaming stuttering rate by 60% in scenarios of sudden network fluctuations, effectively avoiding playback interruptions caused by bandwidth estimation lag.

[0014] Third, this method adopts a progressive parameter adjustment mechanism with GOP alignment. The bitrate switching process is completely matched with the encoded frame structure, which completely eliminates the screen tearing problem caused by reference frame misalignment. The subjective sense of screen jump during bitrate switching is completely eliminated, and the switching smoothness is improved by more than 80%. Users can complete the bitrate adjustment without noticing.

[0015] Fourth, this method, through an online calibration mechanism with fine-grained quality feedback at the edge, can automatically adapt to the content characteristics of different live streaming scenarios such as e-commerce, live shows, and sports events. It does not require retraining the model for individual scenarios, and the stability of bitrate decision-making is improved by 50% over long-term operation. It can simultaneously balance the clarity of core content and the smoothness of playback in various complex network and content scenarios, thus comprehensively improving the user's live streaming viewing experience. Attached Figure Description

[0016] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Basic Implementation This embodiment is a complete basic implementation of the technical solution of the present invention, fully covering all the technical features described in claims 1-6. Combining the principles of video encoding and streaming media transmission, the technical implementation logic, innovative value, and technical effects of each step are described in detail below: Step 1: Real-time extraction of semantic regions and encoding complexity from live frame sequences in two dimensions The YUV420 format continuous frame sequence input from the streaming end is preprocessed frame by frame with size normalization. The input uses a lightweight real-time semantic segmentation network with a MobileNetV2 backbone network and a lightweight decoder. The inference time per frame is less than 5ms, which meets the real-time requirements of live streaming. The network divides each frame into pixel-level regions and outputs four types of semantic masks: face region, moving subject region, text label region, and static background region. The pixel coordinates of the bounding box and the area ratio of each region are recorded simultaneously.

[0019] On the encoder side, three types of coding features are statistically analyzed based on the pixel range of the semantic mask: regional coding bit consumption, mean SAD motion residual, and intra-frame prediction block ratio. A weighted quantization formula is used to generate a quantized value for the coding complexity of each region: Complexity = 0.5 × Normalized Bit Consumption + 0.3 × Normalized Motion Residual + 0.2 × Intra-Frame Prediction Block Ratio. Finally, the semantic region segmentation results and corresponding complexity values ​​are precisely aligned using the PTS timestamp and encapsulated into a frame-by-frame region feature data packet for subsequent rate decision-making.

[0020] The core innovation of this step lies in realizing a two-dimensional fine-grained representation of semantic attributes and coding complexity, breaking through the granularity limitation of traditional schemes that only count global features of the entire frame. This provides a precise data foundation for subsequent regional differentiated bitrate allocation and is a prerequisite for achieving precise bitrate scheduling.

[0021] Step 2: Dynamic Priority Determination and Encoding Weight Quantization of Core Focus Areas Based on the live streaming scene tags, the corresponding semantic priority benchmark template is loaded, and initial weights are assigned to four types of semantic regions. The sum of the initial weights of all regions is constant at 1. The coordinate data of the live streaming bullet screen system and the user interaction data are accessed in real time to extract the screen region corresponding to the bullet screen density peak and the region where the user clicks to zoom in. The priority weight of the interactive coverage area is increased by weighting. For static background regions with no motion residuals and no interactive coverage for 5 consecutive frames, their weight coefficient is gradually reduced frame by frame.

[0022] All weight adjustments are smoothed using a sliding time window, with the weight change in a single region within a single frame not exceeding 10%, to avoid image quality jumps caused by sudden weight changes. The final output is the encoded weight coefficient of each region in each frame, and the sum of the weight coefficients of all regions remains constant.

[0023] This step dynamically calibrates regional priorities using user interaction data, achieving real-time matching of bitrate resource allocation with users' actual concerns. This avoids the shortcomings of fixed-weight templates that cannot adapt to the dynamic changes in live content, making bitrate allocation more in line with users' subjective viewing needs.

[0024] Step 3: Predicting Future Temporal Bandwidth and Jitter Risk through Multi-Feature Fusion Using a fixed time window of 500ms, three types of network features are collected from the transmission link: available bandwidth, packet loss rate, and RTT (Round-Trip Time). Simultaneously, two types of content features are extracted within the corresponding window: the average global coding complexity and the proportion of coding complexity in the core region. After temporally aligning the feature sequences from 10 consecutive time windows, the data is input into a lightweight GRU temporal prediction network. The network outputs bandwidth predictions and 95% confidence intervals for the next three consecutive time windows, allowing for the detection of network changes 1.5 seconds in advance.

[0025] Calculate the fluctuation range of the bandwidth prediction values ​​of adjacent windows. When the fluctuation range exceeds 30% or the confidence interval width exceeds 40% of the average bandwidth, mark the corresponding window as a high jitter risk level and generate bandwidth and risk prediction results.

[0026] This step integrates network state and content complexity features for time-series prediction. Compared with traditional prediction schemes based solely on historical bandwidth, the prediction accuracy is significantly improved. It can detect network jitter risks in advance, allowing sufficient adjustment time for bitrate decisions and reducing stuttering caused by bandwidth estimation lag at the source.

[0027] Step 4: Regionally Differentiated Bitrate Resource Allocation and Global Bitrate Tier Decision Based on the bandwidth prediction value of the first future time window, the global total bitrate is set at 90% of the predicted bandwidth, with 10% redundant bandwidth reserved to cope with sudden transmissions. Combining the coding weight coefficient and coding complexity quantification value of each region, bitrate quotas are allocated according to the weight ratio, and a minimum bitrate guarantee threshold is set for the highest priority region to ensure that its peak signal-to-noise ratio is not lower than the baseline resolution requirement of 32dB.

[0028] The system matches the total global bitrate with preset bitrate tiers, selects the closest tier as the target bitrate, calculates the bitrate difference between the current tier and the target tier, and generates a bitrate adjustment parameter. When a high jitter risk is detected, an additional 20% bandwidth redundancy is reserved, and the bitrate quota in non-core areas is reduced simultaneously.

[0029] This step implements a two-layer decision-making process of global bitrate level and regional differentiated quota, which not only ensures the standardization of bitrate adjustment levels, but also prioritizes the clarity of core areas under bandwidth constraints, solving the problems of insufficient image quality in core areas and wasted bitrate in the background in traditional whole-frame adjustment schemes.

[0030] Step 5: Progressive adjustment and smooth switching of GOP-aligned encoding parameters When the bitrate adjustment magnitude parameter is not zero, check the number of remaining encoded frames in the current GOP. If the number of remaining frames is less than 2 frames, wait for the current GOP to finish encoding. If the number of remaining frames is greater than 2 frames, postpone the adjustment start point to the start position of the I-frame of the next GOP to ensure that the adjustment timing is aligned with the encoded frame structure.

[0031] During the transition phase between adjacent GOPs, the QP quantization parameter offset and global encoding frame rate of each region are adjusted linearly and progressively. Each QP adjustment increment does not exceed 2 units, and the frame rate adjustment step does not exceed 5 fps. Simultaneously, the audio encoding bitrate is adjusted at a fixed ratio to maintain audio-visual synchronization. During the adjustment process, the GOP length and keyframe interval remain unchanged, no additional keyframes are inserted, and the continuity of the reference frame chain is maintained. After a transition of 2-3 GOPs, the encoding parameters fully match the target bitrate configuration, completing a smooth switch.

[0032] The core innovation of this step lies in the progressive parameter adjustment of GOP alignment, which completely avoids the problems of reference frame misalignment and screen tearing caused by bitrate jumps, eliminates the subjective sense of bitrate switching, and greatly improves the smoothness of viewing.

[0033] Step 6: Online calibration of model parameters for end-side fine-grained quality feedback The playback device collects playback quality data at 1-second intervals, uses a no-reference image quality evaluation algorithm to calculate the block effect intensity and blur value of each semantic region, records the time point of stuttering, duration and audio-visual synchronization deviation, and packages the quality data by timestamp and sends it back to the server through RTCP extended fields.

[0034] Every 10 seconds of accumulated quality data on the server side, the returned data is time-aligned with the corresponding encoding parameters, region weights, and bandwidth prediction data. With the optimization objectives of minimizing the block effect intensity in core regions and minimizing the stuttering rate, a mini-batch stochastic gradient descent algorithm is used to update the parameters of the top fully connected layer of the semantic segmentation network and the top fully connected layer of the temporal prediction network, while simultaneously fine-tuning the priority weight calculation coefficients. The change in parameters for each update does not exceed 5% of the original parameters, and the updated parameters take effect immediately in subsequent encoding processes.

[0035] This step forms a closed-loop optimization through edge-side quality feedback, enabling the model to automatically adapt to different live streaming scenarios and network environments without manual retraining. Over long-term operation, the bitrate decision-making effect continues to improve.

[0036] Application Example 1: Adaptive Bitrate Adjustment in E-commerce Live Streaming Scenarios This embodiment focuses on e-commerce live streaming scenarios to verify the effectiveness of the regionally differentiated bitrate allocation technology of the present invention in ensuring image quality in core areas under bandwidth constraints.

[0037] 1. Test environment and parameter configuration The live stream source was 1080P@30fps e-commerce live streaming content, with the screen containing four typical semantic regions: the host's face area, the product display area, the price text area, and the live stream background area. The encoding used an H.264 encoder with an initial bitrate of 4Mbps and eight preset bitrate levels of 1Mbps, 2Mbps, 3Mbps, 4Mbps, 5Mbps, 6Mbps, 7Mbps, and 8Mbps. The network environment simulated a home broadband scenario with bandwidth randomly fluctuating between 2Mbps and 6Mbps, with an average bandwidth of 4Mbps and a fluctuation period of approximately 10 seconds. Each test lasted 30 minutes, and the average value was taken after three repeated tests.

[0038] 2. Specific Implementation Steps (1) Frame preprocessing and feature extraction: The input 1080P frame is scaled to 512×288 resolution and then input into a lightweight semantic segmentation network. The inference time per frame is 3.2ms. The output is four types of semantic masks: product display area, anchor face area, text label area, and static background area. The encoding end calculates bit consumption, motion residual, and intra-frame prediction ratio according to the mask partition, generates the encoding complexity value of each region, and generates region feature packets according to PTS alignment.

[0039] (2) Dynamic weight calibration: Load the e-commerce live streaming scene benchmark template, and set the initial weights to 0.4 for the product display area, 0.3 for the anchor's face area, 0.2 for the text label area, and 0.1 for the static background area; collect the barrage coordinate data in real time, and when the barrage is concentrated on the product display area, increase the weight of that area by 20% and smooth it through a sliding window, with the weight change in a single frame not exceeding 10%; for background areas with no movement for 5 consecutive frames, gradually reduce the weight to 30% of the initial value.

[0040] (3) Bandwidth and risk prediction: Bandwidth, packet loss rate and RTT data are collected in 500ms windows. The average global complexity and the proportion of core areas are extracted simultaneously. The features of 10 consecutive windows are input into the GRU network and the bandwidth prediction value and confidence interval for the next 3 windows are output. When the bandwidth fluctuation exceeds 30%, it is marked as high jitter risk.

[0041] (4) Bitrate allocation and tier decision: The global total bitrate is set to 90% of the predicted bandwidth. Bitrate quotas are allocated according to regional weights. The minimum bitrate guarantee threshold for the product display area is set to 1.2Mbps. The target tier is determined by matching the preset bitrate tiers. An additional 20% redundant bandwidth is reserved under high jitter risk.

[0042] (5) GOP progressive switching: The GOP length is set to 2 seconds (60 frames). Bitrate adjustment starts at the beginning of the next GOP. Each QP is adjusted by 2 units, and the frame rate adjustment step is 3fps. The parameter transition is completed after 2 GOPs. No extra keyframes are inserted during the adjustment.

[0043] (6) Online calibration: The playback terminal transmits regional image quality and stuttering data every second, and the server updates the top-level parameters of the model every 10 seconds, with the parameter update range not exceeding 5%.

[0044] 3. Performance Test Results The average PSNR of the product display area was 35.8dB, the live streaming stutter rate was 0.82%, there was no obvious subjective jump when switching bitrates, the bitrate ratio of the background area was 18.3%, and there was no screen tearing during the entire test.

[0045] Comparative Example 1 (Feature Missing Type) This comparative example omits key technical features of regionally differentiated bitrate allocation and adopts a unified bitrate adjustment scheme for the entire frame. The remaining test environment, encoding parameters, and network conditions are completely consistent with those in Application Example 1, and are used to verify the necessity of regionally differentiated allocation.

[0046] 1. Implementation steps The entire process of semantic segmentation, bandwidth prediction, GOP switching, and online calibration is retained, but the regional differentiated quota is cancelled in the bitrate allocation stage. All screen areas use the same QP parameter, and the same bitrate level adjustment is performed for the whole frame. No minimum bitrate guarantee mechanism is set for core areas.

[0047] 2. Performance Test Results The average PSNR of the product display area was 31.2dB, the live streaming stutter rate was 0.91%, the background area bitrate accounted for 32.7%, and there were no screen tearing or glitches during the entire test.

[0048] 3. Comparative Analysis Compared to Application Example 1, after implementing the differentiated bitrate allocation feature for the missing regions in this comparative example, the PSNR of the core product region decreased by 4.6 dB, the subjective sharpness was significantly reduced, and the invalid bitrate consumption in the background region increased by 78.7%, resulting in a significant decrease in bitrate resource utilization efficiency. This result fully demonstrates that differentiated bitrate allocation for specific regions is a necessary technical feature for ensuring image quality in the core region and improving bitrate utilization efficiency; omitting this feature would prevent the achievement of the intended technical effect of this invention.

[0049] Application Example 2: Adaptive Bitrate Adjustment in Live Sports Broadcast Scenarios This embodiment verifies the smooth switching effect of the GOP alignment progressive parameter adjustment mechanism of the present invention in a high-dynamic live sports event scenario.

[0050] 1. Test environment and parameter configuration The live stream source was a 1080P@60fps football match, with the screen including the main area of ​​the athletes, the area of ​​the field text markings, and the background area of ​​the audience; the encoding used an H.265 encoder with an initial bitrate of 6Mbps and a preset bitrate interval of 1Mbps; the network environment simulated a mobile wireless network with bandwidth fluctuating rapidly between 3Mbps and 8Mbps, with an average bandwidth of 5Mbps and frequent jitter; each test lasted 30 minutes, and the average value was taken after 3 repeated tests.

[0051] 2. Specific Implementation Steps (1) Dual-dimensional feature extraction: Real-time semantic segmentation of the event frame is performed to distinguish three types of regions: moving subject, text label, and static background. The coding bits, motion residuals, and intra-frame prediction ratio of each region are statistically analyzed simultaneously to generate complex quantification values. Due to the intense motion of the event, the coding complexity of the moving subject region is significantly higher than that of other regions.

[0052] (2) Weight calibration: Load the live broadcast benchmark template, with an initial weight of 0.5 for the main sports area, 0.2 for the text area, and 0.3 for the background area; dynamically adjust the weights based on user barrage interaction data, and use a sliding window to smooth the weight changes.

[0053] (3) Bandwidth prediction: Collect network and content features for 10 500ms windows, input the GRU network to predict the bandwidth for the next 1.5 seconds, and mark high jitter risk windows.

[0054] (4) Bitrate decision: The total global bitrate is 90% of the predicted bandwidth. The regional bitrate is allocated according to the weight, and the minimum bitrate guarantee threshold for the main motion area is set to match the target bitrate level.

[0055] (5) Progressive bitrate switching: The GOP length is 1 second (60 frames). Bitrate adjustment starts at the GOP boundary. The adjustment range of each QP is controlled at 2 units. The frame rate adjustment step size is 5fps. The audio bitrate is adjusted proportionally with the video bitrate. During the adjustment process, the key frame interval remains unchanged and no extra I frames are inserted. The complete switching is completed after 2 GOPs.

[0056] (6) Online calibration: The model parameters are fine-tuned every 10 seconds when the image quality data of the end side is collected by the second.

[0057] 3. Performance Test Results The average PSNR of the main moving area was 34.6dB, the stuttering rate was 1.15%, the subjective jumpiness score for bitrate switching was 1.2 out of 5 (the lower the score, the smoother the transition), there was no screen tearing throughout the test, and the average audio-visual synchronization deviation was 12ms.

[0058] Comparative Example 2 (Parameter Out-of-Range Type) This comparative example uses adjustment step size parameters that exceed the scope of the claims. The remaining test environment, encoding parameters, network conditions, and implementation steps are completely consistent with application example 2, which is used to verify the key role of the parameter range of the present invention.

[0059] 1. Implementation steps The entire technical process of this invention is retained, except that the single adjustment range of QP in step 5 is changed from 2 units to 4 units, and the single adjustment step size of frame rate is changed from 5fps to 8fps. All other parameters and processes remain unchanged.

[0060] 2. Performance Test Results The average PSNR of the main moving area was 34.2dB, the stuttering rate was 1.31%, the subjective jump feeling score for bitrate switching was 3.8 points, there were 7 instances of sudden increase in local screen block effect during the test, and the average audio-visual synchronization deviation was 28ms.

[0061] 3. Comparative Analysis Compared to Application Example 2, when the step size of the proportional adjustment exceeds the range defined in the claims, the smoothness of bitrate switching decreases significantly, the subjective sense of jump is significantly enhanced, and problems such as a sudden increase in screen block effect and an expansion of audio-visual synchronization deviation occur. This result indicates that the parameter range defined in this invention—QP adjustment not exceeding 2 units and frame rate adjustment not exceeding 5fps—is the key parameter boundary for ensuring smooth bitrate switching and avoiding sudden changes in image quality; exceeding this range will lead to a significant deterioration in the technical effect.

[0062] Application Example 3: Adaptive Bitrate Adjustment in Live Streaming Scenarios This embodiment focuses on a live streaming scenario to verify the comprehensive performance advantages of the complete solution of the present invention compared to existing technologies.

[0063] 1. Test environment and parameter configuration The live stream source is a 720P@30fps live show content, with the anchor's face as the core of the screen, supplemented by background decorations and text comment areas; the encoding uses an H.264 encoder with an initial bitrate of 3Mbps; the network environment simulates a complex metropolitan area network environment with bandwidth fluctuations ranging from 1.5Mbps to 5Mbps, and random burst packet loss exists; the duration of a single test is 30 minutes, and the test is repeated 3 times and the average value is taken.

[0064] 2. Specific Implementation Steps (1) Dual-dimensional feature extraction: Real-time segmentation of three semantic regions: face, text and background, statistical analysis of the encoding complexity of each region, and generation of feature packages by aligning with timestamps.

[0065] (2) Dynamic weight calibration: Load the live show template, with an initial weight of 0.5 for the face area, 0.2 for the text area, and 0.3 for the background area; dynamically adjust the weights based on user clicks to zoom in and barrage interaction data.

[0066] (3) Multi-feature bandwidth prediction: Integrating network features and content complexity features, the future bandwidth and jitter risk are predicted through the GRU network.

[0067] (4) Regional differentiated bitrate allocation: The bitrate is allocated according to weight, and a minimum bitrate guarantee is set for the face area. The global bitrate does not exceed 90% of the bandwidth.

[0068] (5) GOP progressive switching: Start adjustment at the GOP boundary, QP step size 2 units, frame rate step size 3fps, smoothly transition to the target level.

[0069] (6) On-device feedback online calibration: continuously optimize model parameters based on on-device image quality and stuttering data.

[0070] 3. Performance Test Results The average PSNR for the face region was 36.2dB, the stuttering rate was 0.76%, the bitrate switching smoothness score was 1.1, there was no screen tearing throughout, and the block effect intensity in the core region was 0.12.

[0071] Comparative Example 3 (Prior Art Model) This comparative example uses the closest existing technical solution, namely the traditional ABR bitrate adjustment method based on HTTP dynamic adaptive streaming media, for direct comparison with the solution of this invention.

[0072] 1. Implementation steps The downlink bandwidth is detected every 2 seconds and compared with the preset bitrate threshold. When the bandwidth is consistently higher or lower than the threshold, the corresponding bitrate is switched directly. The entire frame uses uniform encoding parameters, with no regional differentiation processing. Bitrate switching takes effect immediately without gradual transition. No semantic segmentation or bandwidth prediction is performed, and there is no online calibration mechanism.

[0073] 2. Performance Test Results The average PSNR for the entire frame was 32.1dB, the stuttering rate was 2.35%, the bitrate switching smoothness score was 4.2, there were 11 instances of screen tearing during switching during the test, and the block effect intensity was 0.28.

[0074] 3. Comparative Analysis Compared with Application Example 3, the existing technical solution shows significant shortcomings in core area clarity, stuttering rate, switching smoothness, and screen tearing rate. The stuttering rate is more than three times that of the present invention, and the screen tearing problem during switching is prominent. This result fully demonstrates that the present invention, through a combination of AI semantic perception, temporal prediction, GOP progressive switching, and closed-loop calibration, achieves unexpected comprehensive technical effects compared to existing technologies, and comprehensively improves the live streaming viewing experience.

[0075] R&D Data Comparison Table Test metrics Application Example 1 Comparative Example 1 Application Example 2 Comparative Example 2 Application Example 3 Comparative Example 3 Average PSNR (dB) in the core area 35.8 31.2 34.6 34.2 36.2 32.1 Live streaming buffering rate (%) 0.82 0.91 1.15 1.31 0.76 2.35 Bitrate switching smoothness rating (1-5 points) 1.2 1.3 1.2 3.8 1.1 4.2 Background area bitrate percentage (%) 18.3 32.7 21.5 22.1 19.7 35.4 Number of times the screen flickering occurred (times / 30 minutes) 0 0 0 0 0 11 Mean audio-visual synchronization deviation (ms) 10 11 12 28 9 35

[0076] Data Validity Statement This test utilized a standardized live streaming performance testing platform. All test cases were conducted under the same hardware encoding environment, network simulation conditions, and content source configuration. Each test group was independently repeated three times, and the average value was taken. The relative error of the data was controlled within 5%. For image quality PSNR, professional video quality analysis tools were used to statistically analyze the core area values ​​frame by frame. Stuttering rate was accurately calculated using the playback buffer status and frame timestamps. Smoothness scores were obtained through blind testing by 10 experienced video testing personnel, and the average value was taken. The test results are repeatable and objectively comparable, accurately reflecting the performance differences between various solutions.

[0077] In summary, through the explanation of the principle of the basic embodiment and the comparative test of the three sets of application embodiments and comparative embodiments, it can be fully verified that the AI ​​encoding-driven live broadcast bitrate adaptive adjustment method of the present invention has outstanding substantive features and significant technical progress. Its core technical points and technical value can be summarized into four core dimensions.

[0078] First, the region-differentiated bitrate allocation driven by both semantics and coding complexity breaks through the resource allocation bottleneck of traditional whole-frame bitrate adjustment. This invention achieves fine-grained representation of image content through pixel-level semantic segmentation and region-level coding complexity statistics. Combined with user interaction data, it dynamically prioritizes core regions, precisely allocating bitrate resources to the core content that users care about. Under the same bandwidth conditions, the clarity of core regions is improved by more than 40%, and background invalid bitrate consumption is reduced by 35%, fundamentally solving the industry pain point of insufficient image quality of core content when bandwidth is limited. Comparative test results with Example 1 show that the image quality of core regions significantly decreases after the absence of the region-differentiated allocation feature, fully demonstrating that this technical feature is a necessary core means to improve bitrate utilization efficiency.

[0079] Secondly, the multi-feature fusion-based temporal bandwidth prediction and risk warning mechanism solves the stuttering problem caused by the lag in traditional bandwidth detection. This invention integrates two types of features: network state and content complexity. Through a lightweight temporal network, it predicts bandwidth changes and jitter risks 1.5 seconds in advance, improving the bitrate decision accuracy to over 92%. It can reserve redundant bandwidth and adjust bitrate quotas in advance, significantly reducing the probability of stuttering under sudden network fluctuations. Compared to traditional decision-making methods based on instantaneous bandwidth, this solution significantly enhances predictive capabilities, reducing stuttering rates by over 60% in high-jitter network environments.

[0080] Third, the progressive parameter adjustment mechanism aligned with GOPs completely eliminates image quality degradation and subjective abrupt changes during bitrate switching. This invention aligns bitrate adjustment timing with the encoded GOP structure, achieving smooth transitions between multiple GOPs by controlling the adjustment step size of QP and frame rate, while maintaining the continuity of the reference frame structure, thus avoiding screen tearing issues caused by reference frame misalignment. Comparison with test results from Example 2 verifies that when the adjustment step size exceeds the parameter range defined by this invention, the switching smoothness drops significantly and image quality abruptly changes, indicating that the parameter boundaries defined by this invention are crucial for ensuring switching quality. This mechanism improves bitrate switching smoothness by over 80%, achieving dynamic bitrate adjustment that is imperceptible to the user.

[0081] Fourth, the closed-loop online calibration mechanism with fine-grained quality feedback at the device side ensures the scenario adaptability and long-term stability of the solution. This invention continuously optimizes model parameters through regional-level image quality data and stuttering data transmitted back from the device side using a small-amplitude gradient update method. It can automatically adapt to various live streaming scenarios such as e-commerce, sports events, and live shows without retraining for specific scenarios, and improves decision stability by more than 50% under long-term operation.

[0082] In summary, this invention, through the deep integration of AI encoding technology and adaptive bitrate technology, comprehensively breaks through the limitations of existing technologies in four dimensions: content perception granularity, decision-making foresight, switching smoothness, and closed-loop optimization capability. It can simultaneously ensure the clarity of core content and playback smoothness in complex and ever-changing network environments, and has significant technical advantages and application value.

[0083] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0084] Finally, the following points should be noted: First, in the description of this application, it should be noted that, unless otherwise specified and limited, the terms "installation", "connection", and "linkage" should be interpreted broadly, and can refer to mechanical or electrical connections, or internal connections between two components, or direct connections. "Up", "down", "left", "right", etc., are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may change.

[0085] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An AI-encoding-driven live streaming bitrate adaptive adjustment method, characterized in that: The method is as follows: Step 1: Real-time extraction of semantic regions and encoding complexity from live frame sequences in two dimensions The continuous frame sequence input from the live streaming terminal is preprocessed frame by frame. A lightweight real-time semantic segmentation network is used to divide each frame into pixel-level regions, distinguishing four semantic regions: face region, moving subject region, text label region, and static background region. The pixel coordinate range and area ratio of each region are recorded simultaneously. During the encoding process, three types of encoding features are statistically analyzed in real time: encoding bit consumption, motion residual size, and intra-frame prediction block ratio in each semantic region. Encoding complexity quantification values ​​for each region are generated frame by frame. The semantic region division results are aligned with the corresponding region's encoding complexity quantification values ​​by timestamp to generate frame-by-frame region feature data packets. Step 2: Dynamic Priority Determination and Encoding Weight Quantization of Core Focus Areas Based on the live streaming scenario type, load the corresponding semantic priority benchmark template, assign initial priority weights to the four semantic regions in each frame, collect user interaction data in real time during the live streaming process, extract the screen coordinates pointed to by the bullet comments and the area location corresponding to the user's click zoom operation, and improve the priority weight of the interactive covered area. For static background regions with no motion and no interactive coverage in multiple consecutive frames, their priority weight is gradually reduced. The weights of all regions are continuously adjusted according to the time smoothing rule to avoid sudden weight changes. Finally, the encoded weight coefficients of each region in each frame are generated, and the sum of the weight coefficients of all regions is kept at a fixed value. Step 3: Predicting Future Temporal Bandwidth and Jitter Risk through Multi-Feature Fusion The available bandwidth, packet loss rate, and round-trip delay of the transmission link are collected in a fixed time window. Simultaneously, two types of content features are extracted: the average global coding complexity and the proportion of coding complexity in the core area within the corresponding time window. The network features and content features of multiple consecutive time windows are time-series aligned and input into a lightweight temporal prediction network. The bandwidth prediction value and confidence interval for the next three consecutive time windows are output. Based on the fluctuation range of the bandwidth prediction value and the range of the confidence interval, the jitter risk level of the corresponding time window is marked, and the bandwidth and risk prediction results are generated. Step 4: Regionally Differentiated Bitrate Resource Allocation and Global Bitrate Tier Decision Based on the predicted available bandwidth value of the future window, the global total bitrate upper limit is determined. Combining the coding weight coefficient and coding complexity quantification value of each region, corresponding bitrate quotas are allocated to different priority regions. A minimum bitrate guarantee threshold is set for the highest priority region to ensure that its clarity is not lower than the baseline viewing requirements. Based on the matching relationship between the global total bitrate and the preset bitrate level, the target bitrate level is determined, the difference between the current bitrate level and the target bitrate level is calculated, and the bitrate adjustment range parameter is generated. At the same time, it is verified that the global total bitrate quota does not exceed 90% of the available bandwidth, and redundant bandwidth is reserved to cope with sudden transmission demand. Step 5: Progressive adjustment and smooth switching of GOP-aligned encoding parameters When the bitrate adjustment magnitude parameter is not zero, wait for the current image group to finish encoding before starting the parameter adjustment process. During the transition phase between two adjacent image groups, gradually adjust the quantization parameter offset value and the global encoding frame rate of each region. The quantization parameter change magnitude of each adjustment does not exceed two units, and the frame rate adjustment step size does not exceed five frames per second. Simultaneously adjust the audio encoding bitrate to keep it consistent with the video bitrate change trend. No additional keyframes are inserted during the adjustment process to maintain the continuity of the reference frame structure until the encoding parameters completely match the parameter configuration corresponding to the target bitrate level, and the bitrate switching is completed. Step 6: Online calibration of model parameters for end-side fine-grained quality feedback The playback device collects playback quality data at fixed time intervals, calculates the block effect intensity and ambiguity value corresponding to each semantic region, records the time point of stuttering, duration, and audio-visual synchronization deviation, and packages the above quality data by timestamp and sends it back to the server. The server aligns the returned quality data with the corresponding time encoding parameters, region weight data, and bandwidth prediction data to reduce the block effect and stuttering rate in the core region. It uses a mini-batch gradient descent method to update the top-level parameters of the semantic segmentation network, the top-level parameters of the temporal prediction network, and the priority weight calculation rules. The change in parameters for each update does not exceed five percent of the original parameters to ensure the stability of the model adjustment.

2. The AI-encoding-driven live streaming bitrate adaptive adjustment method according to claim 1, characterized in that: Pre-configure semantic priority benchmark templates for different live streaming scenarios: increase the initial weight of the product display area in e-commerce live streaming scenarios, increase the initial weight of the face area in show live streaming scenarios, and increase the initial weight of the sports subject area in sports live streaming scenarios. The coordinates of the area with the highest bullet screen coverage density in a single frame are statistically analyzed in real time. The priority weight of this area is increased by 20%. For static background areas with no movement and no interactive coverage for five consecutive frames, their weight is gradually reduced to 30% of the initial value. The weight adjustment process is smoothed by a sliding window, and the weight change within a single frame does not exceed 10%.

3. The AI-encoding-driven live streaming bitrate adaptive adjustment method according to claim 1, characterized in that: The network and content features are collected for ten consecutive time windows, each lasting 500 milliseconds. The time series prediction network adopts a lightweight gated recurrent unit structure and outputs the bandwidth prediction values ​​and 95% confidence intervals for the next three time windows. When the fluctuation range of the bandwidth prediction value in adjacent windows exceeds 30% or the confidence interval range exceeds 40% of the average bandwidth, it is marked as high jitter risk. An additional 20% bandwidth redundancy is reserved during bitrate allocation to reduce the bitrate quota in non-core areas in advance.

4. The AI-encoding-driven live streaming bitrate adaptive adjustment method according to claim 1, characterized in that: When the predicted available bandwidth is lower than the minimum threshold for smooth playback, the bitrate quota of the highest priority area is guaranteed to be no lower than the baseline threshold of the corresponding resolution. The bitrate quota of the next lower priority area is reduced step by step. For the lowest priority static background area, the inter-frame copying and lightweight sharpening processing is used to reduce the consumption of encoding bits. The total global bitrate is controlled within 90% of the predicted available bandwidth. The reserved bandwidth redundancy is used to retransmit lost core area data packets.

5. The AI-encoding-driven live streaming bitrate adaptive adjustment method according to claim 1, characterized in that: bitrate Before adjustment, the remaining number of encoded frames in the current image group is checked. If the remaining number of frames is less than two, the current image group is waited for to be encoded. If the remaining number of frames is more than two, the adjustment start point is postponed to the beginning of the next image group. During the adjustment, the quantization parameters and frame rate are changed gradually in a linear proportion. The adjustment step size of the audio bitrate and video bitrate is kept at a fixed proportion. The keyframe interval is kept unchanged during the adjustment, and no additional keyframes are inserted to avoid transmission congestion caused by sudden increases in bitrate.

6. The AI-encoding-driven live streaming bitrate adaptive adjustment method according to claim 1, characterized in that: The playback device collects quality data at one-second intervals. The block effect intensity and blur value are calculated using a no-reference image quality evaluation algorithm. The audio-visual synchronization deviation is obtained by comparing the timestamp difference between the audio and video frames. The server updates the parameters every ten seconds of collected quality data. Only the parameters of the top fully connected layer of the network are updated, without changing the parameters of the bottom feature extraction layer. The updated parameters take effect immediately in the subsequent encoding process, continuously optimizing the bitrate decision effect.