Rapid decoding transmission method and system for live video images

Abstract

The invention relates to the technical field of video decoding, in particular to a fast decoding transmission method and system for a video live broadcast image. The method comprises the following steps: acquiring a video live broadcast picture stream; performing image group division on the video live broadcast image flow to generate image group images; analyzing an inter-frame relationship of the picture grouping images, defining priorities of the picture grouping images according to the inter-frame relationship, and generating decoding priority mark data; entropy data fragmentation is carried out on the picture grouping images, and a picture basic entropy slice and a picture increment entropy slice are generated; stage picture transmission is carried out on the picture grouping images based on the picture basic entropy slice and the picture increment entropy slice, and an image streaming preloading strategy is generated; and performing terminal transmission feedback on the picture grouping images by using an image stream type preloading strategy to obtain terminal transmission feedback data. According to the invention, through optimization of video picture grouping, priority scheduling, bandwidth management, decoding acceleration and fault-tolerant reconstruction, the rapid decoding transmission performance of the video live broadcast image is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video decoding, and particularly to a fast decoding and transmission method and system for video live broadcast images. Background Art

[0002] Initially, the transmission of video live broadcast images mainly relied on traditional compression algorithms such as H.263 and MPEG-2. These compression algorithms provided a basis for reducing bandwidth requirements. However, due to high computational complexity and slow decoding speed, they limited the realization of real-time transmission and low-latency live broadcast. With the improvement of video resolution and the increasing demand for transmission quality, more efficient video coding standards such as H.264 and H.265 emerged. These new standards significantly improved the compression efficiency through more advanced compression technologies, reduced bandwidth occupancy, and improved video quality, becoming the mainstream video coding standards. However, even efficient coding standards such as H.264 and H.265 still face problems of real-time performance and decoding speed. To overcome these challenges, in recent years, innovative decoding and transmission methods for video live broadcast images have been developed. Technologies such as parallel computing, hardware acceleration, and artificial intelligence have been used to significantly improve the decoding efficiency. However, currently existing technologies usually rely on static coding and transmission methods, with inflexible bandwidth utilization, resulting in bandwidth waste or transmission bottlenecks. At the same time, traditional decoding methods will face computational pressure, especially in the case of complex pictures or high-resolution videos. Summary of the Invention

[0003] Based on this, it is necessary to provide a fast decoding and transmission method and system for video live broadcast images to solve at least one of the above technical problems.

[0004] To achieve the above object, a fast decoding and transmission method for video live broadcast images, the method includes the following steps:

[0005] Step S1: Obtain a video live broadcast picture stream; divide the video live broadcast picture stream into groups of pictures, generate grouped picture images; analyze the inter-frame relationship of the grouped picture images, and define the priority of the grouped picture images according to the inter-frame relationship to generate decoding priority marking data;

[0006] Step S2: Perform entropy data sharding on the grouped picture images to generate basic picture entropy slices and incremental picture entropy slices; perform stage picture transmission on the grouped picture images based on the basic picture entropy slices and the incremental picture entropy slices to generate an image streaming preloading strategy;

[0007] Step S3: Use the image streaming preloading strategy to perform terminal transmission feedback on the grouped picture images to obtain terminal transmission feedback data; perform dynamic transmission scheduling on the terminal transmission feedback data based on the decoding priority marking data to generate optimized grouped picture images for transmission;

[0008] Step S4: Entropy flow fusion is performed on the optimized packetized images of the video stream by a hardware decoder to generate a reorganized video live stream; error-tolerant reconstruction is performed on the reorganized video live stream to execute a fast decoding and transmission operation for video live images.

[0009] By analyzing the inter-frame relationship and generating decoding priority marking data in step S1, the present invention can achieve dynamic scheduling of key frames and predicted frames, making the decoding order more targeted, improving the overall decoding efficiency and response speed. In step S2, by dividing the basic entropy slices and incremental entropy slices of the video stream and adopting a phased video stream transmission mechanism, the data volume is effectively compressed, reducing the bandwidth resource occupation while ensuring the fast transmission of key content. By using the terminal transmission feedback and dynamic transmission scheduling mechanism in step S3, real-time regulation is performed according to the actual transmission situation to avoid the impact of network fluctuations on image quality and coherence, improving the system robustness. In step S4, a hardware decoder is used to perform entropy flow fusion on the optimized packetized images and combined with an error-tolerant reconstruction mechanism, which can quickly reorganize a high-quality video live stream, ensure the smoothness and clarity of the live images, and reduce latency. The introduction of the video stream preloading strategy enables the terminal to perform pre-rendering processing before fully obtaining the image content, reducing the first-frame loading time and enhancing the user viewing experience. This method is applicable to various types of network environments and terminal devices, with good scalability and compatibility, facilitating deployment and application in multiple scenarios such as video conferencing, live streaming platforms, and telemedicine. Therefore, the present invention improves the fast decoding and transmission performance of video live images by optimizing video stream grouping, priority scheduling, bandwidth management, decoding acceleration, and error-tolerant reconstruction.

[0010] Preferably, step S1 includes the following steps:

[0011] Step S11: Obtain the video live stream.

[0012] Step S12: Extract the frame images of the video live stream and perform GOP division on the frame images to obtain packetized images of the video stream, where GOP division includes key frame division, forward prediction frame division, and bidirectional prediction frame division.

[0013] Step S13: Analyze the inter-frame reference relationship of the packetized image data to obtain the inter-frame relationship of the packetized images; perform structural mapping on the packetized image data according to the inter-frame relationship of the packetized images to generate a decoding dependency graph.

[0014] Step S14: Analyze the dependency levels of the decoding dependency graph to obtain frame type priority marking data; perform label mapping on the packetized images through the frame type priority marking data to generate decoding priority marking data.

[0015] By adopting the GOP (Group of Pictures) partitioning method, the present invention divides video frames into key frames (I-frames), forward predicted frames (P-frames), and bidirectional predicted frames (B-frames), which helps to clarify the reference structure between frames, thereby providing basic data organization for subsequent optimized transmission and decoding. By parsing the inter-frame reference relationship in S13 and constructing a decoding dependency graph, the decoding order and dependency path between frames can be presented in the form of a graph structure, enabling the system to master the decoding logic at the structural level, facilitating decoding scheduling and path optimization. By analyzing the hierarchical structure of the decoding dependency graph in S14, the frame type priority is further extracted, and decoding priority marking data is generated to achieve priority marking and regulation of image grouping. This mechanism not only improves the intelligence level of the decoding strategy but also effectively guarantees the priority transmission of key pictures. The decoding priority marking data serves as the basis for subsequent transmission and scheduling, enabling the entire system to flexibly adapt under different network conditions, adaptively optimize the decoding order, and improve the overall response ability of the system and the user viewing experience.

[0016] Preferably, in step S13, the structural mapping of the picture group image data according to the inter-frame relationship of the picture group image includes:

[0017] Identifying the frame type of the picture group image data according to the inter-frame relationship of the picture group image;

[0018] Extracting the reference index of the picture group image data based on the frame type to obtain reference frame index data;

[0019] Modeling the inter-frame connection relationship of the reference frame index data to generate initial inter-frame dependency graph data;

[0020] Performing a directed graph structure mapping on the initial inter-frame dependency graph data to generate frame dependency path graph data, where nodes represent frames and edges represent reference relationships;

[0021] Performing a layer-level analysis on the frame dependency path graph data to generate decoding dependency level data;

[0022] Performing a structural encoding on the decoding dependency level data to generate a decoding dependency graph.

[0023] By identifying the frame type based on the inter-frame reference relationship, the system can not only accurately distinguish key frames (I), forward-predicted frames (P), and bidirectional-predicted frames (B), but also identify the deep-level dependency relationships between frames, providing a semantic label basis for image scheduling and transmission. The inter-frame reference relationship is transformed into reference frame index data, and through the modeling of the inter-frame connection relationship, the initial inter-frame dependency graph data is formed, realizing the transformation from a linear frame sequence to a graph structure model, and improving the system's understanding and expression ability of the inter-frame dynamic relationship. The frame dependency path graph is constructed by using a directed graph structure mapping method, where nodes represent frames and edges represent inter-frame reference relationships, which is beneficial to subsequent operations such as path traversal, graph traversal optimization, and frame scheduling sorting, enhancing the flexibility of the system's control over the frame structure. By generating decoding dependency level data through layer-level analysis, the system can hierarchically and stage-by-stage control the decoding priority of frames, facilitating the implementation of "decoding on demand" that progresses layer by layer under complex network or limited decoding resource conditions. The finally generated decoding dependency graph has good storability and reusability through structure encoding, and can be used as a unified reference structure for subsequent decoding scheduling, entropy coding strategy, and even multi-threaded parallel decoding, improving the system performance and generality.

[0024] Preferably, the entropy data sharding of the picture group image in step S2 includes:

[0025] Perform frame image pixel statistics on the picture group image to obtain frame set entropy distribution data, where the frame-level entropy distribution data contains the local information entropy value of each frame;

[0026] Perform regional difference clustering on the frame set entropy distribution data to generate picture entropy clustering data; extract the core entropy region of the frame set entropy distribution data according to the picture entropy clustering data, and mark the non-core entropy region as the basic entropy region;

[0027] Reconstruct the codewords for the basic entropy region to generate the picture basic entropy slice, where the picture basic entropy slice contains the minimum decoding information necessary for the frame structure;

[0028] Perform trend analysis on the core entropy region in the picture group image data, and perform mutation metric coding on the core entropy region based on the trend analysis result to generate the picture incremental entropy slice.

[0029] The present invention obtains the frame set entropy distribution data through frame image pixel statistics, enabling the system to accurately measure the information content distribution in each frame of the image, providing a basis for subsequent segmentation, and avoiding information redundancy or loss of key regions caused by traditional uniform compression strategies. The regional difference clustering method is used to structurally classify the high and low entropy distributions, clearly distinguishing the core information regions and redundant regions of the image, and improving the accuracy allocation ability of subsequent encoding and decoding resources. By performing codeword reconstruction on non-core entropy regions and only retaining the necessary minimum decoding information, the basic entropy slices of the picture are generated, effectively compressing the data transmission volume and providing support for scenarios such as video preloading and fast startup. The change trend of the core entropy region is analyzed, and differential coding is performed in combination with the variation metric to generate the incremental entropy slices of the picture, realizing targeted information compensation, especially suitable for incremental decoding transmission in high dynamic scenarios. The division of the basic entropy slices and the incremental entropy slices introduces a hierarchical structure into the video data transmission, enabling the terminal to preferentially load the base layer according to the network condition or computing power condition, and then gradually supplement the enhancement layer, significantly improving the robustness and adaptability of the system. Since the system can first rely on the basic entropy slices to restore the key structure picture during transmission, even in an environment with a high packet loss rate or a weak network, the basic video picture decoding can still be ensured without interruption, optimizing the user viewing experience. The entropy clustering data and the variation metric coding results can be used as input parameters for the dynamic scheduling strategy, facilitating the system to dynamically adjust the transmission order and bit resource allocation according to the inter-frame variation intensity or the buffer status on the user side.

[0030] Preferably, the step S2 of performing stage picture transmission on the picture group image based on the basic entropy slices of the picture and the incremental entropy slices of the picture includes:

[0031] Identify key frames for the basic entropy slices of the picture, and perform chronological reorganization on the identified key frames to generate basic decoding order data;

[0032] Perform hierarchical transmission scheduling on the basic entropy slices of the picture according to the basic decoding order data to generate a preloading priority sequence data, where the key frames are transmitted first; perform prefetch buffer control on the preloading priority sequence data to generate a terminal buffer control instruction;

[0033] Use the terminal buffer control instruction to perform local decoding start processing on the basic entropy slices of the picture to generate a preloading stage;

[0034] Calculate the frequency domain energy of the incremental entropy slices of the picture to generate regional frequency intensity distribution data; perform significance screening on the regional frequency intensity distribution data to generate a set of high-frequency feature regions;

[0035] Perform bandwidth-adaptive compression on the set of high-frequency feature regions to generate a supplementary transmission data packet group under bandwidth limitation; perform network scheduling on the supplementary transmission data packet group to generate dynamic transmission plan data; perform terminal synchronization control on the dynamic transmission plan data to obtain an incremental supplementary transmission stage;

[0036] Based on the stage matching execution of the grouped images of the screen in the preloading stage and the incremental supplementary transmission stage, an image streaming preloading strategy is generated.

[0037] In the present invention, by separately processing the basic entropy slice of the screen and the incremental entropy slice of the screen into the "preloading stage" and the "incremental supplementary transmission stage", a phased decoding model with clear logic and parallel scheduling is formed, significantly improving the loading and response efficiency of the system for live videos. Through the key frame extraction and timing reconstruction operations in the basic entropy slice, the frame dependence continuity during the decoding initialization process is ensured, providing a stable structural basis for subsequent inter-frame interpolation and image completion. Combining the basic decoding sequence data and the preloading priority sequence data to achieve key frame priority scheduling, and at the same time implementing intelligent cache control through the terminal buffer control instruction, effectively reducing the initial playback delay. Through the frequency domain energy analysis of the incremental entropy slice of the screen, high-frequency feature regions are screened, and an efficient region-level image supplementary transmission model is constructed according to the significance calculation result, improving the video detail restoration ability, especially suitable for scenes with rich motion or drastic changes. A bandwidth adaptive compression mechanism is used to perform lossy compression on the high-frequency feature regions and perform supplementary transmission scheduling to ensure that the refined completion of the screen can still be gradually achieved in a limited bandwidth environment, improving the overall visual quality. The network scheduling module generates dynamic transmission plan data and cooperates with the terminal synchronization control mechanism for scheduling execution, enabling the system to flexibly cope with network anomalies such as sudden bandwidth fluctuations and delay changes. The phased preloading and supplementary transmission design enables terminal devices with different performance levels to select appropriate decoding strategies according to their own buffer capabilities and processing capabilities, thus taking into account the fluency of low-end devices and the pursuit of picture quality of high-end devices.

[0038] Preferably, step S3 includes the following steps:

[0039] Step S31: Perform real-time sampling on the terminal decoder to obtain the decoding state vector;

[0040] Step S32: Perform multi-dimensional compression encoding on the decoding state vector to generate a lightweight feedback data packet; encapsulate the lightweight feedback data packet with the RTCP extension protocol to generate terminal feedback transmission data;

[0041] Step S33: Perform timing merging on the terminal feedback transmission data to generate a side transmission state sequence; based on the image streaming preloading strategy, use the side transmission state sequence to perform transmission matching on the grouped images of the screen to generate terminal transmission feedback data;

[0042] Step S34: Perform encoding-side dynamic transmission scheduling on the terminal transmission feedback data based on the decoding priority marking data to generate an optimized grouped image for screen transmission.

[0043] The present invention obtains a multi-dimensional decoding state vector including buffer status, decoding rate, frame loss situation, etc. by performing real-time sampling on the terminal decoder, providing accurate basic data for subsequent optimization scheduling. After performing multi-dimensional compression encoding on the decoding state vector, a lightweight feedback data packet is generated, greatly reducing the occupancy of the uplink bandwidth by terminal feedback and being applicable to low-latency transmission scenarios. By encapsulating the feedback data using the RTCP extension protocol, the transmission standardization and cross-platform adaptability of the data are enhanced, and at the same time, the expansion ability of the system under the standard protocol is improved. By performing temporal merging on the terminal feedback transmission data, a continuous end-side transmission status sequence is generated, enabling the system to perform temporal modeling and dynamic prediction on the terminal decoding trend, thereby achieving more forward-looking scheduling decisions. Using the image streaming preloading strategy to match the transmission of the end-side status sequence, the transmission strategy can be dynamically adjusted according to the actual load of the terminal, realizing precise grouped image stream scheduling. Scheduling the feedback data based on the decoding priority marking data ensures that key frames and high-priority frames can still be preferentially transmitted when the terminal is congested or the performance deteriorates, improving the decoding success rate and playback continuity. Constructing a closed-loop mechanism from terminal feedback to encoding-side scheduling enables the picture transmission scheduling to no longer rely on fixed rules, but to be driven by real-time feedback for dynamic optimization, realizing refined and intelligent data scheduling. When facing different network loads or changes in terminal decoding capabilities, the system can quickly adaptively adjust based on end-side feedback, ensuring the continuity and stability of video live broadcast in a fluctuating network environment.

[0044] Preferably, step S34 includes the following steps:

[0045] Step S341: Perform decoding accumulation analysis on the terminal transmission feedback data to obtain scheduling suppression signal data;

[0046] Step S342: Identify key frame packet loss in the terminal transmission feedback data to obtain key BES loss warning data;

[0047] Step S343: Extract the GPU utilization rate and NPU utilization rate information of the terminal transmission feedback data, and perform utilization rate interval analysis on the GPU utilization rate and NPU utilization rate information to generate resolution degradation factor data;

[0048] Step S344: Perform joint priority mapping on the decoding priority marking data, scheduling suppression signal data, key BES loss warning data, and resolution degradation factor data to generate dynamic priority transmission table data;

[0049] Step S345: Reconstruct the image packets of the picture grouped images through the dynamic priority transmission table data to generate picture transmission optimized grouped image data.

[0050] The present invention decodes and accumulates the terminal transmission feedback data, identifies abnormal states such as terminal cache overload and decoding delay in real time, generates scheduling inhibition signals in time, and effectively alleviates the problems of video freeze and delay. Through the key BES (Basic Entropy Segment) loss alarm mechanism, the transmission and compensation of key frame data are prioritized to ensure that the video structure is not destroyed in the scenario of network fluctuation or insufficient bandwidth, and the picture coherence and decodability are enhanced. The utilization rate of the terminal GPU and NPU is extracted and the interval analysis is performed to automatically determine the terminal computing power bottleneck, thereby generating a resolution degradation factor, supporting the dynamic resolution reduction strategy, and improving the system's compatibility and playback stability for weak terminals. The scheduling inhibition signal, key frame loss alarm, terminal computing power status and decoding priority data are integrated and analyzed to form a multi-dimensional dynamic priority transmission table, so that the scheduling strategy is more in line with real-time scene requirements. Through the dynamic priority transmission table data, the image packet reconstruction is performed on the picture grouping image, which can flexibly replace frames, fill in gaps, and compress low-priority fragments, realizing a streaming reorganization strategy based on feedback perception, and improving the video reconstruction quality in a weak network environment. The overall scheduling process has the ability to quickly respond to terminal anomalies, protect key data, and reconstruct and optimize transmission content in real time, making the system extremely elastic, intelligent, and resistant to disturbances. Through the dynamic priority scheduling mechanism, terminal status perception, feedback compression analysis, and encoding end image reconstruction are connected in series to achieve closed-loop scheduling control from terminal status to transmission content, effectively opening up an end-to-end optimization link.

[0051] Preferably, step S4 comprises the following steps:

[0052] Step S41: preloading the basic entropy slices in the picture transmission optimization group image into a fixed area of ​​the video memory in the computer chip to obtain a basic entropy slice transmission data stream;

[0053] Step S42: asynchronously write the picture incremental entropy slices in the picture transmission optimization group image to obtain the incremental entropy slice data stream; perform entropy stream fusion on the basic entropy slice transmission data stream and the incremental entropy slice data stream through the built-in calculation shader of the terminal decoder to generate a live video reconstructed picture stream;

[0054] Step S43: performing anomaly detection on the reconstructed picture stream of the live video, and performing fault-tolerant reconstruction on the reconstructed picture stream of the live video according to the anomaly detection result, so as to perform a fast decoding and transmission operation for the live video image.

[0055] By preloading the basic entropy slices of the video into a fixed area of the video memory in the computer chip, the present invention can greatly improve the access efficiency of image data, shorten the image decoding latency, and provide data preparation guarantee for subsequent fusion processing. The asynchronous writing method is adopted to send the incremental entropy slices of the video into the decoding path, which effectively improves the parallelism of incremental slice processing, reduces the system blocking time, and realizes the multi-channel efficient scheduling of entropy slice processing. The built-in compute shader of the terminal decoder is used to perform real-time entropy flow fusion on the basic entropy slices and incremental entropy slices, which can reconstruct clear and coherent video images under different coding levels and entropy distribution structures, and improve the playback integrity and viewing experience. The abnormal detection of the reorganized video stream can identify transmission abnormalities such as image distortion, frame order imbalance, and local entropy loss in a timely manner, and improve the response speed of the system to unpredictable problems. Based on the abnormal detection results, a fault tolerance reconstruction mechanism based on video memory pre-caching and entropy slice filling is executed, which can accurately recover the missing image segments and greatly reduce the phenomena such as playback frame drop, freezing, and black screen. The decoding process constructed by the basic and incremental entropy shunt processing, video memory-level fusion, and fault tolerance-level repair has stronger device compatibility and the ability to cope with transmission uncertainties, and is especially suitable for scenarios with weak networks, high concurrency, or mobile devices. By integrating video memory scheduling, data fusion, abnormal identification, and recovery mechanisms, a complete closed-loop of video-side autonomous processing is formed, which improves the system stability and task sustainability, and is applicable to live broadcast application scenarios with extremely high requirements for latency and stability.

[0056] Preferably, step S43 includes the following steps:

[0057] Step S431: Detect the frame structure integrity of the reorganized video stream of the live broadcast to obtain the abnormal detection result and mark it as frame-level abnormal marking data;

[0058] Step S432: Conduct reference chain tracking analysis on the frame-level abnormal marking data to generate key frame abnormal influence chain data;

[0059] Step S433: Model the error propagation path for the key frame abnormal influence chain data to generate error code diffusion prediction map data;

[0060] Step S434: Distinguish the recoverability of the error code diffusion prediction map data to generate fault tolerance recovery priority area data;

[0061] Step S435: Based on the fault tolerance recovery priority area data, perform interpolation estimation and reconstruction on the reorganized video stream of the live broadcast to generate fault tolerance reconstructed video data stream for executing the fast decoding and transmission operation for the live broadcast video image.

[0062] Through the frame structure integrity detection of the reorganized video live stream, the present invention can quickly detect abnormal problems such as structural breaks, frame header damage, and missing reference frames. The generated frame-level abnormal marker data can be used as an important basis for subsequent fault tolerance processing, realizing abnormal capture from coarse to fine. By using the reference chain tracking and analysis mechanism, the influence chain of missing key frames on subsequent predicted frames can be clearly analyzed, generating key frame abnormal influence chain data, avoiding "blind retransmission" and unnecessary compensation operations, and improving the fault tolerance efficiency and scheduling accuracy. Through the path modeling of the key frame abnormal influence chain, an error diffusion prediction map is formed, which can predict the propagation trend of abnormalities in different time axes and image regions, thereby realizing the pre-intervention and active control of potential image quality degradation paths. The distinguishability of recoverability of the error diffusion prediction map helps to demarcate the "high recoverable area" and the "low repairable area", thereby generating fault tolerance recovery priority area data, improving the system resource utilization efficiency, and preferentially processing the picture areas that have a greater impact on visual perception. Based on the fault tolerance recovery priority area, an intelligent interpolation estimation strategy is executed, which can achieve the recovery of visual continuity without relying on the original data packet, generating a fault tolerance reconstructed video data stream, effectively solving problems such as picture jumps and black screens caused by missing key frames and missing reference frames. Through the complete chain processing from frame structure detection, abnormal tracking, propagation modeling to interpolation reconstruction, the self-healing ability of the video live stream is significantly enhanced, ensuring the stability and consistency of the high-frequency and low-latency playback experience in the live broadcast scenario. The entire abnormal analysis and interpolation reconstruction process can be completed locally on the terminal side, reducing the dependence on the server for retransmission, effectively reducing the backhaul bandwidth overhead, and providing a highly adaptable live broadcast solution for weak network or high-load environments.

[0063] In this specification, a fast decoding and transmission system for video live images is provided, which is used to execute the above-mentioned fast decoding and transmission method for video live images. The fast decoding and transmission system for video live images includes:

[0064] A picture image grouping module, configured to obtain a video live picture stream; divide the video live picture stream into image groups to generate grouped picture images; analyze the inter-frame relationship of the grouped picture images, and define the priority of the grouped picture images according to the inter-frame relationship to generate decoding priority marker data;

[0065] An image slicing module, configured to perform entropy data slicing on the grouped picture images to generate basic picture entropy slices and incremental picture entropy slices; perform staged picture transmission on the grouped picture images based on the basic picture entropy slices and the incremental picture entropy slices to generate an image streaming preloading strategy;

[0066] A transmission optimization module, which is used to perform terminal transmission feedback on the grouped images of the screen by using the image streaming preloading strategy to obtain terminal transmission feedback data; and perform dynamic transmission scheduling on the terminal transmission feedback data based on the decoding priority marking data to generate optimized grouped images for screen transmission.

[0067] A screen stream recombination module, which is used to perform entropy stream fusion on the optimized grouped images for screen transmission through a hardware decoder to generate a recombined screen stream for video live broadcast; and perform fault-tolerant reconstruction on the recombined screen stream for video live broadcast to execute a fast decoding and transmission operation for video live broadcast images.

[0068] The beneficial effects of the present invention are as follows: By dividing the video live stream picture stream into groups of pictures, it can be accurately divided into multiple small groups, reducing the processing duration and improving the data transmission efficiency. This can ensure the stable transmission of data under different network conditions and reduce the risk of packet loss. The analysis of the inter-frame relationship helps to better understand and utilize the temporal structure of the picture stream, thereby avoiding unnecessary redundant transmission in the subsequent decoding process and optimizing the decoding order and strategy of the pictures. Through the generation of decoding priority marking data, the system can dynamically adjust the processing priorities of different picture group images, ensure the priority transmission of important frame data, and optimize the overall viewing experience of the picture stream. Based on the generation of picture base entropy slices and incremental entropy slices, effective compression and slicing of image data can be carried out, reducing the bandwidth consumption and also providing a convenient data access format for subsequent image recombination. It effectively reduces the data burden of the video stream and enhances the flexibility of transmission. Through the image streaming preloading strategy, data can be loaded and optimized for transmission in advance, reducing the delay caused by network instability and ensuring the continuity of the picture stream. Especially in an environment with limited bandwidth, dynamic adaptive adjustment can be made to ensure smooth video viewing. Bandwidth adaptive compression of the picture incremental entropy slices enables the picture stream to maintain a high level of smoothness under different network bandwidth conditions, thereby improving the user experience of video live streaming, especially in a mobile or unstable network environment. By using the image streaming preloading strategy to feedback the terminal transmission situation, the network and transmission status data can be obtained in real time, thereby enabling more accurate transmission scheduling, reducing data loss and delay, and optimizing the transmission path of video data. Through dynamic scheduling based on decoding priority marking data, the system can react in real time to the network and terminal states, adjust the video stream transmission strategy, ensure the timely arrival of key frames, reduce the stuttering phenomenon caused by packet loss or delay, and the generated optimized transmission group images of the pictures can intelligently adjust the transmission order and priority of the picture groups to adapt to different network states, improving the quality and stability of the live stream, which is particularly important in an unstable network environment. Through the entropy flow fusion of the optimized transmission group images of the pictures by the hardware decoder, the decoding efficiency of the video stream can be greatly improved, reducing the processing delay, which is particularly suitable for the low-latency video live streaming requirements. The fault-tolerant reconstruction function can perform intelligent recovery in the case of network packet loss or decoding errors, ensuring the continuity and quality of the picture stream. This can effectively cope with sudden network failures and avoid affecting the viewing experience due to stuttering or frame loss during the live broadcast. Through the fault-tolerant reconstruction technology, fast decoding and transmission are ensured, effectively reducing the decoding time and improving the real-time performance of the image stream, which is applicable to scenarios with high timeliness requirements such as real-time live broadcasts and video conferences. Therefore, the present invention improves the fast decoding and transmission performance of video live stream images by optimizing video picture grouping, priority scheduling, bandwidth management, decoding acceleration, and fault-tolerant reconstruction. Description of the Drawings

[0069] Figure 1 It is a schematic diagram of the step process of a fast decoding and transmission method for video live broadcast images;

[0070] Figure 2 It is Figure 1 a detailed implementation step process diagram of step S3 in

[0071] Figure 3 It is Figure 1 a detailed implementation step process diagram of step S4 in

[0072] The realization, functional characteristics and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings. Specific Embodiments

[0073] The technical method of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0074] In addition, the accompanying drawings are only schematic diagrams of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor methods and / or microcontroller methods.

[0075] It should be understood that although the terms "first", "second", etc. may be used here to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, the first unit may be called the second unit, and similarly the second unit may be called the first unit. The term "and / or" used here includes any and all combinations of one or more of the listed related items.

[0076] To achieve the above object, please refer to Figures 1 to 3 , a fast decoding and transmission method for video live broadcast images, the method includes the following steps:

[0077] Step S1: Obtain the video live broadcast picture stream; divide the video live broadcast picture stream into groups of images to generate grouped picture images; analyze the inter-frame relationship of the grouped picture images, and define the priority of the grouped picture images according to the inter-frame relationship to generate decoding priority marking data;

[0078] Step S2: Perform entropy data sharding on the grouped picture images to generate basic picture entropy slices and incremental picture entropy slices; perform staged picture transmission on the grouped picture images based on the basic picture entropy slices and the incremental picture entropy slices to generate an image streaming preloading strategy;

[0079] Step S3: Use the image streaming preloading strategy to perform terminal transmission feedback on the grouped picture images to obtain terminal transmission feedback data; perform dynamic transmission scheduling on the terminal transmission feedback data based on the decoding priority marking data to generate optimized grouped picture images for picture transmission;

[0080] Step S4: Perform entropy flow fusion on the optimized grouped picture images for picture transmission through a hardware decoder to generate a video live broadcast reconstructed picture stream; perform fault-tolerant reconstruction on the video live broadcast reconstructed picture stream to execute a fast decoding and transmission job for video live broadcast images.

[0081] By analyzing the inter-frame relationship and generating decoding priority marking data in Step S1, the present invention can achieve dynamic scheduling of key frames and predicted frames, making the decoding order more targeted, and improving the overall decoding efficiency and response speed. In Step S2, by dividing the basic picture entropy slices and incremental entropy slices, and adopting a staged picture transmission mechanism, the data volume is effectively compressed, reducing the bandwidth resource occupation while ensuring the fast transmission of key content. Using the terminal transmission feedback and dynamic transmission scheduling mechanism in Step S3, real-time regulation is performed according to the actual transmission situation to avoid the impact of network fluctuations on image quality and coherence, and improve the system robustness. In Step S4, a hardware decoder is used to perform entropy flow fusion on the optimized grouped images, and combined with the fault-tolerant reconstruction mechanism, a high-quality video live broadcast stream can be quickly reconstructed, ensuring smooth and clear live broadcast images and reducing latency. The introduction of the picture streaming preloading strategy enables the terminal to perform pre-rendering processing before completely obtaining the image content, reducing the first-frame loading time and improving the user viewing experience. This method is applicable to various types of network environments and terminal devices, has good scalability and compatibility, and is convenient for deployment and application in multiple scenarios such as video conferencing, live broadcast platforms, and telemedicine. Therefore, the present invention improves the fast decoding and transmission performance of video live broadcast images by optimizing video picture grouping, priority scheduling, bandwidth management, decoding acceleration, and fault-tolerant reconstruction.

[0082] In the embodiment of the present invention, refer to Figure 1 As shown, it is a schematic flowchart of the steps of a method for fast decoding and transmission of video live broadcast images according to the present invention. In this example, the method for fast decoding and transmission of video live broadcast images includes the following steps:

[0083] Step S1: Obtain the video live stream; divide the video live stream into groups of pictures to generate grouped picture images; analyze the inter-frame relationship of the grouped picture images, and define the priority of the grouped picture images according to the inter-frame relationship to generate decoding priority marking data;

[0084] In the embodiment of the present invention, the video live stream is obtained through a video acquisition module. The live stream can come from real-time video acquisition terminals such as cameras, drones, and smart devices. The obtained video live stream is a continuous sequence of video frames, including key frames (I frames), predicted frames (P frames), and bidirectional predicted frames (B frames), etc. The collected video live stream is divided according to the GOP (Group of Pictures) structure. Each group of pictures starts with an I frame and contains multiple P frames and B frames to form grouped picture images. The division method can be based on video coding standards (such as H.264 or H.265), or can be custom-set according to time periods (such as every 2 seconds as a group of pictures) to output a grouped picture image data set. For each grouped picture image, a frame dependency relationship analysis algorithm is used to construct an inter-frame reference relationship graph (Reference Graph). This graph clarifies the dependency degree of each frame on other frames by analyzing the prediction reference information during the encoding process. For example: I frames have no dependencies and are decoded independently; P frames depend on the previous I frame or P frame; B frames depend on the previous and next frames, and the inter-frame dependency relationship graph data is output. Based on the above inter-frame dependency relationship graph, the decoding priorities of the frames within each group of pictures are sorted. The definition of the priority can be based on the following principles: High priority: frames with strong decoding independence and low dependency (such as I frames); Medium priority: P frames that depend on the previous frame; Low priority: B frames that depend on bidirectional frames. Using a topological sorting or depth-first search algorithm, the decoding priorities of the frames in each group of pictures are output, and decoding priority marking data is formed to output a decoding priority marking data set.

[0085] Step S2: Perform entropy data sharding on the grouped picture images to generate basic picture entropy slices and incremental picture entropy slices; perform stage picture transmission on the grouped picture images based on the basic picture entropy slices and incremental picture entropy slices to generate an image streaming preloading strategy;

[0086] In the embodiments of the present invention, for each group of picture grouping images, entropy coding methods (such as CABAC: Context-Adaptive Binary Arithmetic Coding or CAVLC: Context-Adaptive Variable Length Coding) are used to analyze and fragment their compressed data. The fragmentation is divided into two categories: the Base Entropy Segment of the picture contains the key frame (I-frame) and its necessary context models and prediction information, has the ability to be decoded independently, and is preferentially transmitted; the Incremental Entropy Segment of the picture contains the compressed information of the P-frame and B-frame that depend on other frames, and is only transmitted and decoded when the decoding dependency conditions are met. During the fragmentation process, hierarchical coding is performed on the data marked according to the decoding priority to ensure that the Base Entropy Segment is preferentially complete and the Incremental Entropy Segment is supplemented later, and the Base Entropy Segment dataset of the picture + the Incremental Entropy Segment dataset of the picture are output. Based on the above two types of entropy segments, a staged picture transmission model is constructed, including the following strategies: Stage 1 (Initialization Stage): Preferentially transmit the Base Entropy Segment to ensure that the basic picture is available as soon as possible, and achieve fast picture loading and first-frame decoding; Stage 2 (Enhancement Stage): Gradually transmit the Incremental Entropy Segment on demand, and schedule according to network bandwidth, user view window or analysis requirements; Stage 3 (Backfill Optimization Stage): Complete the integrity backhaul of the remaining Incremental Entropy Segment in the background for picture quality enhancement, subsequent playback or caching, and output the image streaming preloading strategy data, describing the transmission logic and data scheduling priority of each stage.

[0087] Step S3: Use the image streaming preloading strategy to perform terminal transmission feedback on the picture grouping images to obtain terminal transmission feedback data; perform dynamic transmission scheduling on the terminal transmission feedback data based on the data marked according to the decoding priority to generate optimized grouped pictures for picture transmission;

[0088] In an embodiment of the present invention, when the terminal receives the entropy slice data of the grouped images of the screen, the following feedback metrics are monitored in real time: Success Rate: Records the reception status of each basic entropy slice and incremental entropy slice; Decode Buffer Level: Records the occupancy of the current decoding buffer of the terminal; Network Fluctuation Metrics: Includes packet loss rate, delay jitter, and real-time bandwidth estimation; DecodedFlags: Identifies the current decoding progress of each grouped image of the screen. The system constructs Terminal Transmission Feedback Data based on the above metrics and sends it to the server or edge node. According to the terminal transmission feedback data, the system introduces a scheduling module, which combines the decoding priority marking data to dynamically schedule the remaining untransmitted or retransmitted entropy slices of the screen, including the following processing: Prioritize the scheduling of entropy slices that have not been transmitted and have a higher priority (such as the basic prediction segments required for key P-frames); Delay or compress the transmission of low-priority incremental entropy slices to avoid network congestion; Perform differential retransmission for failed segments (lost entropy slices), and only select the parts that have a greater impact on the recovery of the current frame; If the terminal buffer is overloaded, trigger a data reduction strategy to suspend the transmission of incremental entropy slices of some low-priority frames. This scheduling strategy runs in an event-driven or fixed-period manner to form an intelligent transmission scheduling path adapted to the terminal state. Under the guidance of the above dynamic scheduling results, the system reorganizes the data of the grouped images of the screen to be sent: Eliminate redundant entropy slices (already completely received by the terminal); Prioritize the collation of the set of entropy slices that can reach the terminal smoothly under the current network state; Package the optimized data into a new Optimized Grouped Image for Transfer. This optimized grouped image has a higher transmission success rate and decodability, and can achieve a more continuous, clear, and low-latency video display effect, and outputs the data of the optimized grouped image for transfer.

[0089] Step S4: Perform entropy flow fusion on the optimized grouped image for transfer through a hardware decoder to generate a reorganized video live stream; perform fault-tolerant reconstruction on the reorganized video live stream to execute a fast decoding and transmission operation for the video live image.

[0090] In the embodiments of the present invention, by invoking the hardware decoding unit of the terminal (such as the GPU embedded video codec module, the dedicated ASIC decoder, the SoC-level hardware media engine, etc.), the optimized packet image data received from the network side is loaded and the following operations are performed: First, the received base entropy slice is decoded to reconstruct the picture skeleton and the main view information; on the basis of decoding the main picture, the prediction residuals and motion vectors in the delta entropy slice are fused to complete detail interpolation and quality enhancement; the hardware-level parallel processor realizes the decoding fusion of the intra-frame entropy stream (I-frame) and the inter-frame entropy stream (P / B-frame), and maintains the high concurrency and low latency characteristics of the decoding process. The fusion result is output as: the Reconstructed Live Video FrameStream, which is a video image with correct timing, coherent structure, and continuous main vision. Considering that some entropy slices may not be delivered in time or received successfully due to network fluctuations, the system designs the following fast fault-tolerant reconstruction strategy to ensure the continuity of the live video: For the part missing the key P-frame or B-frame, frame interpolation synthesis is performed based on the motion estimation between the previous frame and the next frame; using the spatial redundancy of the received tiles in the current frame, local reconstruction such as edge extension, color inference, and texture compensation is performed on the missing tile area; in the case where accurate recovery is impossible, a local frame skipping strategy is triggered (skipping the current missing frame to ensure the subsequent frames continue to be decoded); the residual data in the receive buffer is used to pre-fill the decoding path in advance to reduce the probability of consecutive frame loss. The above mechanism combined with hardware parallel execution ensures that even in the presence of errors or incomplete data, the system can still construct a visual output with strong visibility and good continuity, and output the video live image sequence after quick repair.

[0091] Preferably, step S1 includes the following steps:

[0092] Step S11: Obtain the video live video stream;

[0093] Step S12: Extract the frame images of the video live video stream, and perform GOP division on the frame images to obtain the picture group images, where the GOP division includes key frame division, forward prediction frame division, and bidirectional prediction frame division;

[0094] Step S13: Analyze the inter-frame reference relationship of the picture group image data to obtain the inter-frame relationship of the picture group image; perform structure mapping on the picture group image data according to the inter-frame relationship of the picture group image to generate a decoding dependency graph;

[0095] Step S14: Analyze the dependency level of the decoding dependency graph to obtain the frame type priority marking data; perform label mapping on the picture group image through the frame type priority marking data to generate the decoding priority marking data.

[0096] In the embodiments of the present invention, a continuous video live stream is obtained through a network data receiving module or a local acquisition module. The format of the video stream can be an encoded compression format (such as H.264, H.265) or an intermediate format (such as MPEG-TS, RTSP stream), and it is cached in the video stream input buffer for subsequent decoding and analysis. Input sources: network cameras, streaming devices, live platform CDN edge nodes, etc.; supported protocols: RTMP, HLS, RTSP, WebRTC, etc.; processing method: the continuous stream is parsed into processable raw bitstream data through a receiver. The encoded bitstream in the video stream is pre-decoded to extract the frame images, and they are divided into GOP (Group of Pictures) group image structures according to the encoding structure. There are three types of frame images in the GOP: I frame (Intra-coded Frame): key frame, independently encoded, serving as the decoding starting point; P frame (Predicted Frame): forward predicted frame, depending on the previous frame (I frame or P frame); B frame (Bidirectional Predicted Frame): bidirectional predicted frame, depending on the previous and next frames; The GOP division method can be based on a fixed frame interval (such as the I frame interval is 30 frames) or dynamically divided according to the adaptive scene change. Each GOP group is regarded as a picture grouping image unit for independent analysis and scheduling. The frame images within each GOP are parsed to extract their inter-frame reference information, that is, the prediction source (reference frame) information of each frame. The parser is used to identify the reference frame list, frame sequence number (POC), prediction direction, etc. in the frame header (NALU); a directed graph structure (decoding dependency graph) is constructed, where each node in the graph is a frame, and the edge represents the decoding dependency path (such as: P frame depends on I frame, B frame depends on the previous and next P frames); Example dependency relationship: I0→P1→P2→B3←→P4. According to the constructed decoding dependency graph, analyze the dependency level depth and key degree of each frame image, and generate the decoding priority marking data of the frame image. The priority analysis principle is as follows: The frame with a higher priority = the decoding dependency path is shorter and is frequently referenced by other frames; I frame > P frame > B frame; The lower the reference level, the higher the priority; Add the following metadata tags to each frame: frame sequence number, I / P / B, the smaller the value, the higher the priority, reference frame list. Through the above label mapping operation, generate the decoding priority marking data (Decoding Priority Tag Set), and bind it to the picture grouping image data. Output: decoding dependency graph, decoding priority marking data (for use in S2~S4).

[0097] Preferably, in step S13, the structural mapping of the picture grouping image data according to the inter-frame relationship of the picture grouping image includes:

[0098] Identifying the frame type of the picture grouping image data according to the inter-frame relationship of the picture grouping image;

[0099] Extract the reference index of the picture group image data based on the frame type to obtain the reference frame index data;

[0100] Model the inter-frame connection relationship of the reference frame index data to generate the initial inter-frame dependency graph data;

[0101] Perform a directed graph structure mapping on the initial inter-frame dependency graph data to generate the frame dependency path graph data, where the nodes represent frames and the edges represent reference relationships;

[0102] Perform a layer-level analysis on the frame dependency path graph data to generate the decoding dependency level data;

[0103] Perform a structure encoding on the decoding dependency level data to generate the decoding dependency graph.

[0104] In the embodiments of the present invention, by identifying the type of each frame according to the reference relationship between the frames in the picture group image, including key frames (I frames), forward prediction frames (P frames), and bidirectional prediction frames (B frames). Specifically, the system extracts the frame type of each frame by parsing the coding identification field or control information of the frame. According to the reference information of each non-I frame, its reference frame index is extracted. These indexes reflect the other frames on which the frame depends during decoding, constituting the inter-frame reference relationship during the decoding process. By sorting out the reference relationships of all frames, a set of reference index data between frames can be formed. Based on these frame index data, each frame is used as a node and each pair of reference relationships is used as a directed edge to establish an initial inter-frame dependency graph. In this graph, the direction of the edge represents the decoding dependency order of the frames. For example, if frame A is referenced by frame B, a directed edge from A to B is formed in the graph. To further clarify the inter-frame decoding order, this initial inter-frame dependency graph will be transformed into a directed graph structure. During this process, all node connection relationships and paths are retained, and the frame dependency paths are extracted according to the topological structure of the graph. The formation of the dependency paths can be achieved through graph analysis algorithms such as traversal and sorting. Perform a layer-level analysis on the constructed frame dependency path graph to determine the hierarchical position of each frame in the dependency graph. Specifically, the system will calculate the shortest dependency path length required to decode each frame starting from the starting I frame, and evaluate the frequency of reference of this frame, forming a set of hierarchical index data representing the importance of the frame and the decoding priority order. The system fuses the above frame dependency path graph with the hierarchical indexes to generate a structured decoding dependency graph. This graph not only describes the inter-frame decoding dependency relationship, but also embeds the decoding priority information, which can be used as an important basis for subsequent decoding scheduling, entropy coding optimization, and streaming transmission strategy formulation.

[0105] Preferably, the entropy data sharding of the picture group image in step S2 includes:

[0106] Perform pixel statistics on the frame images of the grouped screen images to obtain frame set entropy distribution data, where the frame-level entropy distribution data includes the local information entropy values of each frame;

[0107] Perform regional difference clustering on the frame set entropy distribution data to generate screen entropy clustering data; extract the core entropy regions of the frame set entropy distribution data according to the screen entropy clustering data, and mark the regions with non-core entropy as basic entropy regions;

[0108] Perform codeword reconstruction on the basic entropy regions to generate screen basic entropy slices, where the screen basic entropy slices contain the minimum decoding information necessary for the frame structure;

[0109] Perform trend analysis on the core entropy regions in the grouped screen image data, and perform mutation metric coding on the core entropy regions based on the trend analysis results to generate screen incremental entropy slices.

[0110] In the embodiments of the present invention, through pixel-level statistical analysis of each frame of image. The system calculates the information entropy value of each local area of the frame image based on the brightness distribution, color channel distribution and texture gradient change of the frame image. After processing multiple frame images, a set of frame set entropy distribution data is formed, which describes the local information complexity of each frame image in different regions, that is, the frame-level entropy distribution characteristics. Perform regional difference clustering processing on the obtained frame set entropy distribution data. The system uses clustering algorithms (such as K-means or density-based clustering methods) to aggregate regions with similar entropy value characteristics in the entropy distribution, and identify regions with significantly high entropy characteristics in each frame image, which are defined as core entropy regions. These regions usually contain fast movements, structural edges or dynamic change scenes. The remaining regions with lower entropy values are marked as basic entropy regions, representing regions such as backgrounds and static elements in the screen that have less impact on decoding. After marking the basic entropy regions, the system performs codeword reconstruction processing on them. Specifically, the system uses entropy coding (such as Huffman coding, arithmetic coding, etc.) or differential coding methods to perform compression coding on the basic entropy regions, and extracts the minimum necessary information required to retain the most basic frame structure, thereby generating screen basic entropy slices. The basic entropy slices can be quickly transmitted and preferentially decoded to achieve the presentation of the screen structure under low bandwidth. At the same time, for the core entropy regions in the frame images, the system performs dynamic change trend analysis. By comparing the pixel change rate, texture fluctuation intensity or motion vector difference of this region between consecutive frames, the temporal mutation characteristics of the core region are evaluated. Based on this change trend, the system encodes these mutation characteristics to construct an entropy data supplementary slice reflecting the spatio-temporal change intensity, that is, the screen incremental entropy slice. By using the screen basic entropy slices and the screen incremental entropy slices as scalable transmission data units, it can provide data support for subsequent image streaming preloading strategies, and achieve differential allocation and scheduling of decoding resources for high-entropy regions and low-entropy regions.

[0111] Preferably, the stage screen transmission of the screen grouped images based on the screen basic entropy slices and the screen incremental entropy slices in step S2 includes:

[0112] Identify key frames for the screen basic entropy slices, and perform temporal recombination on the identified key frames to generate basic decoding order data;

[0113] Perform hierarchical transmission scheduling on the screen basic entropy slices according to the basic decoding order data to generate preloading priority sequence data, where key frames are transmitted first; perform prefetch buffer control on the preloading priority sequence data to generate terminal buffer control instructions;

[0114] Use the terminal buffer control instructions to perform local decoding start processing on the screen basic entropy slices to generate the preloading stage;

[0115] Perform frequency domain energy calculation on the screen incremental entropy slices to generate regional frequency intensity distribution data; perform significance screening on the regional frequency intensity distribution data to generate a high-frequency feature region set;

[0116] Perform bandwidth adaptive compression on the high-frequency feature region set to generate a supplementary transmission data packet group under bandwidth limitation; perform network scheduling on the supplementary transmission data packet group to generate dynamic transmission plan data; perform terminal synchronization control on the dynamic transmission plan data to obtain the incremental supplementary transmission stage;

[0117] Perform stage matching execution on the screen grouped images based on the preloading stage and the incremental supplementary transmission stage, so as to generate an image streaming preloading strategy.

[0118] In the embodiments of the present invention, key frames in the basic entropy slice of the video are identified. The system determines which frames are key frames (such as I frames) by analyzing the structural features, motion estimation, and timestamp information of each frame. These frames usually contain the most important image information. After the key frames are identified, the system reorganizes them according to the time sequence to ensure that the key frames are arranged in the correct time order for efficient subsequent decoding and generate the basic decoding order data. This data is used to optimize the decoding order to ensure that each key frame and other frames it depends on can be loaded as needed. According to the generated basic decoding order data, the system performs hierarchical transmission scheduling on the basic entropy slice of the video. The hierarchical structure is determined by the decoding priority, and the key frames are transmitted first to ensure the integrity and clarity of the video structure. Through this scheduling mechanism, the system generates a preloading priority sequence data, in which the key frames are always transmitted first. To further optimize the transmission process, the system adopts a prefetch buffer control mechanism to ensure that the terminal device receives the required basic entropy slice data in advance. By controlling the terminal buffer, terminal buffer control instructions are generated to ensure the smooth transmission of the image data stream. Using the generated terminal buffer control instructions, the system starts the local decoding process of the basic entropy slice of the video. This process mainly focuses on the initialization stage of the video stream to ensure that the key frames and the basic structure part can be quickly decoded and displayed on the terminal device. This stage is defined as the preloading stage, during which the user can quickly obtain the initial video content and start watching the live video. For the incremental entropy slice of the video, the system calculates the frequency domain energy. Through Fourier transform or other frequency domain analysis techniques, the system evaluates the frequency intensity of each region, identifies the regions with large frequency changes, which usually have high visual importance. The generated regional frequency intensity distribution data helps the system understand which parts of the image information are most important for the user's viewing experience. The system performs saliency screening according to the regional frequency intensity distribution data and extracts the high-frequency feature regions. These regions contain information such as details, dynamic changes, or moving objects in the image, which are crucial for the user's viewing experience. Then, the system performs bandwidth-adaptive compression on these high-frequency feature regions to adapt to the network bandwidth limitations and ensure that data transmission does not interrupt under different network conditions. Through compression, a supplementary transmission data packet group under bandwidth limitation is generated, and these packet groups will be used in the subsequent supplementary transmission stage. The generated supplementary transmission data packet group will be network-scheduled. The network scheduling algorithm arranges the transmission order of the supplementary transmission data packets reasonably according to the real-time network status, bandwidth changes, and transmission priority to ensure that the data is transmitted within the scheduled time. At this time, the system generates dynamic transmission plan data and, through the terminal synchronization control mechanism, ensures that the terminal device can receive and correctly decode the required incremental data on time to complete the execution of the incremental supplementary transmission stage. Based on the above preloading stage and incremental supplementary transmission stage, the system combines the transmission requirements and time sequence arrangements of the two and optimizes the image transmission process through a stage matching execution strategy.Finally, an image streaming preloading strategy is generated. This strategy can balance bandwidth utilization and picture quality, ensuring that in real-time video live streaming, while the picture quality gradually improves, users can watch smoothly.

[0119] As an example of the present invention, refer to Figure 2 As shown, in this example, step S3 includes:

[0120] Step S31: Perform real-time sampling on the terminal decoder to obtain a decoding state vector;

[0121] Step S32: Perform multi-dimensional compression encoding on the decoding state vector to generate a lightweight feedback data packet; encapsulate the lightweight feedback data packet with the RTCP extension protocol to generate terminal feedback transmission data;

[0122] Step S33: Perform time-series merging on the terminal feedback transmission data to generate a side transmission state sequence; use the side transmission state sequence to perform transmission matching on the picture grouped images based on the image streaming preloading strategy to generate terminal transmission feedback data;

[0123] Step S34: Perform encoding-side dynamic transmission scheduling on the terminal transmission feedback data based on the decoding priority marking data to generate optimized grouped images for picture transmission.

[0124] In the embodiments of the present invention, a real-time sampling operation is performed on the terminal decoder to periodically obtain the decoding state vector generated during its operation. The decoding state vector includes, but is not limited to, the following data fields: Decoder Frame Rate; Decoder Buffer Availability; Frame Drop Rate; current frame type (I / P / B frame type identifier); Decoder Error Code, etc. The above sampling process can be realized through the cooperation of software and hardware in the embedded decoding module, and the sampling period can be adaptively adjusted according to network fluctuations or the intensity of picture changes to ensure the real-time and effectiveness of feedback. The above decoding state vector is input into the multi-dimensional compression coding module, and an efficient compression algorithm (such as sparse vector coding, differential coding, Huffman coding, etc.) is used to reduce its dimension and compress it to generate a lightweight feedback data packet. The data structure of this data packet is compact and suitable for fast transmission in low-bandwidth scenarios. Subsequently, the lightweight feedback data packet is encapsulated based on the RTCP (Real-Time Control Protocol) extension protocol to generate terminal feedback transmission data. An identification field is introduced during the encapsulation process to be compatible with the conventional RTCP structure, and a custom field is extended to accommodate multi-dimensional decoding state information, so as to achieve lossless expression of the original decoding state and ensure protocol compatibility. The terminal feedback transmission data in multiple periods is processed by time-series merging to construct a terminal-side transmission state sequence containing feedback data in multiple time slices. This state sequence is indexed by timestamps, and a multi-field weight mapping table is established for subsequent transmission scheduling reference. Combining the terminal-side transmission state sequence, an image streaming preloading strategy is applied. According to indicators such as frame type priority, content motion intensity, and predicted transmission delay value, the grouped images of the picture are prioritized, and the transmission matching operation of the grouped images is performed. This operation outputs terminal transmission feedback data, indicating the set of image frames or image blocks that should be preferentially transmitted at the current time sequence. After receiving and parsing the terminal transmission feedback data, the data is marked according to the preset decoding priority, and a dynamic transmission scheduling queue is established at the encoding end. This scheduling process dynamically adjusts the image encoding and pushing order through the following strategies: high-priority pictures use higher bitrate and low-latency encoding parameters; image frames with high latency and high frame-drop risk are pushed in advance; non-key frames or low-priority frames can be postponed or discarded to save bandwidth. Finally, an optimized and sorted picture transmission optimized grouped image with parameter configuration is generated in the scheduling module. This grouped image is organized by image frame granularity or image block granularity and is pushed from the encoding end to the terminal for decoding and rendering.

[0125] Preferably, step S34 includes the following steps:

[0126] Step S341: Perform decoding stack analysis on the terminal transmission feedback data to obtain scheduling suppression signal data;

[0127] Step S342: Identify key frame packet losses in the terminal transmission feedback data to obtain critical BES loss alarm data;

[0128] Step S343: Extract the GPU utilization rate and NPU utilization rate information of the terminal transmission feedback data, and perform utilization rate interval analysis on the GPU utilization rate and NPU utilization rate information to generate resolution degradation factor data;

[0129] Step S344: Perform joint priority mapping on the decoding priority marking data, scheduling suppression signal data, critical BES loss alarm data, and resolution degradation factor data to generate dynamic priority transmission table data;

[0130] Step S345: Reconstruct the image packets of the grouped screen images through the dynamic priority transmission table data to generate optimized grouped screen transmission image data.

[0131] In the embodiments of the present invention, by comprehensively analyzing information such as frame processing delay, decoding buffer accumulation trend, and decoding frame loss in the terminal transmission feedback data, a stack trend function model is constructed to identify whether the decoder is overloaded or blocked. Based on the above analysis, scheduling suppression signal data is generated to indicate whether non-critical frames, low-priority frames, or high-resolution frames should be suppressed during the current stage. For example, when the occupancy of the decoding buffer exceeds a set threshold (such as 80%) in two consecutive cycles, a suppression signal is triggered to limit the transmission of low-priority frames. In the terminal feedback data, packet loss detection is performed on the key frames (I frames, Scene-Start Frames) in the video stream, and mechanisms such as packet sequence number comparison and frame loss reconstruction failure marking are used to identify key frame transmission failure events. The detection results are used to generate key BES (Base Essential Segment) loss alarm data, accompanied by information such as the position of the key frame loss, the occurrence time, and the corresponding scene change information, providing risk prediction for subsequent scheduling. The GPU utilization rate and NPU utilization rate data reported in real time in the terminal transmission feedback data are extracted to construct a dynamic resource utilization range, such as: GPU low: <40%, GPU medium: 40% - 80%, GPU high: >80%, and the NPU range is the same. Based on the above range, the system resource pressure is evaluated, and the current acceptable image complexity range is deduced. Combining the video resolution and the algorithm model load, resolution degradation factor data is output, which will be used to guide the active degradation processing of high-complexity image content at the encoding end (such as reducing texture details and selecting low-complexity encoding modes). The following four types of scheduling-related data are fused and calculated: decoding priority marking data (derived from the encoding strategy), scheduling suppression signal data (from the stack analysis), key BES loss alarm data (from the packet loss detection), and resolution degradation factor data (from the resource utilization analysis). Based on the preset priority weighting mapping rules, the dynamic priority score of each image frame is calculated, and the dynamic priority transmission table data is constructed. The transmission table is indexed by frame number or image block number and includes fields such as transmission priority, target resolution, and necessary transmission marking. According to the dynamic priority transmission table data, the grouped video images are reconstructed, including the following operations: rearranging the image transmission order to give priority to transmitting high-priority frames; replacing some high-resolution image blocks with low-resolution versions according to the resolution degradation factor; performing merge / delete operations on redundant image packets to streamline the transmission packet structure; compensating for the missing areas of key frames with redundant encoded packets. Finally, the optimized grouped video image data after scheduling optimization and structure reconstruction is generated, which is used for dynamic pushing at the encoding end to improve the overall transmission fluency and terminal decoding stability.

[0132] As an example of the present invention, refer to Figure 3 shown, in this example, step S4 includes:

[0133] Step S41: Pre-load the picture base entropy slice in the picture transmission optimized grouped image into a fixed area of the video memory in the computer chip to obtain a base entropy slice transmission data stream;

[0134] Step S42: Asynchronously write the picture delta entropy slice in the picture transmission optimized grouped image to obtain a delta entropy slice data stream; perform entropy stream fusion on the base entropy slice transmission data stream and the delta entropy slice data stream through the built-in compute shader of the terminal decoder to generate a video live reorganized picture stream;

[0135] Step S43: Perform anomaly detection on the video live reorganized picture stream, and perform fault-tolerant reconstruction on the video live reorganized picture stream according to the anomaly detection result to execute a fast decoding transmission job for the video live image.

[0136] In the embodiments of the present invention, by performing structural analysis on the optimized grouped image data of the screen generated in step S34, the base entropy segment (BES) of the screen is extracted. This part of the image data generally includes: basic texture information, static scene background, global brightness, and color distribution reference frame. Through the graphics card driver call method, the DMA engine in the chip is controlled to write the base entropy segment of the screen into a predefined fixed area of the GPU memory in the computer chip in a fixed address mapping mode, constructing a base entropy segment transmission data stream to ensure that the basic scene content is not repeatedly transmitted during the live broadcast, thereby reducing the network load and decoding pressure. The remaining non-basic part of the optimized grouped image of the screen transmission is unpacked structurally, and the incremental entropy segment of the screen is extracted, including: local dynamic objects (people, actions), difference regions between key frames, temporary high-frequency changing textures, etc. The asynchronous writing technology is used to write this part of the data stream into a temporary area in the GPU memory to relieve the resource pressure of the main processing thread. Then, the built-in compute shader unit in the terminal decoder is called, and based on the entropy value matching and timestamp synchronization mechanism, the base entropy segment transmission data stream and the incremental entropy segment data stream are fused. The entropy stream fusion process includes: decompressing and decoding the two types of entropy segments, performing image block-level mapping and composition, and constructing a video frame buffer structure with consistent timing, finally generating a video live recombination screen stream. This screen stream has the integrity of the basic picture quality and the ability to restore dynamic details, and the coding redundancy is minimized. For the generated video live recombination screen stream, the anomaly detection module is accessed in real time, and comprehensive analysis is performed based on the following mechanisms: decoding failure flag detection (such as image misalignment, incomplete frames), entropy block missing detection (comparing the entropy block number with the expected mapping), timestamp drift detection (inconsistent cross-frame timing). If an anomaly is found, a fault tolerance reconstruction mechanism is triggered, including: using the static area of the base entropy segment to perform missing block filling, calling the front and back frame difference algorithm to estimate and reconstruct the moving area, and attempting to quickly repair the missing entropy block through redundant information or requesting a quick retransmission. Finally, a complete video frame image for the decoding end is output, supporting high-frame-rate and low-latency video live transmission operations.

[0137] Preferably, step S43 includes the following steps:

[0138] Step S431: Perform frame structure integrity detection on the video live recombination screen stream to obtain an anomaly detection result and mark it as frame-level anomaly marking data;

[0139] Step S432: Perform reference chain tracking analysis on the frame-level anomaly marking data to generate key frame anomaly influence chain data;

[0140] Step S433: Model the error propagation path for the key frame anomaly impact chain data to generate error code diffusion prediction map data;

[0141] Step S434: Distinguish the recoverability of the error code diffusion prediction map data to generate fault-tolerant recovery priority region data;

[0142] Step S435: Based on the fault-tolerant recovery priority region data, perform interpolation estimation and reconstruction on the video live reorganized picture stream to generate a fault-tolerant reconstructed video data stream for executing a fast decoding and transmission operation for video live images.

[0143] In the embodiments of the present invention, by performing a structural integrity analysis on each frame image in the reorganized video live stream, the following inspection items are included: the consistency of the GOP (Group of Pictures) frame sequence structure; whether there is any omission of the coding reference frame (Reference Frame); whether the key frame (I frame) and the dependent frames (P / B frames) are structurally closed; whether there are obvious entropy block losses or pixel disruptions in the frame content area. Based on the above detection items, corresponding anomaly detection results are generated, and the abnormal frames are marked and output as frame-level anomaly marking data for subsequent analysis and tracking. For the frame-level anomaly marking data, a reference chain between frames is established to trace the upstream key frames and their derived paths on which each abnormal frame depends, especially paying attention to: whether the lost frames depend on the key frame (I frame); whether the abnormal frames form a continuous frame segment; whether the reference path causing the anomaly is recoverable. Through backtracking analysis, key frame anomaly impact chain data is formed, that is, an indirect error propagation path chain caused by abnormal key frames. Using the key frame anomaly impact chain data, an inter-frame error diffusion model is constructed. This model is based on a time series graph and a graph neural network inference mechanism to predict and model the error propagation process, considering the following factors: the inter-frame dependence strength; the coding redundancy; the error propagation probability matrix; the local motion vector information. Finally, error diffusion prediction map data is generated. This map is a frame structure model used to guide the determination of the target area of the subsequent recovery strategy. A partitionable recoverability analysis is performed on the error diffusion prediction map data to judge the reconstruction cost and recovery probability of each propagation area, and based on: the number of available reference frames; the image motion smoothness; the local area continuity; the real-time situation of GPU / NPU computing resources; the image block areas with higher recovery potential are divided and output as fault-tolerant recovery priority area data. Based on the fault-tolerant recovery priority area data, combined with the front and back reference frames and the time-domain context information, the following frame interpolation techniques are used for reconstruction: dynamic area motion estimation based on the optical flow method; frame-internal texture interpolation using reference frame interpolation; introducing a multi-scale time-domain compensation algorithm to enhance the frame interpolation accuracy; using a deep learning frame interpolation model for fine repair of important areas to generate a fault-tolerant reconstructed video data stream. This data stream has high visual continuity and structural integrity and can meet the requirements of fast decoding and transmission of video live images, realizing near-real-time seamless error frame compensation.

[0144] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, it is intended to cover all changes falling within the meaning and scope of the equivalent elements of the application documents within the present invention.

[0145] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features invented herein.

Claims

1. A method for fast decoding and transmission of live video images, characterized in that: A terminal for a display screen, wherein the terminal includes a computer chip and a terminal decoder, and the terminal decoder has a built-in calculation shader, including the following steps: Step S1: obtaining a live video picture stream; dividing the live video picture stream into picture groups to generate picture group images; analyzing the inter-frame relationship of the picture group images, defining the priority of the picture group images according to the inter-frame relationship, and generating decoding priority marking data; Step S2: Slice the picture group image for entropy data to generate picture basic entropy slices and picture incremental entropy slices; perform staged picture transmission on the picture group image based on the picture basic entropy slices and the picture incremental entropy slices to generate an image streaming preloading strategy; Step S3: using the image streaming preloading strategy to perform terminal transmission feedback on the picture group image to obtain terminal transmission feedback data; dynamically performing transmission scheduling on the terminal transmission feedback data based on the decoding priority marking data to generate a picture transmission optimized group image; Step S4: perform entropy stream fusion on the picture transmission optimization grouped images through a hardware decoder to generate a live video reconstructed picture stream; perform fault-tolerant reconstruction on the live video reconstructed picture stream to perform fast decoding and transmission operations for live video images.

2. The method for fast decoding and transmission of live video images according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: Obtain live video stream; Step S12: extracting frame images of the live video picture stream, and performing GOP division on the frame images to obtain picture group images, wherein the GOP division includes key frame division, forward prediction frame division and bidirectional prediction frame division; Step S13: performing inter-frame reference relationship analysis on the picture group image data to obtain the inter-frame relationship of the picture group image; performing structure mapping on the picture group image data according to the inter-frame relationship of the picture group image to generate a decoding dependency graph; Step S14: Analyze the dependency level of the decoding dependency graph to obtain frame type priority tag data; perform label mapping on the picture grouping images through the frame type priority tag data to generate decoding priority tag data.

3. The method for fast decoding and transmission of live video images according to claim 2, characterized in that: The step S13 of performing structural mapping on the picture group image data according to the inter-frame relationship of the picture group image includes: identifying a frame type of the picture group image data according to an inter-frame relationship of the picture group image; Extracting reference indexes of the picture grouping image data based on the frame type to obtain reference frame index data; Modeling the inter-frame connection relationship of the reference frame index data to generate initial inter-frame dependency graph data; Perform directed graph structure mapping on the initial inter-frame dependency graph data to generate frame dependency path graph data, where nodes represent frames and edges represent reference relationships; Performing layer-level analysis on the frame dependency path graph data to generate decoding dependency layer data; The decoding dependency hierarchy data is structurally encoded to generate a decoding dependency graph.

4. The method for fast decoding and transmission of live video images according to claim 1, characterized in that: The step S2 of performing entropy data slicing on the picture group images includes: Performing frame image pixel statistics on the grouped images of the screen to obtain frame set entropy distribution data, wherein the frame level entropy distribution data includes a local information entropy value of each frame; Performing regional difference clustering on the frame set entropy distribution data to generate picture entropy clustering data; extracting the core entropy region of the frame set entropy distribution data according to the picture entropy clustering data, and marking the non-core entropy region as the basic entropy region; Reconstructing the codewords of the basic entropy region to generate a basic entropy slice of the picture, wherein the basic entropy slice of the picture contains the minimum decoding information necessary for the frame structure; The core entropy region in the picture grouping image data is analyzed for its change trend, and the core entropy region is encoded for variation metrics based on the trend analysis results to generate picture incremental entropy slices.

5. The method for fast decoding and transmission of live video images according to claim 1, characterized in that: The step S2 of performing staged picture transmission on the picture grouped images based on the picture basic entropy slice and the picture incremental entropy slice comprises: Identify key frames of basic entropy slices of the picture, and reorganize the identified key frames in time sequence to generate basic decoding sequence data; Perform hierarchical transmission scheduling on the basic entropy slices of the picture according to the basic decoding order data, generate preload priority sequence data, in which key frames are transmitted first; perform pre-fetch buffer control on the preload priority sequence data, and generate terminal buffer control instructions; Using the terminal buffer control instruction, local decoding and starting processing is performed on the basic entropy slice of the picture to generate a preloading stage; Perform frequency domain energy calculation on the incremental entropy slice of the picture to generate regional frequency intensity distribution data; perform significance screening on the regional frequency intensity distribution data to generate a set of high-frequency feature regions; Adaptively compress the high-frequency feature area set to generate a supplementary transmission data packet group under bandwidth restriction; perform network scheduling on the supplementary transmission data packet group to generate dynamic transmission plan data; perform terminal synchronization control on the dynamic transmission plan data to obtain an incremental supplementary transmission stage; Based on the preloading stage and the incremental supplementary transmission stage, stage matching is performed on the screen grouping images to generate an image streaming preloading strategy.

6. The method for fast decoding and transmission of live video images according to claim 1, characterized in that: Step S3 includes the following steps: Step S31: sampling the terminal decoder in real time to obtain a decoding state vector; Step S32: performing multi-dimensional compression encoding on the decoding state vector to generate a lightweight feedback data packet; performing RTCP extended protocol encapsulation on the lightweight feedback data packet to generate terminal feedback transmission data; Step S33: performing time-sequence merging on the transmission data fed back by the terminal to generate a terminal-side transmission state sequence; performing transmission matching on the picture group images using the terminal-side transmission state sequence based on the image streaming preloading strategy to generate terminal transmission feedback data; Step S34: Performing dynamic transmission scheduling at the encoding end on the terminal transmission feedback data based on the decoding priority marking data to generate a picture transmission optimized grouped image.

7. The method for fast decoding and transmission of live video images according to claim 6, characterized in that: Step S34 includes the following steps: Step S341: Decode and stack the terminal transmission feedback data to obtain scheduling suppression signal data; Step S342: performing key frame packet loss identification on the terminal transmission feedback data to obtain key BES loss alarm data; Step S343: extracting GPU utilization and NPU utilization information of the terminal transmission feedback data, and performing utilization interval analysis on the GPU utilization and NPU utilization information to generate resolution degradation factor data; Step S344: performing joint priority mapping on the decoding priority mark data, the scheduling inhibition signal data, the key BES loss alarm data, and the resolution degradation factor data to generate dynamic priority transmission table data; Step S345: reconstructing the picture group image by using the dynamic priority transmission table data to generate picture transmission optimized group image data.

8. The method for fast decoding and transmission of live video images according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: preloading the basic entropy slices in the picture transmission optimization group image into a fixed area of ​​the video memory in the computer chip to obtain a basic entropy slice transmission data stream; Step S42: asynchronously write the picture incremental entropy slices in the picture transmission optimization group image to obtain the incremental entropy slice data stream; perform entropy stream fusion on the basic entropy slice transmission data stream and the incremental entropy slice data stream through the built-in calculation shader of the terminal decoder to generate a live video reconstructed picture stream; Step S43: performing anomaly detection on the reconstructed picture stream of the live video, and performing fault-tolerant reconstruction on the reconstructed picture stream of the live video according to the anomaly detection result, so as to perform a fast decoding and transmission operation for the live video image.

9. The method for fast decoding and transmission of live video images according to claim 8, characterized in that: Step S43 includes the following steps: Step S431: performing a frame structure integrity check on the reconstructed picture stream of the live video broadcast to obtain an abnormality detection result and mark it as frame-level abnormality mark data; Step S432: performing reference chain tracking analysis on the frame-level abnormality mark data to generate key frame abnormality impact chain data; Step S433: Modeling the error propagation path of the key frame abnormal impact chain data to generate error diffusion prediction graph data; Step S434: distinguishing the recoverability of the error spread prediction map data and generating fault-tolerant recovery priority area data; Step S435: Perform frame estimation and reconstruction on the live video reconstructed picture stream based on the fault-tolerant recovery priority area data to generate a fault-tolerant reconstructed video data stream to perform fast decoding and transmission operations for live video images.

10. A fast decoding and transmission system for live video images, characterized in that: The method for fast decoding and transmitting live video images according to claim 1 is used to execute the method for fast decoding and transmitting live video images, and the system for fast decoding and transmitting live video images comprises: The picture image grouping module is used to obtain the live video picture stream; divide the live video picture stream into image groups to generate picture group images; analyze the inter-frame relationship of the picture group images, define the priority of the picture group images according to the inter-frame relationship, and generate decoding priority mark data; An image slicing module is used to slice the entropy data of the picture group images to generate picture basic entropy slices and picture incremental entropy slices; based on the picture basic entropy slices and the picture incremental entropy slices, the picture group images are subjected to stage picture transmission to generate an image streaming preloading strategy; The transmission optimization module is used to perform terminal transmission feedback on the picture group image by using the image streaming preloading strategy to obtain terminal transmission feedback data; dynamically perform transmission scheduling on the terminal transmission feedback data based on the decoding priority marking data to generate the picture transmission optimized group image; The picture stream reassembly module is used to perform entropy stream fusion on the picture transmission optimization grouped images through the hardware decoder to generate a live video reassembled picture stream; and to perform fault-tolerant reconstruction on the live video reassembled picture stream to perform fast decoding and transmission operations for live video images.

Images