Adaptive video low-latency hard-decoding display method and system

Abstract

The application relates to the technical field of video decoding, in particular to an adaptive video low-delay hard decoding display system, which identifies a video stream format and a switching state through an adaptive demultiplexing module, extracts bare stream data and judges a decoding mode; a dynamic decoder allocation module dynamically allocates decoding resources according to mode information, supports whole-frame single-decoder or N-slice multi-decoder concurrent decoding; a frame synchronization and VCXO dynamic adjustment module outputs the latest complete frame in the whole-frame mode, aligns and recombines slice data according to frame numbers in the N-slice mode, and dynamically adjusts a VCXO clock frequency based on a decoding and display time difference. The application supports multi-format compatibility, seamless video stream switching and ultra-low-delay display, and effectively avoids black fields, static frames and picture tearing.

Description

Technical Field

[0001] This invention relates to the field of video decoding technology, and in particular to an adaptive low-latency hardware decoding and display method and system for video. Background Technology

[0002] Video decoding is the process of restoring compressed and encoded video data to its original image. It is widely used in real-time interaction, streaming media playback, digital surveillance, and smart cities. As application scenarios increasingly demand real-time performance, low-latency video decoding technology has become one of the core performance indicators of a system.

[0003] In existing technologies, video decoding typically employs software decoding or fixed-mode hardware decoding schemes. For high-resolution, high-frame-rate video streams, software decoding struggles to meet real-time requirements, hence hardware decoding (hardware decoding) is widely adopted. However, traditional hardware decoding systems are mostly designed for single encoding formats and fixed transmission structures, lacking universal support for multiple video stream formats (such as MP4, MKV, TS, H.264 / H.265 raw streams), and are particularly ill-suited to the special video stream structure of N equal parts independently encoded for ultra-low latency.

[0004] Furthermore, in scenarios involving switching between multiple video streams, existing systems typically employ a "stop old, start new" decoding strategy, which involves first stopping the decoding of the old video stream and then initializing the decoder for the new video stream. This approach can easily lead to video interruptions and instability, resulting in black screens, still frames, or screen tearing, thus impacting the user experience.

[0005] Meanwhile, in traditional decoding systems, the decoding and display clocks are out of sync, resulting in significant end-to-end latency. While some systems use a fixed-frequency display clock, they cannot dynamically adjust it according to the decoding progress, making it difficult to further reduce latency while maintaining smooth visuals. Even ultra-low latency architectures employing segmented concurrent decoding suffer from inconsistent decoding speeds across segments, difficulties in frame synchronization, and splicing misalignments, leading to a decline in image quality. Summary of the Invention

[0006] This invention provides an adaptive low-latency hardware decoding and display method and system for video, which realizes low-latency, seamless switching and high-stability decoding and display of multi-format video streams, effectively solving the shortcomings of traditional systems in terms of versatility, smooth switching and clock synchronization.

[0007] To achieve the objectives of this invention, the technical solution adopted is: an adaptive low-latency hardware decoding and display method for video, comprising:

[0008] 1) Adaptive demultiplexing: Parse the input video stream, determine whether the video stream has switched and update the decoding reference information in real time, demultiplex to extract raw stream data; at the same time, identify whether the video stream is in whole frame decoding or N-segment decoding mode, and realize adaptive processing of the video stream;

[0009] 2) Dynamic decoder allocation: When a video stream switch is detected, a new decoder is allocated to the new video stream. At this time, the old and new decoders work in parallel for a short period of time. For the whole frame decoding mode, a single decoder is allocated for hardware decoding. For the N-segment decoding mode, N decoders are dynamically allocated for concurrent decoding.

[0010] 3) Frame synchronization and VCXO dynamic adjustment: In full-frame decoding mode, the latest decoded frame is output; in N-segment decoding mode, by comparing the frame numbers in each segment buffer, the segment data with the same frame number are selected and combined into a complete frame for output in sequence; at the same time, based on the time difference between the new frame obtained by adaptive demultiplexing and the FPGA reading the frame, the working mode of VCXO is dynamically adjusted to match the decoding and display rhythm.

[0011] 4) FPGA display: The FPGA reads frame data and outputs it for display. It also updates the FPGA status based on the VCXO mode and key video stream information stored in the specified pixel position of the current frame data.

[0012] As an optimized solution of the present invention, in step 1), the input video stream is parsed as parsing decoding reference information. The decoding reference information is based on the video stream header information, including container type, video track information and video encoding format.

[0013] As an optimized solution of the present invention, in step 1), the video stream is determined by matching the decoding reference information with the current key reference information of the decoder. If the matching fails, the input video stream is determined to be a new video stream. This is the default case of video stream switching, and the video stream switching signal is sent to the dynamic decoder allocation, frame synchronization and VCXO dynamic adjustment and FPGA display simultaneously.

[0014] As an optimized solution of the present invention, in step 1), the whole frame decoding mode directly performs conventional video stream demultiplexing and stores the bitstream data in a dynamically allocated buffer. For the N-segment decoding mode, the video stream has been pre-divided into N segments, each segment is independently encoded and sent in sequence. When demultiplexing, the order is identified according to the segment ID carried by each segment, and the extracted N segment raw streams are sent into the dynamically allocated buffer in the order of segment ID. At the same time, the N-segment mode signal is sent to the dynamic decoder allocation and frame synchronization and VCXO dynamic adjustment.

[0015] As an optimized solution of the present invention, in step 2), the dynamic decoder allocation determines the resource allocation strategy based on whether a video stream switching signal is received: if a switching occurs, a decoder and a decoded data buffer are allocated to the new video stream, and initialization is completed based on its decoding reference information, so that the old and new decoders can work in parallel for a short time; if no switching occurs, only the current decoder state is updated; at the same time, initialization is performed according to the decoding mode signal: in the whole frame decoding mode, a single decoder is allocated; in the N-segment decoding mode, N decoders are allocated for concurrent decoding according to the number of segments N, and each decoder records the frame number and segment number to which the current segment belongs.

[0016] As an optimized solution of the present invention, in step 3), in the full frame decoding mode, the latest and most complete frame image is obtained and transmitted to the DDR memory via PCIe;

[0017] In N-segment decoding mode, the frame number of the latest segment data in each segment buffer is first obtained, and the smallest frame number is selected as the output reference. Then, the segment data with the corresponding frame number is extracted from each segment buffer and transmitted to DDR through PCIe in segment ID order.

[0018] As an optimization of the present invention, in step 3), the output frequency of the VCXO is dynamically adjusted to be configured as slow mode, normal mode and fast mode; the working mode of the VCXO is dynamically selected according to the average time difference between the time of acquiring a new frame of video data by adaptive demultiplexing and the time of the FPGA reading the frame and displaying it, so as to adjust the clock frequency of the FPGA display.

[0019] As an optimized solution of the present invention, after receiving the video stream switching signal, the frame synchronization and VCXO dynamic adjustment execute the following coordinated control strategy:

[0020] If the decoding frame buffer of the new video stream has not yet generated the latest and most complete frame image, then continue to extract the latest and most complete frame image from the decoding frame buffer of the old video stream and output it to the FPGA display, and adjust the VCXO to slow mode to ensure that the picture is not interrupted.

[0021] When the new decoding frame buffer already contains the latest and most complete frame, switch the VCXO to normal mode to allow sufficient response time for the FPGA display's internal state switching; at this time, continue to output the last frame of the old video stream.

[0022] In each frame of image data transmitted to DDR, the first pixel position contains VCXO mode information and FPGA status information corresponding to the next frame, which is used to ensure the normal display of the next frame after the video stream is switched.

[0023] As an optimized solution of the present invention, in step 4), when the FPGA reads the frame data to be displayed from the DDR, it synchronously sends a frame read signal to the frame synchronization and VCXO dynamic adjustment. The frame read signal is used to record the actual time point when the FPGA reads the frame. Combined with the timestamp of the frame data obtained by adaptive demultiplexing, the average time difference between the two is calculated to provide a basis for the dynamic adjustment of the VCXO working mode. The VCXO mode information takes effect immediately during the current frame screen output and display process, and the FPGA status information for the next frame is processed when the current frame output and display is completed.

[0024] To achieve the objectives of this invention, the technical solution adopted is: an adaptive low-latency hardware decoding display system for video, comprising:

[0025] Adaptive demultiplexing module: parses the input video stream, determines whether the video stream has switched and updates the decoding reference information, demultiplexes and extracts the raw stream data; at the same time, it identifies whether the video stream is encoded as a whole frame or N-segment independent encoding mode, and realizes adaptive processing of the video stream;

[0026] Dynamic decoder allocation module: When a video stream switch is detected, a new decoder is allocated to the new video stream. At this time, the old and new decoders work in parallel for a short period of time. For the whole frame encoding mode, a single decoder is allocated for hardware decoding. For the N-segment independent encoding mode, N decoders are dynamically allocated for concurrent decoding.

[0027] Frame synchronization and VCXO dynamic adjustment module: In full frame decoding mode, output the latest decoded frame; in N-segment decoding mode, by comparing the frame numbers in each segment buffer, select the segment data with the same frame number and combine them in order to form a complete frame for output; at the same time, based on the time difference between the new frame obtained by adaptive demultiplexing and the FPGA reading the frame, dynamically adjust the working mode of VCXO to match the decoding and display rhythm.

[0028] FPGA display module: The FPGA reads frame data and outputs it for display. It also updates the FPGA status based on the VCXO mode and key video stream information stored in the pixel at the specified position of the current frame data.

[0029] This invention has the following positive effects: 1) This invention can adaptively decode and display various video streams. The adaptive demultiplexing module can parse and identify video streams of various encapsulation formats (such as MP4, MKV, TS) and encoding formats (such as H.264, H.265), and especially supports the parsing of whole-frame N-equal-division independently encoded video streams designed to achieve ultra-low latency; the decoder can dynamically adjust its configuration according to changes in the reference information of the video stream, and dynamically allocate N decoders for parallel decoding in N-segment decoding mode; the FPGA can also synchronously adapt to changes in the format and state of the video stream. The above multi-level collaborative mechanism jointly ensures the high versatility and system compatibility of this decoding method;

[0030] 2) This invention ensures smooth and stable image output during video stream switching. The adaptive demultiplexing module detects real-time whether a video stream switch has occurred; the dynamic decoder allocation module assigns a decoder to the new video stream, enabling the old and new decoders to work in parallel for a short period; the frame synchronization and VCXO dynamic adjustment module, in conjunction with the FPGA display, executes the switching strategy: while the new frame is not yet ready, the last frame of the old video stream continues to be output, and the display cycle is extended through the VCXO slow mode to avoid image interruption; once the new decoded frame is ready, a smooth transition to the new video stream occurs. This multi-module linkage mechanism effectively avoids black screens, tearing, or still frames during video stream switching.

[0031] 3) This invention can achieve low-latency decoding and display: In full-frame decoding mode, the output clock frequency of the FPGA display module is dynamically adjusted by VCXO to shorten the time difference from decoding completion to screen display, thereby reducing end-to-end latency; In N-slice decoding mode, N decoders are dynamically allocated to perform parallel decoding of each slice, and the frame synchronization module selects the latest and complete slice data from them and combines them in sequence for output. Combined with the VCXO dynamic adjustment mode, ultra-low latency decoding and display are further achieved. Attached Figure Description

[0032] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0033] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0034] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 An adaptive low-latency hardware decoding display method for video, as described in this embodiment of the invention, includes:

[0036] 1) Adaptive Demultiplexing: Parses the input video stream, determines whether the video stream has switched and updates the decoding reference information in real time, demultiplexes to extract raw stream data (separates the audio and video streams in the container format, such as MP4, and extracts the pure video compressed bitstream, i.e., the "raw stream", such as H.264 Annex B stream, for use by the hardware decoder); dynamically allocates decoders based on the information obtained from the adaptive demultiplexing analysis of the video stream, and identifies whether the video stream is in whole-frame decoding or N-segment decoding mode. In N-segment decoding mode, concurrent decoding is performed to reduce latency, thereby achieving adaptive processing of the video stream.

[0037] In step 1), the input video stream is parsed to obtain decoding reference information. The decoding reference information is based on the video stream header information, including container type, video track information, and video encoding format.

[0038] The process involves matching the decoding reference information with the current key reference information of the decoder. If the match fails, the input video stream is considered a new video stream, which is considered a video stream switch by default. Simultaneously, a video stream switch signal is sent to the dynamic decoder allocation, frame synchronization, VCXO dynamic adjustment, and FPGA display to ensure seamless video stream transitions. If the match is successful, it further determines whether the current video stream requires full-frame decoding or N-segment decoding. In full-frame decoding mode, regular video stream demultiplexing is performed, storing the bitstream data in a dynamically allocated buffer. For N-segment decoding mode, an overall ultra-low latency mode is triggered. The video stream is pre-divided into N segments from the acquisition end. Each segment is independently encoded and sent sequentially. During demultiplexing, the segment IDs carried by each segment identify their order, and the extracted N raw stream segments are sequentially sent to the dynamically allocated buffer according to their segment IDs. The uncertainty of N here fully reflects the adaptability of demultiplexing. Simultaneously, the N-segmentation mode signal is sent to the dynamic decoder for allocation and frame synchronization with VCXO dynamic adjustment.

[0039] The adaptive demultiplexing module supports the demultiplexing of video streams encoded individually in N equal parts for low-latency processing. It also supports the demultiplexing of general-purpose video streams encoded in whole frames—such as MP4, MKV, and MOV program streams, TS transport streams, and raw streams like .H264 and .H265. It can read video stream information to update decoder reference information in real time and achieve seamless picture transitions when switching video streams. The dynamic decoder enables short-term parallel decoding of the old and new video stream decoders during video stream switching, then distinguishes between whole-frame decoding and N-part decoding modes. In N-part decoding mode, it dynamically allocates N decoders for independent concurrent decoding of the N parts.

[0040] 2) Dynamic decoder allocation: When a video stream switch is detected, a new decoder is allocated to the new video stream. At this time, the old and new decoders work in parallel for a short period of time. For the whole frame decoding mode, a single decoder is allocated for hardware decoding. For the N-segment decoding mode, N decoders are dynamically allocated for concurrent decoding.

[0041] When a video stream switching signal is received, the dynamic decoder allocation module assigns a decoder and corresponding decoded data buffer to the new video stream. It then initializes the decoder based on the new video stream's decoding reference information, allowing the new decoder to work in parallel with the old video stream decoder for a short period, thus ensuring the smoothness and stability of the displayed image. Conversely, it only updates the decoder status by obtaining the latest decoding reference information for the current video stream.

[0042] The dynamic decoder allocation module initializes the reference decoding mode signal and carries the reference information. In the full frame decoding mode, only one decoder is allocated for video hardware decoding. In the N-segment decoding mode, N decoders are allocated for concurrent decoding according to the number of segments and the reference information. In addition to storing the decoded video segment data in the corresponding segment data buffer, each decoder also stores the frame number (identifying which frame the segment belongs to) and the segment number (identifying the position of the segment within the frame).

[0043] 3) Frame Synchronization and Dynamic Adjustment of VCXO (Voltage-Controlled Crystal Oscillator): In full-frame decoding mode, the latest decoded frame is output. In N-segment decoding mode, by comparing the frame numbers in each segment buffer, segment data with the same frame number are selected and sequentially combined to form a complete frame for output. Simultaneously, based on the time difference between adaptive demultiplexing to acquire a new frame and the FPGA reading the frame, the VCXO's operating mode is dynamically adjusted to match the decoding and display rhythm. Frame synchronization is used to control the output of new and old decoded frame data during video stream switching to ensure stable display and prevent issues such as still frames and black screens. Furthermore, to address the inconsistent decoding speed caused by differences in reference frames and motion compensation data between different segments of the same frame in N-segment independent decoding, strategies such as buffering multiple frame segments, frame number matching, and ensuring the integrity of the same frame segment are comprehensively judged to select appropriate complete frame data for output and display. Dynamic adjustment of the VCXO is mainly used to shorten the time difference between adaptive demultiplexing to acquire a new frame and the FPGA outputting and displaying the frame, reducing decoding latency while ensuring image output.

[0044] Frame synchronization, in full-frame decoding mode, directly acquires the latest and most complete frame image in the decoding frame buffer and transmits it to DDR memory via PCIe;

[0045] In N-fragment decoding mode, the frame number of the latest fragment data in each fragment buffer is first obtained, and the smallest frame number is selected as the output reference. Then, the fragment data with the corresponding frame number is extracted from each fragment buffer and transmitted to DDR via PCIe in fragment ID order. That is, in full-frame decoding mode, the latest and most complete frame image in the decoding frame buffer is directly retrieved and transmitted to DDR via PCIe; in N-fragment decoding mode, the smallest frame number of the latest fragment data in each fragment is first retrieved, and the fragment data with the corresponding frame number in each fragment buffer is extracted and transmitted to DDR via PCIe in fragment ID order.

[0046] The VCXO's dynamically adjusted output frequency is configured with slow, normal, and fast modes. Based on the average time difference between the time it takes for the adaptive demultiplexing module to acquire a new frame of video data and the time it takes for the FPGA to read and display that frame, the VCXO's operating mode is dynamically selected to adjust the FPGA's display clock frequency. In other words, the VCXO output frequency is set to slow, normal, and fast modes based on statistics. The dynamic mode selection, based on the average time difference between the time the adaptive demultiplexing module acquires a new frame and the time the FPGA reads that frame, shortens decoding latency while ensuring optimal image output.

[0047] Upon receiving a video stream switching signal, frame synchronization and VCXO dynamic adjustment execute the following coordinated control strategy:

[0048] If the decoding frame buffer of the new video stream has not yet generated the latest and most complete frame image, then continue to extract the latest and most complete frame image from the decoding frame buffer of the old video stream and output it to the FPGA display, and adjust the VCXO to slow mode to ensure that the picture is not interrupted.

[0049] When the new decoding frame buffer already contains the latest and most complete frame, switch the VCXO to normal mode to allow sufficient response time for the FPGA display's internal state switching; at this time, continue to output the last frame of the old video stream.

[0050] In each frame of image data transmitted to DDR, the first pixel position contains VCXO mode information and FPGA status information corresponding to the next frame, which is used to ensure the normal display of the next frame after the video stream is switched.

[0051] In summary, after receiving a video stream switching signal, the frame synchronization and VCXO dynamic adjustment module still outputs the latest and most complete frame from the old decoding frame buffer while the new decoding frame buffer does not yet contain the latest and most complete frame, and the VCXO is set to slow mode. When the new decoding frame buffer contains the latest and most complete frame, the VCXO is set to normal mode to allow sufficient time for FPGA state switching, and the latest and most complete frame from the old decoding frame buffer is still sent to the FPGA. The first pixel of each frame data transmission stores VCXO mode information and FPGA state information for the next frame to ensure normal display of the next frame after video stream switching. Fast mode is activated when the average time difference between the time the adaptive demultiplexing module acquires the new frame and the time the FPGA reads the frame is large (>0); slow mode is activated when the time difference is less than 0; and normal mode is activated when the time difference is equal to 0.

[0052] 4) FPGA Display: The FPGA reads frame data for output display and updates its status based on the VCXO mode and key video stream information stored in the specified pixel of the current frame data. The FPGA reads the display frame from DDR and updates it in real-time based on the VCXO mode and key video stream information stored in the specified pixel of the display frame image, ensuring reduced decoding latency and smooth transition between frames during video stream switching. When reading frame data from DDR, the FPGA sends a frame read signal to the frame synchronization and VCXO dynamic adjustment module to calculate the time difference, and then parses the first pixel data to obtain the VCXO mode and the FPGA status information for the next frame. The VCXO mode takes effect immediately when the current frame is output and displayed; the FPGA status information for the next frame is obtained after the current frame is output and displayed.

[0053] This invention also discloses an adaptive low-latency hardware decoding display system for video, comprising:

[0054] Adaptive demultiplexing module: parses the input video stream, determines whether the video stream has switched and updates the decoding reference information, demultiplexes and extracts the raw stream data; at the same time, it identifies whether the video stream is encoded as a whole frame or N-segment independent encoding mode, and realizes adaptive processing of the video stream;

[0055] Dynamic decoder allocation module: When a video stream switch is detected, a new decoder is allocated to the new video stream. At this time, the old and new decoders work in parallel for a short period of time. For the whole frame encoding mode, a single decoder is allocated for hardware decoding. For the N-segment independent encoding mode, N decoders are dynamically allocated for concurrent decoding.

[0056] Frame synchronization and VCXO dynamic adjustment module: In full frame decoding mode, output the latest decoded frame; in N-segment decoding mode, by comparing the frame numbers in each segment buffer, select the segment data with the same frame number and combine them in order to form a complete frame for output; at the same time, based on the time difference between the new frame obtained by adaptive demultiplexing and the FPGA reading the frame, dynamically adjust the working mode of VCXO to match the decoding and display rhythm.

[0057] FPGA display module: The FPGA reads frame data and outputs it for display. It also updates the FPGA status based on the VCXO mode and key video stream information stored in the pixel at the specified position of the current frame data.

[0058] This invention also provides a computer device, which may be a server. The computer device includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor in this computer design provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store the data corresponding to this embodiment. The network interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the above-described method.

[0059] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0060] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0061] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive low-latency hardware decoding and display method for video, characterized in that, include: 1) Adaptive demultiplexing: Parse the input video stream, determine whether the video stream is switched and update the decoding reference information in real time, and demultiplex to extract the raw stream data; Simultaneously, it identifies whether the video stream is in whole-frame decoding or N-segment decoding mode, enabling adaptive processing of the video stream; 2) Dynamic decoder allocation: When a video stream switch is detected, a new decoder is allocated to the new video stream. At this time, the old and new decoders work in parallel for a short period of time. For the whole frame decoding mode, a single decoder is allocated for hardware decoding. For the N-segment decoding mode, N decoders are dynamically allocated for concurrent decoding. 3) Frame synchronization and VCXO dynamic adjustment: In full-frame decoding mode, output the latest decoded frame; In the N-segment decoding mode, by comparing the frame numbers in each segment buffer, the segment data with the same frame number are selected and combined in sequence to form a complete frame for output; at the same time, based on the time difference between the adaptive demultiplexing to obtain a new frame and the FPGA reading a complete frame, the working mode of the VCXO is dynamically adjusted to match the decoding and display rhythm. 4) FPGA display: The FPGA reads frame data and outputs it for display, and updates the FPGA status according to the VCXO mode and key video stream information stored in the pixel at the specified position of the current frame data; In step 3), in full-frame decoding mode, the latest and most complete frame of image is acquired and transmitted to DDR memory via PCIe; In N-segment decoding mode, the frame number of the latest segment data in each segment buffer is first obtained, and the smallest frame number is selected as the output reference. Subsequently, the fragment data corresponding to the frame number is extracted from each fragment cache and transmitted to DDR via PCIe in fragment ID order.

2. The adaptive low-latency hardware decoding and display method for video according to claim 1, characterized in that: In step 1), the input video stream is parsed to obtain decoding reference information. The decoding reference information is based on the video stream header information, including container type, video track information, and video encoding format.

3. The adaptive low-latency hardware decoding and display method for video according to claim 2, characterized in that: In step 1), determining whether the video stream has been switched includes: matching the decoding reference information with the current key reference information of the decoder. If the matching fails, the input video stream is determined to be a new video stream. This is the default case for video stream switching, and the video stream switching signal is sent to the dynamic decoder allocation, frame synchronization and VCXO dynamic adjustment and FPGA display simultaneously.

4. The adaptive low-latency hardware decoding and display method for video according to claim 3, characterized in that: In step 1), the full-frame decoding mode directly performs conventional video stream demultiplexing and stores the bitstream data in a dynamically allocated buffer. For the N-segment decoding mode, the video stream has been pre-divided into N segments, each segment is independently encoded and sent in sequence. During demultiplexing, the order of each segment is identified by the segment ID carried by each segment, and the extracted N segment raw streams are sent into the dynamically allocated buffer in the order of segment ID. At the same time, the N-segment mode signal is sent to the dynamic decoder for allocation, frame synchronization and VCXO dynamic adjustment.

5. The adaptive low-latency hardware decoding and display method for video according to claim 4, characterized in that: In step 2), the dynamic decoder allocation determines the resource allocation strategy based on whether a video stream switching signal is received: if a switching occurs, a decoder and a decoded data buffer are allocated to the new video stream, and initialization is completed based on its decoding reference information, so that the old and new decoders can work in parallel for a short time. If no switch is made, only the current decoder state is updated; Simultaneously, initialization is performed based on the decoding mode signal: a single decoder is allocated in full-frame decoding mode; In N-fragment decoding mode, N decoders are allocated for concurrent decoding according to the number of fragments N. Each decoder records the frame number and fragment number to which the current fragment belongs.

6. The adaptive low-latency hardware decoding and display method for video according to claim 5, characterized in that: In step 3), the output frequency of the VCXO is dynamically adjusted to be configured as slow mode, normal mode and fast mode; the working mode of the VCXO is dynamically selected according to the average time difference between the time of adaptive demultiplexing to acquire a new frame of video data and the time of the FPGA to read the complete frame for display, so as to adjust the clock frequency of the FPGA display.

7. The adaptive low-latency hardware decoding and display method for video according to claim 6, characterized in that: Upon receiving a video stream switching signal, frame synchronization and VCXO dynamic adjustment execute the following coordinated control strategy: If the decoding frame buffer of the new video stream has not yet generated the latest and most complete frame image, then continue to extract the latest and most complete frame image from the decoding frame buffer of the old video stream and output it to the FPGA display, and adjust the VCXO to slow mode to ensure that the picture is not interrupted. When the new decoding frame buffer already contains the latest and most complete frame, switch the VCXO to normal mode to allow sufficient response time for the FPGA display's internal state switching; at this time, continue to output the last frame of the old video stream. In each frame of image data transmitted to DDR, the first pixel position contains VCXO mode information and FPGA status information corresponding to the next frame, which is used to ensure the normal display of the next frame after the video stream is switched.

8. The adaptive low-latency hardware decoding and display method for video according to claim 7, characterized in that: In step 4), when the FPGA reads the frame data to be displayed from the DDR, it synchronously sends a frame read signal to the frame synchronization and VCXO dynamic adjustment. This frame read signal is used to record the actual time point when the FPGA reads the frame. Combined with the timestamp of the frame data obtained by adaptive demultiplexing, the average time difference between the two is calculated, which provides a basis for the dynamic adjustment of the VCXO working mode. The VCXO mode information takes effect immediately during the output and display of the current frame, and the FPGA status information for the next frame is updated when the output and display of the current frame is completed.

9. A system employing the adaptive low-latency hardware decoding and display method for video as described in claim 8, characterized in that: include: Adaptive demultiplexing module: parses the input video stream, determines whether the video stream has switched and updates the decoding reference information, and demultiplexes to extract the raw stream data; Simultaneously, it identifies whether the video stream is encoded as a whole frame or in N-segment independent encoding mode, enabling adaptive processing of the video stream; Dynamic decoder allocation module: When a video stream switch is detected, a new decoder is allocated to the new video stream. At this time, the old and new decoders work in parallel for a short period of time. For the whole frame encoding mode, a single decoder is allocated for hardware decoding. For the N-segment independent encoding mode, N decoders are dynamically allocated for concurrent decoding. Frame synchronization and VCXO dynamic adjustment module: In full-frame decoding mode, outputs the latest decoded frame; In the N-segment decoding mode, by comparing the frame numbers in each segment buffer, the segment data with the same frame number are selected and combined in sequence to form a complete frame for output; at the same time, based on the time difference between the adaptive demultiplexing to obtain a new frame and the FPGA reading a complete frame, the working mode of the VCXO is dynamically adjusted to match the decoding and display rhythm. FPGA display module: The FPGA reads frame data and outputs it for display. It also updates the FPGA status based on the VCXO mode and key video stream information stored in the pixel at the specified position of the current frame data.

Images