A video encoding method, chip architecture, storage device and encoding device thereof

Abstract

The application discloses a video encoding method, comprising the following steps: reading video frame information; dividing the video frame into a plurality of CTU rows; the plurality of CTU rows are batched into the cache module by rows, and the number of cache rows in the cache module is kept at multiple rows to form row buffer data; a plurality of encoding engines respectively perform independent entropy encoding on the plurality of CTU rows in the cache module; and NAL encapsulation is performed on the encoded CTU rows to generate NAL units and output. The application can reduce the bandwidth and capacity pressure of the external memory, improve the encoding efficiency, and shorten the delay.

Description

Technical Field

[0001] This invention relates to the field of video encoding technology, and in particular to a video encoding method, its chip architecture, storage device, and encoding equipment. Background Technology

[0002] Most existing H.265 / HEVC encoders employ a frame-level processing architecture. A typical processing flow is as follows: after a video frame is input, it is stored in an external memory frame buffer. The encoder analyzes and segments the entire frame image, then performs prediction, transform, quantization, and entropy coding on a per-CTU basis. Once the entire frame is encoded, a complete frame bitstream is formed and output as an encapsulated image. In this architecture, the frame buffer needs to completely cache the current encoded frame and multiple reference frames. Taking 1080p resolution as an example, the frame storage overhead is significant, placing considerable pressure on the bandwidth and capacity of external memory. Furthermore, the encoder must wait for the entire frame to be encoded before outputting the bitstream, which can easily lead to high end-to-end encoding latency.

[0003] To alleviate the problem of low efficiency in frame-level serial processing, existing technologies introduce wavefront parallel processing (WPP) mechanism, which uses CTU lines as processing units to achieve parallel encoding: the CTU of line 0 encodes sequentially from left to right, and the encoding of line 1 starts after the second CTU of line 0 is completed. Subsequent lines must wait for the second CTU of the previous line to finish processing before they can be executed, thereby improving the encoding throughput to a certain extent.

[0004] However, this scheme still has significant shortcomings: First, it still relies on the complete frame buffer to store the current frame and the reference frame, and the memory usage and bandwidth pressure have not been fundamentally improved; Second, the bitstream output is still in units of frames, the amount of data transmitted in a single transmission is large, and the end-to-end latency is relatively high; Third, due to the constraints of CABAC context dependency, subsequent CTU lines need to wait for the specified CTU of the previous line to complete encoding, resulting in high inter-line start latency, and there is still room for optimization in parallel efficiency and low-latency encoding performance. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention proposes a video encoding method, its chip architecture, storage device, and encoding equipment, aiming to reduce the bandwidth and capacity pressure of external memory, improve encoding efficiency, and shorten latency.

[0006] To solve the above-mentioned technical problems, the basic technical solution proposed by this invention is as follows: A video encoding method includes the following steps: Read video frame information; Divide the video frames into several CTU lines; The CTU rows are cached in batches to the cache module, and the number of cached rows in the cache module is kept at multiple rows to form row buffer data; Multiple encoding engines perform independent entropy encoding on multiple CTU rows in the cache module; The encoded CTU lines are encapsulated using NAL to generate NAL cells and then output.

[0007] Preferably, each of the encoding engines uses CABAC encoding to encode each CTU line in the cache module, and each encoding engine performs the encoding operation independently for each CTU line to ensure that the encoding process is independent of the encoding status of other CTU lines.

[0008] Preferably, during the encoding operation of the encoding engine, after the encoding engine completes the encoding of the previous CTU line, the caching module deletes the relevant cached data of the previous CTU line, and the encoding engine initializes the CABAC context data formed by the previous CTU line in the CABAC context memory.

[0009] Furthermore, the cache module includes an SRAM unit, and the number of cache lines in the SRAM unit is maintained at 2-4 lines.

[0010] Preferably, the encoding engine must meet the following limitations during the encoding process: In motion estimation: the vertical component of the motion vector (MV) is ≥0, and in the search range, only the top 2 rows are searched; During intra-frame prediction, disable vertical / angle modes and retain only DC / Planar / horizontal modes; During loop filtering, only inline deblocking is performed, and strong filtering is applied to the boundaries.

[0011] The NAL header structure of the NAL unit is 1 byte in size and includes: 1-bit forbidden bit; A 3-digit line type field to identify the first line, middle line, last line, or skipped line; A 2-bit QP index field identifies the offset relative to the baseline QP; A 2-bit extended flag field identifies the encoding mode.

[0012] Preferably, during NAL encapsulation, a hardware autoencapsulation state machine is used for data encapsulation. This state machine is implemented by a hardware FSM, as follows: Initially in the IDLE state, it enters the HEADER state after receiving the CTU line encoding completion signal. This state lasts for one cycle and is used to process NAL header-related operations. It then enters the PAYLOAD state, which lasts for N cycles and is used to process payload data; Next, it enters the ALIGN state, which lasts for 0-7 cycles, to align the data. Then it enters DMA state, which lasts for only 1 cycle, and performs DMA transfer operations; Finally, it enters the DONE state, which lasts for one cycle, indicating the end of this packaging process.

[0013] A video encoding chip architecture, comprising: The reading module is used to read video frame information; The line division and scheduling control unit is used to divide video frames into several CTU lines and allocate them reasonably; A row buffer array, configured to store multiple rows of pixel data, is used to receive and buffer portions of CTU rows divided by the row division and scheduling control unit; An array of encoding engines is configured to encode CTU rows in parallel, with each encoding engine independently performing CABAC entropy encoding. The encapsulation engine is configured to automatically generate NAL headers and encapsulate the bitstream using a hardware state machine. The radio frequency DMA module is used to transfer packaged data.

[0014] A storage device storing program instructions that can be executed by a processor to implement any of the video encoding methods described above.

[0015] A video encoding device includes a processor and a communication circuit. The processor includes an SRAM unit and is coupled to the communication circuit. When the processor, the SRAM unit, and the communication circuit are in operation, they can implement any of the video encoding methods described above.

[0016] The beneficial effects of this invention are: The technical solution of this invention divides video frames into several CTU lines and adopts a batch multi-line caching method, which eliminates the need to completely cache the entire frame image and reference frame, effectively reducing the pressure on the storage module and the bandwidth requirements. It can even be implemented using only on-chip SRAM without the need for external storage. At the same time, multiple encoding engines are used to independently entropy encode the multiple CTU lines in the cache module, completely eliminating the dependency problem of inter-line CABAC context in traditional encoding, eliminating inter-line waiting delay, and greatly improving the encoding parallelism and throughput efficiency. In addition, the encoded CTU lines are encapsulated and output in real time without waiting for the entire frame to be encoded, which significantly reduces the end-to-end encoding latency. Moreover, the NAL unit is generated by CTU line, resulting in a smaller amount of data transmitted at one time, more flexible bitstream organization, and better suitability for real-time transmission and low-latency application scenarios. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the method steps of a video encoding method according to the present invention; Figure 2 This is a schematic diagram of a row buffer array for a video coding method according to the present invention; Figure 3 This is a schematic diagram of the data flow of a row buffer array in a video encoding method according to the present invention; Figure 4 This is a schematic diagram of the row-level streaming coding architecture of a video coding method according to the present invention; Figure 5 This is a schematic diagram of a video encoding chip architecture according to the present invention; Figure 6 This is a schematic diagram illustrating a video encoding method of the present invention, which incorporates row-level independent CABAC encoding. Detailed Implementation

[0018] The following will be combined with the appendix Figure 1 To be continued Figure 6 The technical solutions in the embodiments of the present invention have been clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] As described in the background section, existing technologies rely on a complete frame buffer to store the current frame and the reference frame, resulting in significant memory usage and bandwidth pressure, requiring the involvement of external storage. Furthermore, the bitstream output is still based on frames, leading to a large amount of data transmitted per transmission and high end-to-end latency. Additionally, due to CABAC context dependencies, subsequent CTU lines must wait for the specified CTU in the previous line to complete encoding, resulting in high inter-line start latency.

[0020] Therefore, this invention proposes a video encoding method. The core idea is as follows: video frame information is divided into several CTU lines so that multiple CTU line information can be buffered in a buffer module. Since multiple CTU line information occupies less memory, it can reduce memory usage and bandwidth pressure. It can even directly use on-chip SRAM units for buffering without the participation of external memory. Then, multiple encoding engines are used to simultaneously perform independent entropy encoding on multiple CTU lines (preferably using CABAC encoding, and the encoding process of each encoding engine for the corresponding CTU line is independent and does not interfere with each other), avoiding inter-line context dependency constraints and reducing inter-line startup latency. Finally, the encoded CTU lines are encapsulated in NAL in real time and output, thereby achieving low storage, low latency, and high parallelism video encoding.

[0021] Specifically, such as Figure 1 As shown, a video encoding method includes the following steps: Read video frame information; Divide the video frames into several CTU lines; The CTU rows are cached in batches to the cache module, and the number of cached rows in the cache module is kept at multiple rows to form row buffer data; Multiple encoding engines perform independent entropy encoding on multiple CTU rows in the cache module; The encoded CTU lines are encapsulated using NAL to generate NAL cells and then output.

[0022] This video encoding method divides video frames into several CTU lines and uses a batch multi-line buffering approach. It eliminates the need to completely buffer the entire frame image and reference frame, effectively reducing the pressure on storage modules and bandwidth requirements. It can even be implemented using on-chip SRAM without the need for external storage. At the same time, it uses multiple encoding engines to independently entropy encode multiple CTU lines in the buffer module, completely eliminating the dependency problem of inter-line CABAC context in traditional encoding, eliminating inter-line waiting latency, and significantly improving encoding parallelism and throughput efficiency. In addition, it encapsulates and outputs the encoded CTU lines in real time without waiting for the entire frame to be encoded, which significantly reduces end-to-end encoding latency. Furthermore, generating NAL units on a CTU line basis results in a smaller amount of data transmitted per transmission, improving output efficiency.

[0023] The cache module can be an external memory or an on-chip SRAM.

[0024] Preferably, the cache module is an on-chip SRAM, which has fast read and write speeds and low latency, can quickly respond to the parallel reading requirements of multiple encoding engines for different CTU rows, and can meet the requirement of caching multiple CTU rows of data at once.

[0025] During the encoding process, the number of CTU rows cached in the SRAM cell is kept between 2 and 4. This satisfies the requirement of multiple encoding engines reading CTU row data in parallel, ensuring encoding parallelism and avoiding engine idleness. It also minimizes SRAM capacity requirements, reducing on-chip SRAM hardware usage and power consumption. Furthermore, it forms a stable row buffer pipeline, preventing encoding engines from stalling while waiting for data, further reducing end-to-end encoding latency. It also effectively isolates data from each CTU row, avoids read / write conflicts and inter-row interference, and achieves low-storage, low-latency, and highly parallel encoding results.

[0026] Independent entropy coding methods include CABAC coding, CAVLC, ECB, Rice coding, Golomb-Rice coding, and Huffman coding.

[0027] Preferably, each encoding engine uses CABAC encoding to encode each CTU line in the cache module, and each encoding engine performs the encoding operation independently for each CTU line to ensure that the encoding process is independent of the encoding status of other CTU lines.

[0028] In detail, during the encoding process of the encoding engine, such as Figure 6 As shown, after the encoding engine completes encoding the previous CTU line, the caching module deletes the relevant cached data of the previous CTU line, and the encoding engine initializes the CABAC context data formed by the previous CTU line in the CABAC context memory. By initializing the CABAC context data, inter-line CABAC context dependencies are eliminated, cross-line synchronization is not required, and a natural pipeline is achieved.

[0029] To achieve independent parallel encoding between CTU rows and avoid encoding wait caused by inter-row data dependencies, the following constraints are imposed on the encoding engine during encoding: During motion estimation: the vertical component of the motion vector (MV) is ≥0, and reference is only allowed to the image region above or at the same position. Reference is not allowed to the uncoded lines below. In terms of search range, the matching block is only searched within the range of the two lines above the current line. The entire frame and the lower frame are no longer searched. This ensures that when encoding the current line, it only depends on the few lines above that have already been encoded. The lines below do not need to wait and will not depend on each other.

[0030] It is worth noting that the two lines above the search bar are for reference only and are not for context; the two are not contradictory.

[0031] In one implementation, the row buffer array design uses 128KB of SRAM as a cache module, dividing the entire SRAM into four independent banks (storage areas), such as... Figures 2 to 3 As shown, these are Bank0, Bank1, Bank2, and Bank3, which do not interfere with each other.

[0032] The workflow is as follows: Bank0: Current line, stores the CTU being encoded, written from sensor / image data for the encoding engine to read and process; the encoder is currently encoding this line. Bank1: Stores the image that has been edited in the previous row and uses it as reference 1 for motion estimation and intra-frame prediction in the current row; Bank2: Stores the image that has been edited in the previous row as reference 2, used for motion estimation and intra-frame prediction in the current row; Bank3: Reconstructed line (output), stores the encoded and reconstructed pixel data for subsequent reference, or directly output for display / transmission.

[0033] The entire process supports 2 reads and 1 write, simultaneously reading Bank1 and Bank2 for reference while writing new row data to Bank0. It supports parallel access from multiple ports, resulting in high hardware efficiency, no lag, and no waiting. Furthermore, it only caches 4 rows of CTU, minimizing storage usage. It supports parallel reading from multiple encoding engines, row-level pipelines, and row-level overlays, achieving extremely low-latency encoding.

[0034] In detail, using 1080p (34 lines) as an example, this demonstrates how the 4X parallel architecture (4 encoding engines encoding in parallel) ensures a natural flow: clock cycle Engine 0 Engine 1 Engine 2 Engine 3 Data ready status T0 Row 0 Row 1 Row 2 Row 3 Bank1 / 2: Empty (IDR frame reference) T1 Row 4 Row 5 Row 6 Row 7 Bank 1 / 2: Rows 2 / 3 are ready (for reference for Row 4) T2 Row 8 Row 9 Row 10 Row 11 Bank 1 / 2: Rows 6 / 7 are ready (for reference in Row 8) Row 4 references Rows 2 & 3: When Engine 0 starts Row 4, Engine 2 has already completed Row 2, and Engine 3 has completed Row 3; the data is already in SRAM. Engine 0 does not need to send a "please complete encoding" request signal to Engines 2 / 3; instead, data availability is naturally guaranteed through a fixed timing stagger (4-row interval). Data dependencies are naturally satisfied through timing design, thus avoiding cross-row synchronization blocking.

[0035] During intra-frame prediction, the vertical / angle mode is disabled, and vertical prediction and angle prediction are disabled. Only the DC / Planar / Horizontal modes are retained. DC is the mean prediction mode and Planar mode is the plane fitting mode. These three modes mainly rely on the pixels already encoded to the left of the current row and have very weak or even almost no dependence on the row above. This further reduces inter-row dependence, allowing each row to be encoded almost independently, which is suitable for multi-engine parallel processing.

[0036] During loop filtering, only intra-line deblocking is performed; that is, block artifacts across line boundaries are not removed, and filtering is only performed within the current CTU line. Furthermore, due to the restriction of the prediction mode, the image is prone to obvious block boundaries, so strong filtering is required at the boundaries to compensate for image quality loss. This avoids introducing cross-line dependencies in the filtering stage while ensuring that the image does not become unsightly due to the restriction of the prediction mode.

[0037] By implementing the above constraints, the data dependency between CTU lines can be significantly reduced, making the encoding processes of each line independent of each other, adapting to multi-engine parallel encoding architecture, and ensuring the quality of encoded images.

[0038] To achieve low latency and lightweight transmission at the row level, the NAL header structure of the NAL unit is simplified to 1 byte, compressing the originally very long NAL header in the H.265 standard to its minimum size, specifically for carrying one line of CTU key information.

[0039] Specifically, including: One-bit forbidden bit, occupying 1 bit, is used to mark whether this line of data is valid or has an error; The 3-bit line type field occupies 3 bits and is used to identify the position of this line in the whole frame, indicating the first line (the first line of a frame), the middle line, the last line (the last line of a frame), or the skipped line (this line is not encoded and is skipped directly), so that the decoder can know which type of line it is and how to splice it into a frame. The 2-bit QP index field, occupying 2 bits, identifies the offset relative to the baseline QP. QP is a quantization parameter that controls sharpness and compression ratio. Here, the complete QP is not stored, only the offset relative to the reference QP is stored, so that the complete QP is not transmitted, saving bits and reducing overhead. The 2-bit extended flag field is used to identify the encoding mode used for this line, such as intra / inter-frame, strong filtering / normal filtering, independent CABAC mode, etc., so that the decoder can quickly know which method to use for decoding.

[0040] Employing a simplified 1-byte NAL header structure, the identification and control of the line-level bitstream are completed with minimal overhead through the prohibition bit, line type field, QP index field, and extended flag field. This allows each CTU line to be transmitted and decoded as an independent NAL unit without relying on the entire frame information, further reducing end-to-end latency and improving transmission efficiency.

[0041] Preferably, during NAL encapsulation, a hardware automatic encapsulation state machine is used for data encapsulation. This state machine is implemented by a hardware FSM (finite state machine). Data encapsulation is achieved through a fixed, step-by-step circuit logic in the hardware. That is, after a line of CTU is encoded, NAL is automatically packaged and sent out automatically. The whole process runs in hardware, without occupying the CPU, without waiting, and with extremely low latency.

[0042] The entire packaging process consists of 5 steps, such as... Figure 4 As shown, the details are as follows: Initially in the IDLE state, it starts working after receiving the CTU line encoding completion signal and enters the HEADER state. This state lasts for one cycle and is used to process NAL header-related operations, that is, the hardware automatically constructs the 1-byte simplified NAL header mentioned above and writes information such as line type, QP offset, and encoding mode into it. Then it enters the PAYLOAD state (payload state), which lasts for N cycles (N = the length of the encoded data line), and is used to process the payload data, taking the CABAC encoded line of bitstream data as the content of the NAL packet and sending it out. Next, it enters the ALIGN state (alignment state), which lasts for 0-7 cycles. During this state, the data is aligned to ensure that the data length is aligned to bytes / 32 bits / 64 bits, which facilitates subsequent DMA and transmission and prevents misalignment. Then it enters DMA state (transfer state), which lasts for only 1 cycle, performs DMA transfer operation, starts hardware DMA, and sends the entire packaged NAL packet directly to the output interface / memory without CPU handling; Finally, it enters the DONE state (end state), which lasts for 1 cycle, indicating that the encapsulation process is over. That is, the NAL encapsulation of this line is completed, and it returns to IDLE to wait for the next line.

[0043] By quickly completing the "header generation → data loading → alignment → DMA transmission" process in 5 states, the system can encapsulate and output each line immediately after encoding, significantly reducing end-to-end latency.

[0044] In one embodiment, to facilitate decoding and improve decoding efficiency, the following SPS parameter set is used, transmitted once per frame, to achieve extremely low overhead and low latency transmission, as follows: byte content 0-1 Width - 1 (13 bits) + Height - 1 (13 bits) 2 Bit depth (2 bits) + chroma format (2 bits) + level (4 bits) 3 Frame rate -1 (8bit) 4-5 Global QP (8 bits) + number of rows - 1 (8 bits) 6-7 Frame sequence number (16 bits) 8-11 CRC32 (lower 24 bits) Optionally, each CTU line can be assigned a very small header, named the line-level PPS (4-6 bytes), which can be carried in conjunction with the line-level encoding and independent NAL unit mentioned above, as follows: byte content 0 Row number (8 bits) 1 Slice type (2 bits) + Reference row (6 bits) 2 Row QP offset (8 bits) 3-4 Target / Actual Bitrate (16bit) 5 CRC8 In this data structure, byte 0 indicates the row number, which is used by the decoder to concatenate the data sequentially and prevent out-of-order or missing rows. Byte 1, the slice type (2 bits), indicates whether the row is an I-row / intra-frame row, a P-row / inter-frame row, etc., and the reference row (6 bits), which is used by the decoder to understand which previous rows were referenced during the encoding of this row. Byte 2, the row QP offset (8 bits), is the image quality fine-tuning value for each row, which is added with the offset to achieve dynamic control of the image quality of each row. Bytes 3-4, the target / actual bitrate (16 bits), record the bitrate information after encoding this row, which is used for real-time bitrate control and bandwidth smoothing. Byte 5, CRC8, is used for single-row data verification to check whether this row has been transmitted incorrectly or is damaged.

[0045] In one implementation, adaptive rate control (ABR) is performed during encoding by the encoding engine. That is, before sending each line, the encoder automatically and in real time fine-tunes the quality (QP) of that line based on network conditions, buffer fullness, and frame movement to prevent video stuttering, screen tearing, overflow, etc., and to ensure that the transmission is smooth, without data loss or overflow.

[0046] Specifically as follows: Input signals (processed per line <0.05ms): • Actual bitrate of forward stroke (from the encapsulation engine register) • Channel state (SNR / packet loss rate / RTT, from RF MAC) • Buffer zone level (row-level FIFO occupancy) • Scene detection (motion complexity, hardware variance calculation) Decision logic (hardware implementation): If packet loss rate > 5% OR RTT > 2ms: QI = min(QI+1, 3) / / Increase QP, decrease bitrate RT = 100 / / Enable skip line mode ELIF buffer zone level >80% QI = min(QI+1, 3) / / Prevent overflow ELIF Motion Complexity > THRESHOLD: QI = max(QI-1, 0) / / Mass required for rapid motion EF = 10 / / Enable Fast Motion Mode ELSE: QI = Maintain or fine-tune (±1) Output: QI, RT, EF → Directly written to the NAL header The hardware-implemented line-level adaptive rate control (ABR), based on CTU lines, makes rapid hardware decisions within less than 0.05ms according to real-time network status, buffer level, motion complexity, and historical bitrate. It dynamically adjusts the QP index, encoding mode, and line skipping strategy for each line, achieving real-time, fine-grained, and stable bitrate control. This effectively avoids network fluctuations and buffer overflows while ensuring image quality in motion scenes and adapting to low-latency, high-real-time line-level parallel encoding architectures.

[0047] In one specific embodiment, when the drone is moving at high speed and the packet loss rate increases from 1% to 8%, the RF MAC reports the channel status every 50ms. When the hardware ABR module detects that the packet loss rate is >5%, it automatically makes adjustments, as follows: The current encoding line QI is adjusted from 01 (baseline) to 11 (+4QP); The RT field is set to 100 (skip line mode); The next line copies pixels from the previous line, reducing the bitrate by 50%. After three consecutive rows, if the packet loss rate recovers, the QI will gradually decrease.

[0048] Based on the same technical concept, in one specific implementation, the real-time encoding of 1080p@60fps involves the following steps: Step 1: The sensor inputs pixel data line by line through the MIPI CSI-2 interface; Step 2: The row buffer array receives the current row and stores it in Bank0 (the current encoded row); Step 3: Encoding engine 0 detects that Bank 0 is full and starts CTU line encoding: a) Read the top two lines from Bank1 / Bank2 as a reference. b) Motion estimation: Search range ±32 pixels, MVy≥0 c) Intra-frame prediction: DC / Planar / Horizontal mode only d) Transform quantization: Standard H.265 DCT / DST e) CABAC encoding: Independent initialization of probability tables, parallel processing with 4 bins / cycles. f) Rebuild: Write back to Bank3 (rebuild row); Step 4: Output the CABAC stream to the encapsulation engine FIFO; Step 5: The hardware FSM automatically generates the 1-byte NAL header mentioned above; Step 6: Insert mini PPS (every 16 rows) and anti-race bytes; Step 7: Trigger RF DMA, and data is sent directly to RF MAC; Step 8: The Bank0 marker can be overwritten; wait for the next line of input. The loop executes steps 1-8, each line is executed independently, and 34 lines are executed in parallel (4 engines).

[0049] In a specific application, an on-chip full-buffer architecture is adopted. The pixel data output by the image sensor is directly stored in the on-chip SRAM line buffer. After the encoder completes the line-level video encoding process, the reconstructed image data is stored in the on-chip reconstruction SRAM and output to the outside. At the same time, the reconstructed data is fed back to the line buffer as reference data for subsequent encoding. The entire data acquisition, encoding, reference and output process is completed in the on-chip SRAM without the need for external DDR memory, realizing low latency, high real-time performance and extremely simple hardware structure video encoding.

[0050] It should be understood that the above video encoding method is preferably implemented using dedicated hardware logic circuits, but it can also be implemented in an integrated circuit chip that integrates a processor, memory, and interface circuits, or it can be implemented using a combination of hardware and software / firmware. For example, some control flows can be implemented as computer program code / instructions, executed by the processor, and stored in a computer-readable storage medium; while key modules such as core encoding, line buffer scheduling, encapsulation, and bitrate control are preferably implemented using dedicated hardware logic / circuits. The hardware logic / circuits may include application-specific integrated circuit chips (ASICs) and field-programmable gate arrays (FPGAs), which integrate modules such as parallel encoding engine arrays, line buffer arrays, encapsulation state machines, bitrate control hardware circuits, and communication interfaces, and can independently or in conjunction with embedded firmware / program code to complete all encoding functions.

[0051] Based on the same technical concept, this application provides a video encoding chip architecture. The relevant working logic can be referred to in the related embodiments of the video encoding method described above, and will not be repeated here. Figure 5 As shown, it specifically includes: The reading module is used to read video frame information; The line division and scheduling control unit is used to divide video frames into several CTU lines and allocate them reasonably, distributing the CTU lines to the subsequent encoding engines so that they can work simultaneously; A row buffer array, configured to store multiple rows of pixel data, is used to receive and buffer portions of CTU rows divided by the row division and scheduling control unit; An array of encoding engines is configured to encode CTU rows in parallel, with each encoding engine independently performing CABAC entropy encoding. The encapsulation engine is configured to automatically generate NAL headers and encapsulate the bitstream using a hardware state machine. The radio frequency DMA module is used to transfer packaged data.

[0052] The line partitioning and scheduling control unit is implemented by a hardware line counter, a scheduling control finite state machine (FSM), and a multiplexing interconnection circuit. It is used to complete the CTU line partitioning of video frames, the writing control of the line buffer array, and the task scheduling of the encoding engine array. All of them are executed in a hardware pipeline manner to ensure low latency and parallel processing efficiency.

[0053] Specifically, the line counter / line address generator counts line by line, such as line 0 → line 1 → line 2… until the end of a frame, and is responsible for segmenting a frame of image by CTU lines; the scheduling control FSM (finite state machine) controls the rhythm of the entire pipeline, when to read a line → when to write to the line buffer → which encoding engine to allocate to, and determines whether the line buffer has empty space and whether the encoding engine is idle; the multiplexer / interconnect switch sends the current CTU line data to the corresponding bank or encoding engine according to the scheduling result, realizing timely automatic allocation.

[0054] The row buffer array is implemented through the SRAM cells inside the chip, dividing the SRAM cells into multiple storage areas.

[0055] When the encapsulation engine performs encapsulation, it uses the hardware state machine described above to automatically generate the NAL header and encapsulate the bitstream.

[0056] Preferably, during encapsulation, the NAL header uses the 1-byte NAL header described above.

[0057] In a specific application, a storage device, which is a computer-readable storage medium, stores program instructions thereon. When these program instructions are loaded and executed by a processor, they can implement the steps in the aforementioned video coding method, such as CTU line partitioning, line buffer scheduling, multi-engine parallel independent entropy coding, NAL encapsulation, and line-level adaptive bitrate control, thereby completing low-latency line-level video coding on a general-purpose processor or embedded processing platform.

[0058] In one embodiment, a video encoding device includes a processor and a communication circuit. The processor includes an SRAM unit, the processor is coupled to the communication circuit, and the processor, the SRAM unit, and the communication circuit can implement any of the video encoding methods described above when they are in operation.

[0059] In this implementation, the video encoding device uses a processor as the core processing unit. The built-in SRAM unit is configured to implement a multi-line CTU line buffer array function, used to temporarily store pixel data to be encoded, historical reference line data, and reconstructed data. The communication circuit is used to complete the input of the original video data and the output transmission of the encoded bitstream. During operation, the processor, through internal hardware logic or by executing corresponding program instructions, completes the CTU line division and scheduling of the video frame, writes pixel data into the SRAM unit for line-level buffering, then calls the parallel encoding processing flow to perform independent entropy encoding on the multi-line CTU lines, and then generates a simplified NAL header and completes bitstream encapsulation through hardware encapsulation logic. Simultaneously, it implements line-level adaptive bitrate control based on channel state, buffer level, and image complexity. Finally, the encapsulated line-level NAL unit data is sent out through the communication circuit, thus fully realizing the aforementioned low-latency, line-level parallel video encoding method.

[0060] Based on the disclosure and teachings of the foregoing specification, those skilled in the art can make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the present invention should also fall within the protection scope of the claims of the present invention. Furthermore, although some specific terms are used in this specification, these terms are only for convenience of explanation and do not constitute any limitation on the present invention.

Claims

1. A video encoding method, characterized in that, Includes the following steps: Read video frame information; divide the video frame into several CTU lines; cache the several CTU lines in batches according to the line into the cache module, and keep the number of cache lines in the cache module at multiple lines to form line buffer data; multiple encoding engines perform independent entropy encoding on the multiple CTU lines in the cache module; encapsulate the encoded CTU lines into NAL units and output them.

2. The video encoding method according to claim 1, characterized in that, When each of the encoding engines performs encoding, it uses CABAC encoding to encode each CTU line in the cache module. Each encoding engine performs encoding operation independently for each CTU line to ensure that the encoding process is independent of the encoding status of other CTU lines.

3. The video encoding method according to claim 2, characterized in that: During the encoding operation of the encoding engine, after the encoding engine completes the encoding of the previous CTU line, the cache module deletes the relevant cache data of the previous CTU line, and the encoding engine initializes the CABAC context data formed by the previous CTU line in the CABAC context memory.

4. The video encoding method according to claim 1, characterized in that, The cache module includes SRAM units, and the number of cache lines in the SRAM units is maintained at 2-4 lines.

5. The video encoding method according to claim 1, characterized in that, During the encoding process, the encoding engine must meet the following restrictions: In motion estimation: the vertical component of the motion vector (MV) is ≥0, and in the search range, only the top two rows are searched; In intra-frame prediction, vertical / angle mode is disabled, and only DC / Planar / horizontal mode is retained; In loop filtering, only in-line deblocking is performed, and strong filtering is applied to the boundaries.

6. The video encoding method according to claim 1, characterized in that, The NAL header structure of the NAL unit is 1 byte in size and includes: 1 forbidden bit; 3 line type field, which identifies the first line, middle line, last line or skipped line; 2 QP index field, which identifies the offset relative to the baseline QP; and 2 extended flag field, which identifies the encoding mode.

7. The video encoding method according to claim 1, characterized in that, During NAL encapsulation, a hardware automatic encapsulation state machine is used for data encapsulation. This state machine is implemented by a hardware FSM, as follows: Initially, it is in the IDLE state. After receiving the CTU line encoding completion signal, it enters the HEADER state, which lasts for 1 cycle and is used to handle NAL header-related operations. Then, it enters the PAYLOAD state, which lasts for N cycles and is used to process payload data. Next, it enters the ALIGN state, which lasts for 0-7 cycles and performs data alignment. After that, it enters the DMA state, which lasts for only 1 cycle and performs DMA transfer operations. Finally, it enters the DONE state, which lasts for 1 cycle, indicating that the encapsulation process is over.

8. A video encoding chip architecture, characterized in that, include: The reading module is used to read video frame information; the line division and scheduling control unit is used to divide the video frame into several CTU lines and allocate them reasonably. A row buffer array is configured to store multiple rows of pixel data, used to receive and buffer portions of CTU rows divided by the row partitioning and scheduling control unit; an encoding engine array is configured to encode CTU rows in parallel, with each encoding engine independently completing CABAC entropy encoding; an encapsulation engine is configured to automatically generate a NAL header and encapsulate the bitstream through a hardware state machine; and a radio frequency DMA module is used to transmit the encapsulated data.

9. A storage device, characterized in that, The device stores program instructions that can be executed by a processor to implement the video encoding method according to any one of claims 1 to 7.

10. A video encoding device, characterized in that, include: A processor and a communication circuit, the processor including an SRAM unit, the processor being coupled to the communication circuit, and the processor, the SRAM unit and the communication circuit being configured to implement the video encoding method according to any one of claims 1-7.

Images