Video coding acceleration method and device based on heterogeneous architecture, electronic equipment and medium

Abstract

The application relates to a video coding acceleration method and device based on a heterogeneous architecture, an electronic device and a medium. The method is applied to a video processing system comprising a hardware encoder and a software encoder, and comprises the following steps: the hardware encoder performs rate-distortion optimization processing on input video data to generate mode decision data comprising block-level coding information; the mode decision data is exported from the hardware encoder to a storage area accessible by the software encoder through a special export interface independent of compressed stream output; and the software encoder obtains the mode decision data from the storage area and uses the mode decision data to accelerate the coding of the input video data when performing a mode decision step in a video coding process.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and more specifically to video coding acceleration methods, apparatuses, electronic devices, and media based on heterogeneous architectures. Background Technology

[0002] In existing technologies, as video resolution increases to 4K, 8K, and even higher, the computational complexity of video encoding increases dramatically. While traditional software encoders support flexible encoding strategies and high-quality output, they face significant challenges: the mode decision step is extremely time-consuming, accounting for 40%-60% of the total encoding time for high-resolution videos, making it difficult to meet real-time requirements for single-frame encoding; search coverage is difficult to guarantee, as the excessive computation makes it difficult to fully search the massive number of candidate modes, leading to difficulties in converging encoding efficiency and balancing quality and speed. While standalone hardware encoders are fast and consume less power, once encoding is complete, the detailed mode decision data generated internally is discarded, and only a compressed bitstream is output. Therefore, a video encoding acceleration solution is needed. Summary of the Invention

[0003] The purpose of this application is to provide a video coding acceleration method, apparatus, electronic device and medium based on heterogeneous architecture.

[0004] According to a first aspect of the embodiments of this application, a video encoding acceleration method based on a heterogeneous architecture is provided, applied to a video processing system including a hardware encoder and a software encoder. The method includes: the hardware encoder performing rate-distortion optimization processing on input video data to generate mode decision data containing block-level coding information; exporting the mode decision data from the hardware encoder to a storage area accessible to the software encoder through a dedicated export interface independent of the compressed bitstream output; the software encoder obtaining the mode decision data from the storage area and using the mode decision data to accelerate the encoding of the input video data when performing the mode decision step in the video encoding process.

[0005] Optionally, the hardware encoder performs rate-distortion optimization processing on the input video data to generate mode decision data containing block-level coding information, including: dividing the input video data into multiple coding blocks; performing rate-distortion optimization calculation on each coding block to obtain the corresponding rate-distortion cost; determining the block partitioning structure, predicted mode information, and motion vector information of each coding block based on the rate-distortion cost; and generating mode decision data using at least one of the block partitioning structure, predicted mode information, motion vector information, and rate-distortion cost.

[0006] Optionally, the mode decision data also includes a skip flag; the mode decision data is used to accelerate the encoding of the input video data, including: determining whether the skip flag in the mode decision data is set; when the skip flag is set, the software encoder skips the rate-distortion optimization search process for the coding block.

[0007] Optionally, mode decision data can be exported from the hardware encoder to a storage area accessible by the software encoder via a dedicated export interface independent of the compressed bitstream output. This includes: determining the data transmission channel type between the hardware encoder and the software encoder; enabling the dedicated export interface if the data transmission channel type is a direct memory access channel, a shared memory interface, or a system bus bypass interface; and writing the mode decision data directly to the storage area via the enabled dedicated export interface.

[0008] Optionally, the software encoder obtains mode decision data from the storage area and utilizes the mode decision data to accelerate the encoding of the input video data during the mode decision step in the video encoding process. This includes: in response to the software encoder initiating a mode decision search, determining whether mode decision data exists in the storage area; if mode decision data exists, reading the mode decision data as a reference for mode decision; reducing the search range for rate distortion optimization based on the reference; or, skipping some calculation steps to accelerate the encoding of the input video data.

[0009] Optionally, the search scope for rate distortion optimization can be narrowed based on reference criteria; or, some computational steps can be skipped, including any of the following strategies: a constrained search strategy, which uses pattern decision data as high-probability candidates and performs rate distortion optimization search within a local range; a full trust strategy, which directly uses pattern decision data as the final encoding decision and skips the rate distortion optimization search of the software encoder; and a cue strategy, which uses pattern decision data as the initial state seed and performs a complete rate distortion optimization search.

[0010] Optionally, the execution method of the constraint search strategy includes: if the pattern decision data contains motion vector information, then the motion vector information in the pattern decision data is used as the center point of the motion estimation search, and the search range is limited to the preset pixel range around the center point; the block partitioning structure in the pattern decision data is used as the primary candidate for rate-distortion optimization search, and the alternative structures with a partitioning granularity one level different from the primary candidate are evaluated.

[0011] Optionally, generating mode decision data containing block-level coding information includes: mapping at least one of block partitioning structure, prediction mode information, motion vector information, and rate-distortion cost to a common format that supports interoperability between the first video coding standard and the second video coding standard; the mode decision data is encapsulated in a codec-decoding-independent common data structure.

[0012] Optionally, the mapping is to a general format that supports interoperability between the first video coding standard and the second video coding standard, including: when the hardware encoder generates mode decision data, converting the internal mode identifier of the first video coding standard into a general mode identifier in a general data structure; when the software encoder uses the mode decision data, mapping the general mode identifier in the general data structure to the mode identifier of the second video coding standard; wherein the general mode identifier is an intermediate representation format between the first video coding standard and the second video coding standard.

[0013] According to a second aspect of the embodiments of this application, a video encoding acceleration device based on a heterogeneous architecture is provided, applied to a video processing system including a hardware encoder and a software encoder. The device includes: a generation module, used to perform rate-distortion optimization processing on input video data through the hardware encoder to generate mode decision data containing block-level coding information; an export module, used to export the mode decision data from the hardware encoder to a storage area accessible by the software encoder through a dedicated export interface independent of the compressed bitstream output; and an encoding module, used to obtain the mode decision data from the storage area through the software encoder, and to accelerate the encoding of the input video data using the mode decision data when performing the mode decision step in the video encoding process.

[0014] According to a third aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the method described above.

[0015] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that, when the executable computer program in the storage medium is executed by a processor, the above-described method can be implemented.

[0016] The beneficial effects of this application are that by reusing the rate-distortion optimization results already completed by the hardware encoder, redundant calculations in the software encoder are eliminated, significantly improving the overall encoding throughput of the heterogeneous system and achieving an increase in encoding speed while maintaining essentially the same video quality. Furthermore, by using a dedicated export interface independent of the compressed bitstream output mode decision data, the output video bitstream is ensured to conform to standard specifications, without affecting compatibility with subsequent decoding and playback, while also enabling flexible transmission of internal acceleration data within the encoding process. By fully utilizing the high-speed computing power of the hardware encoder and the high-quality control capabilities of the software encoder, complementary advantages are achieved in the heterogeneous architecture, reducing the overall system power consumption and computing resource consumption. Attached Figure Description

[0017] Figure 1This is a flowchart illustrating a video coding acceleration method based on a heterogeneous architecture provided in an embodiment of this disclosure.

[0018] Figure 2 A schematic flowchart illustrating the pattern decision data generation method provided in this embodiment of the disclosure.

[0019] Figure 3 This is a flowchart illustrating the encoding acceleration method provided in an embodiment of the present disclosure.

[0020] Figure 4 A schematic flowchart illustrating the pattern decision data export method provided in this embodiment of the disclosure.

[0021] Figure 5 This is a flowchart illustrating the software encoder encoding acceleration method provided in an embodiment of the present disclosure.

[0022] Figure 6 This is a flowchart illustrating the constraint search strategy execution method provided in an embodiment of the present disclosure.

[0023] Figure 7 A schematic flowchart illustrating the pattern decision data generation method provided in this embodiment of the disclosure.

[0024] Figure 8 This is a flowchart illustrating the general format mapping method provided in the embodiments of this disclosure.

[0025] Figure 9 This is a schematic diagram of the structure of a video coding acceleration device based on a heterogeneous architecture proposed in an embodiment of this application.

[0026] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0027] Unless otherwise defined, the technical or scientific terms used in this specification and claims shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. Specific embodiments of the invention will be described below with reference to the accompanying drawings. It should be noted that, in order to provide a concise description, this specification cannot provide a detailed description of all features of the actual embodiments. Without departing from the spirit and scope of the invention, those skilled in the art can make modifications and substitutions to the embodiments of the invention, and the resulting embodiments are also within the protection scope of the invention.

[0028] The implementation process of this solution will be explained below through specific embodiments.

[0029] like Figure 1This is a flowchart illustrating a video encoding acceleration method based on a heterogeneous architecture provided in an embodiment of this disclosure. It is applied to video processing systems that include hardware encoders and software encoders, such as... Figure 1 As shown, the method includes: Step 101: The hardware encoder performs rate-distortion optimization processing on the input video data to generate mode decision data containing block-level coding information.

[0030] It should be noted that the video processing system mentioned here refers to a computing platform that includes at least one hardware encoder and one software encoder. The hardware encoder refers to an encoding device implemented based on an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a graphics processing unit (GPU), or a system-on-a-chip (SoC). It offers high encoding speed and low power consumption but relatively low flexibility. The software encoder refers to an encoding program running on a general-purpose processor (CPU). It offers flexible encoding strategies and controllable quality but has high computational complexity. Rate-distortion optimization refers to the process of selecting the optimal encoding mode during video encoding by comparing the distortion and bitrate costs of different encoding modes.

[0031] It should be further explained that the physical topology of the video processing system is as follows: the hardware encoder is integrated into the hardware acceleration island of the system-on-a-chip (SoC), while the software encoder runs on the application processor cluster of the SoC. The two can communicate via a network-on-chip (NoC). During transmission, the pattern decision data undergoes Quality of Service (QoS) scheduling through the NoC's virtual channel, ensuring that the transmission priority of the pattern decision data is higher than that of ordinary memory access requests. This guarantees that the software encoder can obtain the hardware-generated pattern decision data with low latency before starting the pattern decision search.

[0032] Furthermore, the shared storage region specifically refers to a contiguous set of pre-allocated physical page frames in the system's main memory. These page frames are mapped to the hardware encoder's device address space via the Input / Output Memory Management Unit (IOMMU). The hardware encoder can access the shared storage region directly through the device's physical address without going through the operating system kernel, thus avoiding the latency overhead caused by context switching.

[0033] In existing video encoding workflows, the rate-distortion optimization module within the hardware encoder divides the input video data into multiple coding blocks. For each coding block, it attempts various prediction modes (such as intra-frame prediction and inter-frame prediction) and partitioning structures, calculating the rate-distortion cost for each. Typically, after selecting the optimal mode, the hardware encoder only retains the final decision result used to generate the bitstream, discarding intermediate data such as detailed cost information and candidate mode information. In this embodiment, the hardware encoder is configured to retain this intermediate data and encapsulate it as mode decision data. This mode decision data includes block-level coding information, such as at least one of the partitioning structure, prediction mode information, motion vector information, and rate-distortion cost for each coding block. This information serves as the basis for the hardware encoder to determine the optimal coding strategy for the current video content.

[0034] Step 102: Export the mode decision data from the hardware encoder to a storage area accessible to the software encoder via a dedicated export interface independent of the compressed bitstream output.

[0035] It should be further explained that the dedicated export interface is implemented at the hardware level as a set of independent register mapping regions and DMA descriptor linked lists. Specifically, the hardware encoder internally sets up a Mode Decision Buffer (MDB), the physical address of which is exposed to the software encoder through the Base Address Register (BAR) of the PCIe configuration space. After the hardware encoder completes the mode decision calculation for a frame of video, its internal DMA controller automatically reads the data in the MDB and directly moves the data to the pre-allocated shared memory area of ​​the software encoder through a memory write transaction on the PCIe bus. This transmission process is independent of the compressed bitstream output channel. Specifically, the compressed bitstream is transmitted through the output FIFO queue specified by the video coding standard, while the mode decision data is transmitted through a non-posted write transaction on the PCIe bus. The two are completely separated on the physical signal lines and logical protocol stack.

[0036] Furthermore, the dedicated export interface may also include a Status Flag Register, which contains three bits: a Data_Ready bit, a Transfer_Error bit, and a Data_Valid bit. Before reading the mode decision data, the software encoder first polls the Data_Ready bit of the Status Flag Register. The DMA read operation is only initiated when this bit is set to 1 by the hardware encoder, thereby avoiding data races and reading dirty data.

[0037] Existing hardware encoders typically provide only one output channel for transmitting the encoded compressed bitstream. This compressed bitstream is a binary data stream conforming to video coding standards (such as H.264, H.265, AV1, etc.), dedicated to the decoder's video reconstruction. Embedding mode decision data into the compressed bitstream increases the bitstream's overhead and may affect standard compatibility. Therefore, this embodiment designs a dedicated export interface, which is physically or logically separated from the output path of the compressed bitstream. Through this dedicated export interface, the hardware encoder can directly write mode decision data to a storage area. This storage area can be a pre-allocated buffer in the system's main memory, a shared memory area between the CPU and the hardware encoder, or a device memory area mapped via the PCIe bus. This independent export mechanism ensures that the transmission of mode decision data does not interfere with the normal output of the compressed bitstream, while also guaranteeing that the software encoder can obtain the mode decision data with low latency.

[0038] Step 103: The software encoder retrieves mode decision data from the storage area and uses the mode decision data to accelerate the encoding of the input video data during the mode decision step in the video encoding process.

[0039] When encoding the same input video data, a software encoder would typically need to perform a time-consuming mode decision step from scratch, traversing a large number of candidate modes to find the optimal solution. In this embodiment, the software encoder first reads the mode decision data already generated by the hardware encoder from the storage area. Since the hardware encoder has already completed complex rate-distortion optimization calculations, the mode decision data contains prior knowledge about the optimal partition and motion vectors of the current video block. The software encoder uses this mode decision data as a reference when performing the mode decision step. For example, the software encoder can directly adopt the partition structure determined by the hardware, or use the motion vectors provided by the hardware as the search center for local refinement, thereby significantly narrowing the search range or skipping some calculation steps. This reuse mechanism eliminates the need for the software encoder to repeatedly perform the complete rate-distortion optimization search, significantly reducing computational complexity and thus accelerating the encoding of the input video data.

[0040] Based on the publicly available solutions described above, by reusing the rate-distortion optimization results already completed by the hardware encoder, redundant calculations in the software encoder are eliminated, significantly improving the overall encoding throughput of the heterogeneous system. Encoding speed can be increased while maintaining essentially the same video quality. Furthermore, a dedicated export interface, independent of the compressed bitstream output mode decision data, ensures that the output video bitstream conforms to standard specifications, does not affect compatibility with subsequent decoding and playback, and enables flexible transmission of internal acceleration data. By fully utilizing the high-speed computing power of the hardware encoder and the high-quality control capabilities of the software encoder, complementary advantages are achieved in the heterogeneous architecture, reducing the overall system power consumption and computing resource consumption.

[0041] In one or more embodiments of this disclosure, such as Figure 2 This is a flowchart illustrating the pattern decision data generation method provided in an embodiment of this disclosure. Figure 2 As shown, the hardware encoder performs rate-distortion optimization on the input video data to generate mode decision data containing block-level coding information, including: Step 201: Dividing the input video data into multiple coding blocks. Step 202: Performing rate-distortion optimization calculation on each coding block to obtain the corresponding rate-distortion cost. Step 203: Determining the block partitioning structure, predicted mode information, and motion vector information of each coding block based on the rate-distortion cost. Step 204: Generating mode decision data using at least one of the block partitioning structure, predicted mode information, motion vector information, and rate-distortion cost.

[0042] In practical applications, after receiving input video data, the hardware encoder divides the video frames into several coding blocks according to preset division rules. For example, when processing 4K resolution video, the frame can first be divided into multiple 64x64 pixel coding tree units, and then further divided into smaller coding blocks according to the complexity of the content (the specific size can be set as needed).

[0043] Next, rate-distortion optimization calculations are performed on each coding block to obtain the corresponding rate-distortion cost. The rate-distortion optimization module inside the hardware encoder tries multiple candidate partitioning methods and prediction modes for each coding block. For each candidate method, the module calculates its corresponding distortion (such as residual energy) and estimated bit rate, thereby calculating the rate-distortion cost. The aforementioned process is the most computationally intensive part of video coding. Traditional hardware encoders usually only retain the optimal result after completing this calculation, while this embodiment requires retaining the key outputs of the calculation process.

[0044] It should be further explained that the rate-distortion optimization calculation unit specifically includes multiple Mode Evaluation Engines (MEEs) arranged in parallel. Each MEE contains: a residual calculation subunit, used to calculate the difference between the original pixel and the predicted pixel; a transform quantization subunit, used to perform discrete cosine transform and quantization on the residual; an entropy coding estimation subunit, used to estimate the number of bits after coding; and a distortion metric subunit, used to calculate the structural similarity index SSIM or peak signal-to-noise ratio PSN.

[0045] Furthermore, the mode decision data buffer employs a double buffering mechanism, comprising a first buffer and a second buffer. While the hardware encoder is writing the current frame's mode decision data to the first buffer, the software encoder simultaneously reads the previous frame's mode decision data from the second buffer. The two buffers are synchronized via a buffer switching signal (Buffer_Swap_Signal), which is automatically triggered by the hardware encoder after completing the mode decision calculation for a frame, thereby avoiding read / write conflicts and achieving pipelined encoding acceleration.

[0046] Furthermore, the data structure of the pattern decision data can be specifically defined as follows: typedef struct { uint32_t block_position : 16; / / Encoded block position index uint32_t partition_type : 4; / / Partition type encoding uint32_t prediction_mode : 6; / / Prediction mode index uint32_t mv_x : 12; / / Motion vector X component (1 / 4 pixel precision) uint32_t mv_y : 12; / / Motion vector Y component (1 / 4 pixel precision) uint32_t rd_cost : 16; / / Rate-distortion cost (fixed-point representation) uint32_t skip_flag : 1; / / Skip flag uint32_t reserved : 15; / / Reserved bits ModeDecisionData_t; The total length of the data structure is 64 bits, which is aligned with the data bus width of the hardware encoder, ensuring that the mode decision data transmission of an entire encoding block can be completed in a single memory transaction.

[0047] Then, based on the rate-distortion cost, the block partitioning structure, prediction mode information, and motion vector information for each coding block are determined. The hardware encoder compares the rate-distortion costs of different candidate methods and selects the method with the lowest cost as the optimal decision. For example, if the inter-frame prediction mode under a certain partitioning structure has the lowest rate-distortion cost, then the block partitioning structure of that coding block is determined to be that partitioning structure, the prediction mode information is determined to be that inter-frame prediction mode, and the corresponding motion vector information is recorded. This ensures that the mode decision data contains the optimal coding strategy obtained by the hardware encoder after complex calculations.

[0048] Furthermore, pattern decision data is generated using at least one of the following: block partitioning structure, predicted pattern information, motion vector information, and rate-distortion cost. The hardware encoder encapsulates the determined information into pattern decision data. The content of the pattern decision data can be customized based on actual bandwidth or storage limitations, allowing for the export of all information or only key information. For example, in some scenarios, exporting only motion vector information and block partitioning structure may provide sufficient acceleration cues for the software encoder, without needing to export the entire rate-distortion cost.

[0049] As can be seen from the above-disclosed solutions, traditional hardware encoders often only output the final compressed bitstream, losing intermediate decision data such as block partitioning structure, prediction mode information, and motion vector information, making it impossible for software encoders to reuse them. This embodiment retains and generates mode decision data containing this key information, and also includes rate-distortion cost as one of the generateable contents. This allows the software encoder to not only obtain the hardware's decision results, but also the basis for the reliability of the decisions (e.g., by comparing intra-frame and inter-frame rate-distortion costs), thereby more intelligently determining the reuse strategy. That is, when reusing across codecs, the software encoder can evaluate the reliability of the hardware decisions based on the rate-distortion cost, further optimizing the acceleration effect.

[0050] In one or more embodiments of this disclosure, such as Figure 3 This is a flowchart illustrating the encoding acceleration method provided in an embodiment of this disclosure. Figure 3 As shown, the mode decision data also includes a skip flag. Encoding acceleration of the input video data using the mode decision data includes: Step 301: Determining whether the skip flag in the mode decision data is set. Step 302: When the skip flag is set, the software encoder skips the rate-distortion optimization search process for the coding block.

[0051] In practical applications, when generating mode decision data, the hardware encoder not only determines the block partitioning structure, predicted mode information, and motion vector information, but also generates a skip flag. When performing rate-distortion optimization calculations, if the hardware encoder finds that the residual energy of a certain coding block is close to zero and the rate-distortion cost is below a preset threshold, it determines that the coding block is suitable for the skip mode and sets the skip flag in the mode decision data to the set state. This ensures that the skip flag reflects the high-confidence decision made by the hardware encoder after precise calculation.

[0052] It should be further explained that the determination logic of the skip flag bit is specifically implemented by the skip decision circuit inside the hardware encoder. This circuit includes: a residual energy comparator, used to compare the SAD (Sum of Absolute Differences) value of the current coding block with a first threshold; a motion vector zero detector, used to detect whether the motion vector is a zero vector; and a rate-distortion cost comparator, used to compare the rate-distortion cost of the current coding block with a second threshold.

[0053] Furthermore, the formula for determining whether to skip the decision circuit is: Skip_Flag=(SAD <TH_SAD)AND(MV_X==0)AND(MV_Y==0)AND(RD_Cost<TH_RD); TH_SAD is the residual energy threshold, which is dynamically adjusted according to the video resolution: for 1080p video, TH_SAD is set to 256; for 4K video, TH_SAD is set to 1024. TH_RD is the rate-distortion cost threshold, which is dynamically adjusted according to the quantization parameter QP: TH_RD = Base_TH_RD × (1 + QP / 51), where Base_TH_RD is the baseline threshold.

[0054] Furthermore, when the skip flag is set, the hardware encoder simultaneously generates a confidence score, which is an 8-bit unsigned integer ranging from 0 to 255. The confidence score is calculated as: Confidence = 255 - (SAD / TH_SAD × 128 + RD_Cost / TH_RD × 127). After reading the skip flag, the software encoder further reads the confidence score. It only performs the skip operation if the confidence score is greater than a preset threshold (e.g., 200); otherwise, it reverts to a constraint search strategy.

[0055] Next, when the software encoder accelerates the encoding of the input video data using the pattern decision data, it checks whether the skip flag in the pattern decision data is set. After reading the pattern decision data, the software encoder parses the skip flag. If the skip flag is not set, the software encoder performs the rate-distortion optimization search process according to the normal procedure, that is, iterates through candidate patterns to determine the optimal encoding strategy. At this time, the pattern decision data can be used as a reference to assist the search, but does not directly determine the result.

[0056] When the skip flag is set, the software encoder skips the rate-distortion optimization search process for the encoded block. This means the software encoder directly trusts the hardware encoder's decision, determining the encoding mode of the block as a skip mode, without performing additional mode traversal and cost calculations. Even if the hardware encoder internally determines a block is in a skip mode, the software encoder cannot know this information and still needs to re-execute the complete rate-distortion optimization search process, resulting in a significant waste of computational resources. This embodiment, by deriving and utilizing the skip flag, enables the software encoder to recognize and trust the hardware's skip decision, thereby directly skipping the most time-consuming search step.

[0057] Based on the publicly available solutions described above, by introducing a skip flag as a high-confidence decision cue, the software encoder can directly skip the computationally intensive rate-distortion optimization search process for specific coding blocks, thereby achieving maximum encoding acceleration. This significantly reduces the computational complexity of the software encoder, especially in scenarios where the video content is static or has minimal motion, where the proportion of skipped modes is high. By skipping the rate-distortion optimization search process, the CPU usage of the software encoder can be significantly reduced, increasing the encoding frame rate. Furthermore, since the skip flag is generated based on the rate-distortion cost of the hardware encoder, it has high reliability. Therefore, directly skipping the search process does not lead to a significant decrease in video quality, achieving a good balance between speed and quality. In other words, in the overall reuse strategy, near-zero computational cost encoding is achieved for skipped blocks, improving acceleration efficiency under heterogeneous architectures.

[0058] In one or more embodiments of this disclosure, such as Figure 4 This is a flowchart illustrating the pattern decision data export method provided in an embodiment of this disclosure. Figure 4 As shown, mode decision data is exported from the hardware encoder to a storage area accessible by the software encoder via a dedicated export interface independent of the compressed bitstream output. This includes: Step 401: Determining the data transmission channel type between the hardware encoder and the software encoder. Step 402: If the data transmission channel type is a direct memory access channel, a shared memory interface, or a system bus bypass interface, then enabling the dedicated export interface. Step 403: Directly writing the mode decision data to the storage area via the enabled dedicated export interface.

[0059] In practical applications, during the initialization phase, the driver for a hardware encoder or video processing system detects the data transmission capabilities supported by the current hardware platform. For example, the system queries information such as bus type and memory mapping configuration to determine the types of data transmission channels currently available.

[0060] Next, if the data transmission channel type is a direct memory access channel, a shared memory interface, or a system bus bypass interface, the dedicated export interface is enabled. Specifically, the detected data transmission channel type is compared with a preset list of supported types. If the data transmission channel type belongs to any of the direct memory access channel, shared memory interface, or system bus bypass interface, the condition is met, and the system sends a control signal to the hardware encoder to activate the logic circuit or register configuration of the dedicated export interface, preparing it for data export. If the data transmission channel type does not belong to any of the above types, the dedicated export interface is not enabled, and the system may use other alternative transmission methods or terminate the current export process. This ensures that the dedicated export interface is only activated on channels with high-speed transmission capabilities, guaranteeing the reliability of data transmission.

[0061] It should be further explained that the activation process of the dedicated export interface specifically includes the following register configuration sequence: The first step is for the software encoder to access the control register group of the hardware encoder through the PCIe configuration space and set bit 0 of the export enable register (Export_Enable_Reg) to 1.

[0062] The second step involves the software encoder writing the physical address of the shared memory region into the base address register (Base_Address_Reg), which is 64 bits wide and supports up to 48 bits of physical address addressing.

[0063] The third step involves the software encoder writing the size of the pattern decision data into the data length register (Data_Length_Reg), which is in bytes.

[0064] Fourth, the software encoder triggers the Start_Transfer_Reg register, which is achieved by writing an arbitrary value. Once the hardware encoder detects that the Start_Transfer_Reg register has been written, it starts the DMA controller to begin data transfer.

[0065] Fifth, after the hardware encoder completes the data transmission, it sends an MSI (Message Signaled Interrupt) signal to the interrupt controller. The software encoder reads the status register in the interrupt service routine to confirm that the transmission is complete.

[0066] Furthermore, the register configuration sequence is executed only once during the system initialization phase. Subsequent frame mode decision data transmission reuses the configured register values, updating only the data length register and triggering the start transmission register, thereby reducing register configuration overhead.

[0067] The mode decision data is written directly to the storage area via a dedicated export interface. Once the dedicated export interface is enabled, the hardware encoder packages the generated mode decision data and moves it directly to the storage area accessible to the software encoder via a direct memory access channel, shared memory interface, or system bus bypass interface. This eliminates the need for encoding and encapsulation paths in the compressed bitstream, achieving physical separation between the data stream and the bitstream. Direct write operations are typically performed automatically by the hardware controller, eliminating the need for byte-by-byte copying by the central processing unit, thus reducing transmission latency.

[0068] Based on the publicly available solutions described above, high-speed data transmission is achieved by utilizing direct memory access channels, shared memory interfaces, or system bus bypass interfaces as dedicated export interfaces, avoiding performance bottlenecks caused by central processing unit (CPU) relay. Simultaneously, the mechanism of determining the channel type before activation enhances system adaptability, ensuring correct establishment of export paths across different hardware platforms. Independent of the compressed bitstream transmission method, the output video bitstream conforms to standard specifications, does not affect subsequent decoding and playback, and achieves both encoding acceleration and standard compatibility.

[0069] In one or more embodiments of this disclosure, such as Figure 5 This is a schematic flowchart illustrating the software encoder encoding acceleration method provided in an embodiment of this disclosure. Figure 5 As shown, the software encoder retrieves mode decision data from the storage area and utilizes this data to accelerate the encoding of the input video data during the mode decision step in the video encoding process. This includes: Step 501: In response to the software encoder initiating a mode decision search, determining whether mode decision data exists in the storage area. Step 502: If mode decision data exists, reading it as a reference for mode decision-making. Step 503: Reducing the search range for rate distortion optimization based on the reference data; or, skipping some calculation steps to accelerate the encoding of the input video data.

[0070] In practical applications, in response to the software encoder initiating a mode decision search, the system determines whether mode decision data exists in the storage area. Specifically, before starting to encode the current block, the software encoder triggers the mode decision search process. At this time, the software encoder queries the status flags or pointers in the storage area to confirm whether the hardware encoder has completed the export and writing of mode decision data. If mode decision data does not exist in the storage area, for example, due to hardware export delay or the current frame not having export enabled, the software encoder performs a complete mode decision search according to the normal process, without relying on external data, ensuring that the encoding task can be completed normally. If mode decision data exists in the storage area, the subsequent acceleration process begins.

[0071] Furthermore, if pattern decision data exists, it is read as a reference for pattern decision-making. The software encoder loads the pattern decision data from the storage area and parses the block-level encoded information within it. At this point, the pattern decision data is no longer merely static information, but is transformed into a reference for the software encoder's internal decision-making logic. For example, the motion vectors provided by the hardware can serve as the initial center for the software search, and the partition structure determined by the hardware can serve as the primary candidate for software traversal.

[0072] Furthermore, the search range can be reduced based on reference criteria to optimize rate distortion; or, some computational steps can be skipped to accelerate encoding of the input video data. Specifically, the first path is to reduce the search range, for example, by shrinking the motion estimation search window from a large area to a small area centered on the motion vectors provided by the hardware, thereby reducing the number of candidate point traversals. The second path is to skip some computational steps, for example, if the reference criteria indicate that a certain mode has an extremely low cost, then the cost calculation for other candidate modes of that mode can be skipped directly. These two paths can be used individually or in combination, with the ultimate goal of reducing the computational load on the software encoder and accelerating encoding of the input video data.

[0073] Based on the publicly available solutions described above, by utilizing pattern decision data in the storage area as a reference, the software encoder can perform searches according to the actual situation, significantly reducing invalid computations. This on-demand acceleration method based on data existence judgment can still degrade the system to ensure compatibility when hardware export anomalies occur, while maximizing acceleration benefits when the data is normal, thus improving the stability and efficiency of heterogeneous systems.

[0074] In one or more embodiments of this disclosure, the search range for rate-distortion optimization is narrowed based on reference criteria; or, some calculation steps are skipped, including any of the following strategies: a constrained search strategy, which uses pattern decision data as a high-probability candidate and performs rate-distortion optimization search within a local range; a full trust strategy, which directly uses pattern decision data as the final encoding decision and skips the rate-distortion optimization search of the software encoder; and a hint strategy, which uses pattern decision data as an initial state seed and performs a complete rate-distortion optimization search.

[0075] If the application scenario has extremely high real-time requirements, such as live streaming, then choose the full trust strategy to maximize speed; if the application scenario has high quality requirements and allows for some computational overhead, such as offline transcoding, then choose the hint strategy to maximize quality; if a balance needs to be struck between speed and quality, then choose the constrained search strategy. If not explicitly configured, the system will default to the constrained search strategy.

[0076] It should be further noted that the dynamic switching between the three strategies is automatically executed by the StrategySelector. The inputs to the StrategySelector include: current system CPU load rate, network bandwidth utilization, video content motion intensity, and encoding latency constraints. The output of the StrategySelector is a strategy selection signal, which controls the software encoder to adopt a constraint search strategy, a fully trusted strategy, or a cueing strategy.

[0077] Furthermore, the switching logic of the strategy selector is specifically as follows: When CPU load is >80% and encoding latency constraint is <33ms (i.e., frame rate >30fps), select the full trust strategy; When the CPU load rate is less than 50% and the motion intensity of the video content is greater than the preset motion threshold, select the prompting strategy; In other cases, choose a constrained search strategy.

[0078] Furthermore, the formula for calculating the motion intensity of the video content is: Motion_Intensity = Average(|MV_X| + |MV_Y|) / Block_Count, where Average represents the average of the absolute values ​​of the motion vectors of all coded blocks in the current frame, and Block_Count is the total number of coded blocks in the current frame. When Motion_Intensity > 16 (in pixels), it is determined to be high motion intensity.

[0079] Furthermore, the policy switching is triggered at the GOP (Group of Pictures) boundary or scene switching detection signal. Scene switching detection is achieved by comparing the histogram difference between the current frame and the previous frame. When the histogram difference exceeds a preset threshold, it is determined to be a scene switch, and at this time, the policy is forcibly switched to the cue policy to ensure the encoding quality of the first frame after the scene switch.

[0080] If a constrained search strategy is chosen, the software encoder uses the pattern decision data as high-probability candidates and performs rate-distortion optimization search within a local range. This means the software encoder does not completely ignore its own search process, but focuses its search on patterns recommended by the hardware, thus narrowing the search space. If a full trust strategy is chosen, the software encoder directly uses the pattern decision data as the final encoding decision, skipping the software encoder's rate-distortion optimization search. This means the software encoder completely trusts the hardware's decision results and does not perform any additional pattern comparisons, thus achieving maximum speedup. If a hint strategy is chosen, the software encoder uses the pattern decision data as the initial state seed and performs a complete rate-distortion optimization search. This means the software encoder uses hardware data as the starting point for the search, but still traverses the entire candidate set to ensure that no potential optimal solutions are missed.

[0081] Based on the publicly available solutions described above, the optional configuration of constraint search strategy, full trust strategy, and prompting strategy enables the video processing system to adapt to diverse business needs. Within the same heterogeneous architecture, different performance characteristics can be achieved through software configuration without altering the hardware design, thus improving the system's applicability.

[0082] In one or more embodiments of this disclosure, such as Figure 6 This is a flowchart illustrating the constraint search strategy execution method provided in an embodiment of this disclosure. Figure 6 As shown, the execution method of the constraint search strategy includes: Step 601: If the pattern decision data contains motion vector information, the motion vector information in the pattern decision data is used as the center point of the motion estimation search, and the search range is limited to a preset pixel range around the center point. Step 602: The block partitioning structure in the pattern decision data is used as the primary candidate for rate-distortion optimization search, and the alternative structures with a partitioning granularity one level different from the primary candidate are evaluated.

[0083] In practical applications, the software encoder parses the mode decision data to determine whether it contains motion vector information. If the mode decision data contains motion vector information, an acceleration process is executed. Specifically, the motion vector information in the mode decision data is used as the center point for motion estimation search, and the search range is limited to a preset pixel range around the center point. For example, if the motion vector provided by the hardware points to coordinates (x, y), the software encoder only searches within the range of (x±4, y±4), rather than searching the entire frame. If the mode decision data does not contain motion vector information, such as when the current block is in intra-frame prediction mode, the software encoder does not execute motion vector-based search constraints, but instead performs motion estimation search according to the default full range or standard range to ensure the integrity of the encoding process.

[0084] It should be further explained that the value of the preset pixel range is dynamically adjusted according to the quantization parameter QP of the coding block. The specific adjustment formula is: Search_Range = Base_Range × (1 + QP / 51), where Base_Range is the baseline search range, which is 4 pixels. When QP is large (e.g., QP > 30), the search range expands to 8 pixels; when QP is small (e.g., QP < 15), the search range shrinks to 2 pixels. This dynamic adjustment mechanism is based on the following technical principle: when QP is large, the video quality requirement is low, so the search range can be expanded to capture greater motion; when QP is small, the video quality requirement is high, so the search range needs to be narrowed to avoid introducing noise.

[0085] Furthermore, the alternative structures with a partition granularity one level different from the primary candidate specifically include: when the primary candidate is a 64x64 partition, the alternative structure is a 32x32 partition; when the primary candidate is a 32x32 partition, the alternative structures are 64x64 partitions and 16x16 partitions; when the primary candidate is a 16x16 partition, the alternative structures are 32x32 partitions and 8x8 partitions. This hierarchical evaluation strategy ensures local optimization within the neighborhood of the hardware decision, avoiding the computational overhead of global search.

[0086] Furthermore, under the constrained search strategy, the software encoder sets the upper limit of the search time for each coding block to T_max = T_base × (1 + Motion_Intensity / 32), where T_base is the baseline search time, which is 100 microseconds. When the search time reaches T_max, the search is forcibly terminated and the current best candidate is adopted as the final coding decision, thereby ensuring the real-time performance of the coding process.

[0087] Furthermore, the software encoder uses the block partitioning structure in the pattern decision data as the primary candidate for rate-distortion optimization search. When traversing partitioning patterns, the hardware-determined partitioning structure is calculated and evaluated first. Subsequently, the software encoder evaluates candidate structures with a partitioning granularity one level different from the primary candidate. For example, if the primary candidate is a 32x32 partitioning structure, the software encoder only additionally evaluates 64x64 (one level coarser) or 16x16 (one level finer) candidate structures, skipping other partitioning structures that are many levels different or irrelevant. Utilizing the high confidence of the hardware partitioning decision, the search focus is concentrated on the neighborhood near the optimal solution, avoiding traversing all possible partitioning levels.

[0088] Based on the aforementioned publicly available solutions, by using the motion vector information provided by the hardware as the center point and limiting the preset pixel range, the complexity of motion estimation is reduced from being proportional to the search window area to being proportional to a local small window. Simultaneously, by evaluating only candidate structures differing by one level, the number of candidates for partitioning decisions is significantly reduced. With only a minimal loss in coding efficiency, an order-of-magnitude improvement in rate-distortion optimization search speed is achieved, making it suitable for latency-sensitive practical applications.

[0089] In one or more embodiments of this disclosure, such as Figure 7 This is a flowchart illustrating the pattern decision data generation method provided in an embodiment of this disclosure. Figure 7 As shown, generating mode decision data containing block-level coding information includes: Step 701: Mapping at least one of the block partitioning structure, prediction mode information, motion vector information, and rate-distortion cost to a common format that supports interoperability between the first video coding standard and the second video coding standard. Step 702: Encapsulating the mode decision data in a codec-decoding-independent common data structure.

[0090] In practical applications, at least one of the following—block partitioning structure, prediction mode information, motion vector information, and rate-distortion cost—is mapped to a universal format that supports interoperability between the first and second video coding standards. When generating mode decision data, the hardware encoder does not directly output its internal, proprietary, first-video coding standard-specific raw data; instead, it performs a translation or normalization process. For example, if the motion vector precision in the first video coding standard is 1 / 4 pixel, while the second video coding standard supports 1 / 8 pixel, it is mapped to a unified universal format, such as being uniformly normalized to 1 / 4 pixel precision. Similarly, the complex intra-frame prediction mode index in the first video coding standard is mapped to a simplified mode identifier in a universal format. The mapping process can select key information to map based on actual needs. If only motion information needs to be reused, only motion vector information is mapped; if partitioning decisions need to be reused, the block partitioning structure is mapped. Through this mapping, data originally specific to the first video coding standard is converted into a neutral universal format, thus providing a basis for interoperability with the second video coding standard.

[0091] After mapping is complete, the hardware encoder fills the obtained generic format data into a predefined codec-independent generic data structure. In one specific embodiment, the generic data structure (omd_block_t) includes the following fields: - block_x, block_y: Represents the top-left corner coordinates of the coded block; - skip_flag: A 1-bit flag, set to 1 to indicate a suggested skip; - mv_x, mv_y: Motion vector components normalized to 1 / 4 pixel precision; - partition_type: An enumeration value, 0 indicates no partitioning, 1 indicates horizontal partitioning, and 2 indicates vertical partitioning; - intra_mode: The normalized intra-frame mode index (0-3 represent DC, Planar, Vertical, and Horizontal, respectively). The field definitions of this data structure are fixed and do not change with changes in the first or second video coding standard. For example, the structure always includes a field for storing normalized motion vectors, and the meaning of this field remains unchanged regardless of changes in the underlying standard. Once encapsulated, the generated pattern decision data will have a universal parsing interface. The software encoder does not need to care about the specific standard adopted by the hardware encoder; it only needs to parse according to the definition of the universal data structure to obtain the required universal format data.

[0092] Based on the publicly available solutions described above, it is evident that the barriers between standards are broken down through a universal format and a codec-independent universal data structure. By leveraging the consistent spatiotemporal correlation of video content across different coding standards, efficient acceleration between heterogeneous standards is achieved, expanding the application scenarios and flexibility of video processing systems.

[0093] In one or more embodiments of this disclosure, such as Figure 8 This is a flowchart illustrating the general format mapping method provided in the embodiments of this disclosure. Figure 8 As shown, the mapping is a universal format supporting interoperability between the first video coding standard and the second video coding standard, including: Step 801: When the hardware encoder generates mode decision data, the internal mode identifier of the first video coding standard is converted into a universal mode identifier in a universal data structure. Step 802: When the software encoder uses the mode decision data, the universal mode identifier in the universal data structure is mapped to the mode identifier of the second video coding standard; wherein, the universal mode identifier is an intermediate representation format between the first video coding standard and the second video coding standard.

[0094] In practical applications, when the hardware encoder generates mode decision data, the internal mode identifiers of the first video coding standard are converted into general mode identifiers in a general data structure. The hardware encoder maintains a first mapping table that establishes the correspondence between the internal mode identifiers of the first video coding standard and the general mode identifiers. This mapping is a general format supporting interoperability between the first and second video coding standards, specifically including: establishing a first mapping table that maps the 67 intra-frame mode indices of the H.265 standard to the aforementioned four general mode indices (DC, Planar, Vertical, and Horizontal); and establishing a second mapping table that maps the 56 intra-frame mode indices of the AV1 standard to the aforementioned four general mode indices. When the hardware encoder determines the optimal prediction mode for a coding block, it queries the first mapping table. If a corresponding mapping entry exists for the internal mode identifier in the first mapping table, it is converted to the corresponding general mode identifier and filled into the general data structure. If no corresponding mapping entry exists for the internal mode identifier in the first mapping table, for example, if a private or undefined mode is encountered, it is converted to a default general mode identifier, such as a general intra-frame or general inter-frame identifier, to ensure data integrity. Ensure that the exported data is standardized and does not carry proprietary features of the first video coding standard.

[0095] Furthermore, when the software encoder utilizes mode decision data, it maps the general mode identifiers in the general data structure to mode identifiers of the second video coding standard. The software encoder internally maintains a second mapping table that establishes the correspondence between general mode identifiers and mode identifiers of the second video coding standard. When the software encoder parses the general data structure and reads the general mode identifier, it queries the second mapping table. If a valid mapping target exists for the general mode identifier in the second mapping table, it is converted into a mode identifier of the second video coding standard to guide subsequent encoding. If a valid mapping target does not exist for the general mode identifier in the second mapping table, the software encoder ignores the mode identifier or uses the default mode for encoding. This ensures that standardized data can be correctly converted into instructions understandable by the software encoder.

[0096] The universal mode identifier serves as an intermediate representation format between the first and second video coding standards. This means that the universal mode identifier is neither a proprietary format of the first nor the second video coding standard, but rather an independent intermediate state. For example, regardless of whether the first video coding standard is H.265 or H.264, its DC prediction mode can be converted to the same universal mode identifier; similarly, regardless of whether the second video coding standard is AV1 or VVC, it can be mapped back to its respective DC prediction mode from the same universal mode identifier.

[0097] Based on the publicly available solutions described above, mode reuse between heterogeneous codecs is achieved, overcoming the limitation in existing technologies where hardware and software must use the same encoding standard to reuse data. Through an intermediate representation format, the hardware encoder does not need to adapt to the software encoder's standard, significantly reducing the complexity of system integration. This bidirectional mapping method, meaning that on the same hardware platform, new encoding standards can be supported by replacing the software encoder without modifying the hardware design, significantly improves the lifespan and compatibility of the video processing system.

[0098] Based on the same idea, this application also proposes a video coding acceleration device based on a heterogeneous architecture. For example... Figure 9 This is a schematic diagram of a video encoding acceleration device based on a heterogeneous architecture, as proposed in an embodiment of this application. It is applied to a video processing system that includes a hardware encoder and a software encoder. Figure 9As can be seen, the device includes: a generation module 91, used to perform rate-distortion optimization processing on the input video data using a hardware encoder to generate mode decision data containing block-level coding information; an export module 92, used to export the mode decision data from the hardware encoder to a storage area accessible to the software encoder via a dedicated export interface independent of the compressed bitstream output; and an encoding module 93, used by the software encoder to retrieve the mode decision data from the storage area and to accelerate the encoding of the input video data using the mode decision data during the mode decision step in the video encoding process.

[0099] The generation module 91 is used to divide the input video data into multiple coding blocks; perform rate-distortion optimization calculation on each coding block to obtain the corresponding rate-distortion cost; determine the block partitioning structure, prediction mode information and motion vector information of each coding block based on the rate-distortion cost; and generate mode decision data using at least one of the block partitioning structure, prediction mode information, motion vector information and rate-distortion cost.

[0100] The pattern decision data also includes a skip flag; the generation module 91 is used to determine whether the skip flag in the pattern decision data is set; when the skip flag is set, the software encoder skips the rate-distortion optimization search process for the coding block.

[0101] Export module 92 is used to determine the data transmission channel type between the hardware encoder and the software encoder; if the data transmission channel type is a direct memory access channel, a shared memory interface, or a system bus bypass interface, a dedicated export interface is enabled; through the enabled dedicated export interface, mode decision data is directly written to the storage area.

[0102] Encoding module 93 is used to respond to the software encoder initiating a mode decision search, determine whether mode decision data exists in the storage area; if mode decision data exists, read the mode decision data as a reference for mode decision; reduce the search range for rate distortion optimization based on the reference; or, skip some calculation steps to accelerate the encoding of the input video data.

[0103] Encoding module 93 is used to narrow the search range for rate-distortion optimization based on reference criteria; or to skip some computational steps, including any of the following strategies: a constrained search strategy, which uses the pattern decision data as a high-probability candidate and performs rate-distortion optimization search within a local range; a full trust strategy, which directly uses the pattern decision data as the final encoding decision and skips the rate-distortion optimization search of the software encoder; and a hint strategy, which uses the pattern decision data as an initial state seed and performs a complete rate-distortion optimization search.

[0104] The encoding module 93 is used to, if the pattern decision data contains motion vector information, use the motion vector information in the pattern decision data as the center point of the motion estimation search, and limit the search range to a preset pixel range around the center point; use the block partitioning structure in the pattern decision data as the primary candidate for rate-distortion optimization search, and evaluate the alternative structures whose partitioning granularity is one level different from that of the primary candidate.

[0105] The generation module 91 is used to map at least one of the block partitioning structure, prediction mode information, motion vector information and rate distortion cost into a common format that supports interoperability between the first video coding standard and the second video coding standard; the mode decision data is encapsulated in a codec-decoding-independent common data structure.

[0106] The generation module 91 is used to convert the internal mode identifier of the first video coding standard into a general mode identifier in a general data structure when the hardware encoder generates mode decision data; and to map the general mode identifier in the general data structure to a mode identifier of the second video coding standard when the software encoder uses the mode decision data; wherein, the general mode identifier is an intermediate representation format between the first video coding standard and the second video coding standard.

[0107] Embodiments of this application also propose an electronic device, including a processor and a memory; the memory is used to store a computer program executable by the processor; the processor is used to execute the computer program in the memory to implement the heterogeneous architecture-based video coding acceleration method described in any of the above embodiments.

[0108] Embodiments of this application also propose a computer-readable storage medium that, when an executable computer program in the storage medium is executed by a processor, enables the implementation of the heterogeneous architecture-based video coding acceleration method described in any of the above embodiments.

[0109] Regarding the apparatus in the above embodiments, the specific manner in which the processor performs the operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0110] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment. For example, electronic device 900 may be provided as a server. (Refer to...) Figure 10The device 900 includes a processing component 922, which further includes one or more processors, and memory resources represented by memory 932 for storing instructions, such as application programs, that can be executed by the processing component 922. The application programs stored in memory 932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 922 is configured to execute instructions to perform the aforementioned video coding acceleration method for heterogeneous architectures.

[0111] Device 900 may also include a power supply component 926 configured to perform power management of device 900, a wired or wireless network interface 950 configured to connect device 900 to a network, and an input / output (I / O) interface 958. Device 900 can operate on an operating system stored in memory 932, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or similar.

[0112] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 932 including instructions, which can be executed by a processing component 922 of the device 900 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0113] In this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0114] The above description of the embodiments is intended to enable those skilled in the art to understand and apply this application. It will be apparent to those skilled in the art that various modifications can be easily made to these embodiments, and the general principles described herein can be applied to other embodiments without creative effort. Therefore, this application is not limited to the embodiments described herein, and any improvements and modifications made by those skilled in the art based on the disclosure of this application without departing from the scope and spirit of this application are within the scope of this application.

Claims

1. A video coding acceleration method based on heterogeneous architecture, characterized in that, The method, applied to a video processing system comprising a hardware encoder and a software encoder, includes: The hardware encoder performs rate-distortion optimization processing on the input video data to generate mode decision data containing block-level coding information. The mode decision data is exported from the hardware encoder to a storage area accessible by the software encoder via a dedicated export interface independent of the compressed bitstream output. The software encoder obtains the mode decision data from the storage area and uses the mode decision data to accelerate the encoding of the input video data during the mode decision step in the video encoding process.

2. The video coding acceleration method based on heterogeneous architecture according to claim 1, characterized in that, The hardware encoder performs rate-distortion optimization on the input video data to generate mode decision data containing block-level coding information, including: The input video data is divided into multiple coded blocks; Rate-distortion optimization calculations are performed on each of the coded blocks to obtain the corresponding rate-distortion cost; Based on the rate-distortion cost, determine the block partitioning structure, prediction mode information, and motion vector information for each coding block; The mode decision data is generated using at least one of the block partitioning structure, the prediction mode information, the motion vector information, and the rate-distortion cost.

3. The video coding acceleration method based on heterogeneous architecture according to claim 2, characterized in that, The pattern decision data also includes a skip flag; The acceleration of encoding the input video data using the pattern decision data includes: Determine whether the skip flag bit in the mode decision data is set; When the skip flag is set, the software encoder skips the rate-distortion optimization search process for the encoded block.

4. The video coding acceleration method based on heterogeneous architecture according to claim 1, characterized in that, The step of exporting the mode decision data from the hardware encoder to a storage area accessible by the software encoder via a dedicated export interface independent of the compressed bitstream output includes: Determine the data transmission channel type between the hardware encoder and the software encoder; If the data transmission channel type is a direct memory access channel, a shared memory interface, or a system bus bypass interface, then the dedicated export interface is enabled; The pattern decision data is written directly to the storage area via the enabled dedicated export interface.

5. The video coding acceleration method based on heterogeneous architecture according to claim 1, characterized in that, The software encoder obtains the mode decision data from the storage area and, during the mode decision step of the video encoding process, utilizes the mode decision data to accelerate the encoding of the input video data, including: In response to the software encoder initiating a pattern decision search, it is determined whether the pattern decision data exists in the storage area; If the pattern decision data exists, then the pattern decision data is read as a reference for pattern decision-making; The search range is optimized based on the reference criteria to reduce rate distortion; or, some calculation steps are skipped to accelerate the encoding of the input video data.

6. The video coding acceleration method based on heterogeneous architecture according to claim 5, characterized in that, The search range optimized based on the reduction rate distortion according to the reference criteria; or, skipping some calculation steps, including any of the following strategies: The constrained search strategy uses the pattern decision data as high-probability candidates and performs rate-distortion optimization search within a local range. The full-trust strategy directly uses the pattern decision data as the final encoding decision, skipping the rate-distortion optimization search of the software encoder; The suggested strategy is to use the pattern decision data as the initial state seed and perform a complete rate-distortion optimization search.

7. The video coding acceleration method based on heterogeneous architecture according to claim 6, characterized in that, The execution methods of the constraint search strategy include: If the pattern decision data contains motion vector information, then the motion vector information in the pattern decision data is used as the center point of the motion estimation search, and the search range is limited to a preset pixel range around the center point. The block partitioning structure in the pattern decision data is used as the primary candidate for rate-distortion optimization search, and alternative structures with a partitioning granularity one level different from the primary candidate are evaluated.

8. The video coding acceleration method based on heterogeneous architecture according to claim 1, characterized in that, Generate pattern decision data containing block-level coding information, including: Map at least one of the block partitioning structure, prediction mode information, motion vector information, and rate distortion cost to a common format that supports interoperability between the first video coding standard and the second video coding standard. The pattern decision data is encapsulated in a general data structure that is independent of encoding and decoding.

9. The video coding acceleration method based on heterogeneous architecture according to claim 8, characterized in that, The mapping is a universal format that supports interoperability between the first video coding standard and the second video coding standard, including: When the hardware encoder generates the mode decision data, the internal mode identifier of the first video coding standard is converted into the general mode identifier in the general data structure; When the software encoder uses the mode decision data, it maps the general mode identifier in the general data structure to the mode identifier of the second video coding standard. The general mode identifier is an intermediate representation format between the first video coding standard and the second video coding standard.

10. A video coding acceleration device based on a heterogeneous architecture, characterized in that, An apparatus for use in a video processing system comprising a hardware encoder and a software encoder, the apparatus comprising: The generation module is used to perform rate-distortion optimization processing on the input video data through the hardware encoder to generate mode decision data containing block-level coding information. The export module is used to export the mode decision data from the hardware encoder to a storage area accessible by the software encoder through a dedicated export interface independent of the compressed bitstream output. An encoding module is used to obtain the mode decision data from the storage area through the software encoder, and to accelerate the encoding of the input video data by using the mode decision data during the mode decision step in the video encoding process.

11. An electronic device comprising a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement the method according to any one of claims 1 to 9.

12. A computer-readable medium having stored thereon at least one instruction, at least one program, code set, or instruction set, said at least one instruction, at least one program, code set, or instruction set being loaded and executed by a processor to implement the method according to any one of claims 1 to 9.

Images