HARDWARE VIDEO ENCODER ARCHITECTURE FOR MULTI-LINE PARALLEL CODING

Abstract

Several embodiments include methods for the parallel encoding of multiple lines of a media image. The disclosed video encoder has multiple controllers, each controller encoding components or blocks contained in a different line of a media image. Each controller encodes the respective blocks of the different lines by sending commands and data to different encoding resources, each performing different encoding functions. The controllers have concurrent and independent access to the encoding resources. Therefore, the controllers can access any encoder resource to perform encoding functions, regardless of which other encoder resources are performing encoding functions for other controllers. This increases the utilization of the encoder resources and improves encoding performance compared to conventional methods.

Description

BACKGROUND Area of ​​the different designs

[0001] Different embodiments generally relate to video coding architectures and, in particular, to a hardware video encoder architecture for multi-line parallel coding. Description of the state of the art

[0002] When streaming live or recorded video, a first computer system, such as a server, data center, cloud storage system, and / or the like, transmits a video stream to a second computer system, such as a smartphone, tablet, laptop, and / or the like. Transmitting video streams between computer systems can consume a significant amount of network bandwidth, reducing the bandwidth available for other purposes. Therefore, one goal of computer systems transmitting video streams is to compress and encode the video streams before transmission without significantly affecting the video quality. Computer systems receiving such video streams decompress and decode them before displaying them on one or more display devices.

[0003] When compressing and encoding a video stream, a computer system typically includes a hardware video encoder that divides each media frame in the video stream into blocks, with each block containing a group of adjacent pixels from the media frame. Each block of adjacent pixels can be an 8x8 pixel block, a 16x16 pixel block, and / or the like. Depending on the video format used for encoding, these blocks are referred to as macroblocks, coding tree units (CTUs), and / or the like. The video encoder typically encodes the blocks as a series of lines, with the blocks in each line encoded sequentially from left to right and the lines themselves from top to bottom.

[0004] Typically, video encoders divide each block into multiple partitions. To achieve higher encoding quality when encoding the pixels in each partition, the video encoder can access data such as motion vector data from blocks at similar positions in the previous media frame to generate a prediction known as inter-frame prediction. The video encoder can also access neighboring pixels within the same media frame to generate a prediction known as intra-frame prediction. These neighboring pixels can include pixels from previously encoded partitions, such as partitions to the left and / or partitions above the current partitions. In this way, the prediction for a current partition is based on neighboring pixels from adjacent partitions.Therefore, video encoding generally features sequential encoding of partitions from left to right and from top to bottom in each media frame of a video stream.

[0005] One problem with this approach to video encoding is that, with hardware encoders, sequential video encoding can lead to significant waiting times for various hardware components within the encoder. The sequential encoding of partitions therefore results in underutilization and reduced efficiency of CPUs, GPUs, and / or hardware video encoders. Furthermore, parallel encoding of partitions is not possible because each partition may depend on pixels in partitions to its left and / or above. Consequently, video encoding cannot take advantage of the parallel processing capabilities of modern central processing units (CPUs) and graphics processing units (GPUs).

[0006] One approach to solving this problem is to integrate multiple hardware video encoders into the computer system. With this approach, a first video encoder can encode a first video stream, a second video encoder can encode a second video stream, and so on. A disadvantage of this approach is that additional hardware video encoders can double, triple, or more the integrated circuitry, surface area, and power consumption used for encoding video streams. Another disadvantage of this approach is that while multiple video streams can be encoded in parallel, the encoding speed of each individual video stream is not increased because inter-image dependencies prevent parallel encoding.Furthermore, this approach does not solve efficiency problems related to intra-image dependencies caused by dependencies between partitions in a single encoder.

[0007] As the above shows, more effective methods for encoding video streams in a computer system are needed in technology. SUMMARY

[0008] Various embodiments of the present disclosure define a computer-implemented method for the parallel encoding of multiple block lines in a media image. The method includes encoding a first plurality of blocks contained in a first line of a media image by a first controller. The method further includes encoding a second plurality of blocks contained in a second line of the media image by a second controller. The method further includes accessing a first hardware computing resource by the second controller to perform a first video encoding function. In the disclosed method, the first hardware computing resource can be used simultaneously by the first controller and the second controller.

[0009] Other embodiments include, without limitation, a system implementing one or more aspects of the disclosed methods, and one or more computer-readable media containing instructions for executing one or more aspects of the disclosed methods, as well as a method for executing one or more aspects of the disclosed methods.

[0010] A technical advantage of the disclosed methods over the prior art is that a video encoder using the disclosed methods has multiple line control units that encode multiple lines of blocks in the media image in parallel. Furthermore, various other hardware computing resources included in the video encoder are available in parallel with the line control units. Therefore, each line control unit can access all available functional units of the video encoder without having to wait for other line control units to reach a specific stage of the video encoding process. This allows for improved parallel processing of block lines and the utilization of other functional units of the video encoder compared to previous conventional approaches. These advantages represent one or more technological improvements over prior art approaches. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] To enable a detailed understanding of the nature of the aforementioned features of the various embodiments, a more precise description of the inventive concepts summarized above can be provided by reference to various embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings only illustrate typical embodiments of the concepts according to the invention and are therefore in no way to be considered as limiting the scope, and that other equally effective embodiments exist. Fig. Figure 1 is a block diagram of a computer system designed to implement one or more aspects of the various embodiments; Fig. Figure 2 is a block diagram of a parallel processing unit (PPU) which, according to various embodiments, is located in the accelerator processing subsystem of Fig. 1 is present; Fig. Figure 3 is a block diagram of a general processing cluster (GPC) located in the parallel processing unit (PPU) of Fig. 2 according to different embodiments; Fig. Figure 4 is a block diagram of a video encoder designed to process multiple block lines in parallel for the computer system of the Fig. 1-3 to be coded according to different embodiments; Fig. Figure 5 shows a functional view of a video encoder that converts a media image for the computer system of the Fig. 1-4 can be coded according to different embodiments; Fig. Figure 6 is a block diagram of a video encoder designed for sequential video coding, which is used in the computer system of the Fig. 1 can be implemented according to different embodiments; The Fig. 7A-B illustrate how blocks of a media image are processed by the video encoders of the Fig. 4-6 are coded according to different embodiments; Fig. Figure 8 is a flowchart of process steps for the parallel coding of multiple block lines in the computer system of the Fig. 1-7B according to various embodiments. DETAILED DESCRIPTION

[0012] The following description presents numerous specific details to enable a more thorough understanding of the various embodiments. However, it is clear to those skilled in the art that the concepts according to the invention can also be implemented without one or more of these specific details. System overview

[0013] Fig. Figure 1 is a block diagram of a computer system 100, designed to implement one or more aspects of the various embodiments. As shown, the computer system 100 without restriction has a central processing unit (CPU) 102 and a system memory 104, which is coupled to an accelerator processing subsystem 112 via a memory bridge 105 and a communication path 113. The memory bridge 105 is furthermore coupled to an input / output (I / O) bridge 107 via a communication path 106, and the I / O bridge 107 is in turn coupled to a switch 116.

[0014] In operation, the I / O bridge 107 is configured to receive user input information from input devices 108, such as a keyboard or mouse, and forwards the input information to the CPU 102 for processing via the communication path 106 and the memory bridge 105. In some examples, the input devices 108 are used to verify the identity of one or more users, to grant authorized users access to the computer system 100, and to deny unauthorized users access. The switch 116 is configured to establish connections between the I / O bridge 107 and other components of the computer system 100, such as a network adapter 118 and various add-in cards 120 and 121.In some examples, network adapter 118 serves as the primary or exclusive device for receiving input data for processing by the disclosed methods.

[0015] As also shown, the I / O bridge 107 is coupled to a system disk 114, which can be configured to store content, applications, and data for use by the CPU 102 and the accelerator processing subsystem 112. Generally, the system disk 114 provides non-volatile storage for applications and data and can include fixed or removable hard disk drives, flash memory devices, and CD-ROM (Compact Disc Read-Only Memory), DVD-ROM (Digital Versatile Disc-ROM), Blu-ray, HD-DVD (High Definition DVD), or other magnetic, optical, or solid-state storage devices. Finally, although not explicitly shown, other components, such as Universal Serial Bus or other connectors, Compact Disc drives, Digital Versatile Disc drives, movie recording devices, and the like, can also be connected to the I / O bridge 107.

[0016] In various embodiments, the memory bridge 105 can be a northbridge chip and the I / O bridge 107 a southbridge chip. Furthermore, the communication paths 106 and 113, as well as other communication paths within the computer system 100, can be implemented using any technically suitable protocols, including, but not limited to, Peripheral Component Interconnect Express (PCle), HyperTransport, or other bus or point-to-point communication protocols known in the art.

[0017] In some embodiments, the accelerator processing subsystem 112 includes a graphics subsystem that delivers pixels to a display device 110, which may be a conventional cathode ray tube, liquid crystal display, light-emitting diode display, or the like. In such embodiments, the accelerator processing subsystem 112 includes a circuit optimized for graphics and video processing, which, for example, includes a video output circuit. As shown in Fig. As described in more detail in Section 2, such circuits can be contained in one or more accelerators present in Accelerator Processing Subsystem 112. An accelerator has one or more processing units capable of executing instructions, such as a central processing unit (CPU) or a parallel processing unit (PPU). Fig. 2-4, a graphics processing unit (GPU), a direct memory access (DMA) unit, an intelligence processing unit (IPU), a neural accelerator unit (NAU), a tensor processing unit (TPU), a neural network processor (NNP), a data processing unit (DPU), a vision processing unit (VPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) and / or the like.

[0018] In some embodiments, the accelerator processing subsystem 112 includes two processors, referred to herein as the primary processor (normally a CPU) and the secondary processor. Typically, the primary processor is a CPU and the secondary processor is a GPU. Additionally or alternatively, both the primary processor and the secondary processor may consist of one or more of the accelerator types disclosed herein, in any technically feasible combination. The secondary processor receives secure instructions from the primary processor via an unsecured communication path. The secondary processor accesses memory and / or other storage systems, such as system memory 104, Compute eXpress Link (CXL) memory extensions, memory-managed disk storage, on-chip memory, and / or the like.The secondary processor accesses this memory and / or another storage system over an insecure connection. The primary and secondary processors can communicate with each other via a GPU-to-GPU communication channel, such as Nvidia Link (NVLink). Additionally, the primary and secondary processors can communicate with each other via network adapter 118. Generally, the distinction between an insecure and a secure communication path is application-dependent. A given application program generally considers communication within a die, chip, or package to be secure. The communication of unencrypted data over a standard communication channel such as PCIe is considered insecure.

[0019] In some embodiments, the accelerator processing subsystem 112 includes circuits optimized for general-purpose and / or data processing. Such circuitry may also be contained within one or more accelerators included in the accelerator processing subsystem 112, configured to perform such general-purpose and / or computational operations. In other embodiments, the one or more accelerators included in the accelerator processing subsystem 112 may be configured to perform graphics processing, general-purpose processing, and arithmetic operations. The system memory 104 includes at least one device driver 103 configured to manage the processing operations of the one or more accelerators within the accelerator processing subsystem 112.

[0020] In other configurations, the accelerator processing subsystem 112 can be combined with one or more other elements of the Fig. 1 can be integrated into a single system. For example, the accelerator processing subsystem 112 can be integrated with the CPU 102 and other interconnect circuitry on a single chip to form a system on a chip (SoC).

[0021] It is clear that the system shown here is illustrative and that variations and modifications are possible. The connection topology, which includes the number and arrangement of the bridges, the number of CPUs 102, and the number of accelerator processing subsystems 112, can be changed as desired. In some embodiments, for example, the system memory 104 could be connected directly to the CPU 102, rather than via the memory bridge 105, and other devices could communicate with the system memory 104 via the memory bridge 105 and the CPU 102. In other alternative topologies, the accelerator processing subsystem 112 could be connected to the I / O bridge 107 or directly to the CPU 102, rather than via the memory bridge 105. In other embodiments, the I / O bridge 107 and the memory bridge 105 could be integrated on a single chip, instead of existing as one or more discrete devices.Finally, in certain embodiments, one or more of the features described in . Fig. The components shown in Figure 1 may not be present. For example, the switch 116 can be omitted, and the network adapter 118 and the add-on cards 120, 121 can be directly connected to the I / O bridge 107.

[0022] Fig. Figure 2 is a block diagram of a parallel processing unit (PPU) 202, which is located in the accelerator processing subsystem 112 of Fig. 1 according to various embodiments. Although in Fig. Since PPU 202 is shown in Figure 2, the accelerator processing subsystem 112, as indicated above, can have any number of PPUs 20. Furthermore, PPU 202 is the Fig. 2 an example of an accelerator that is in the accelerator processing subsystem 112 of the Fig. 1 is included. Alternative accelerators include, without limitation, CPUs, GPUs, DMA units, IPUs, NPUs, TPUs, NNPs, DPUs, VPUs, ASICs, FPGAs and / or the like. The ones in the Fig. 2 to Fig. The methods disclosed in Section 3 relating to the PPU 202 apply equally to any type of accelerator(s) included in the accelerator processing subsystem 112, in any combination. As shown, the PPU 202 is coupled to a local parallel processing (PP) memory 204. The PPU 202 and the PP memory 204 can be implemented with one or more integrated circuits, such as programmable processors, application-specific integrated circuits (ASICs), or memory devices, or in any other technically feasible form.

[0023] In some embodiments, the PPU 202 includes a graphics processing unit (GPU) that may be configured to implement a graphics rendering pipeline for performing various operations related to generating pixel data based on graphics data supplied by the CPU 102 and / or the system memory 104. When processing graphics data, the PP memory 204 can be used as graphics memory, storing one or more conventional image buffers and, if necessary, one or more other rendering targets. The PP memory 204 can be used, among other things, to store and update pixel data and to deliver the final pixel data or display images to the display device 110 for display. In some embodiments, the PPU 202 may also be configured for general-purpose processing and computational operations.

[0024] In operation, the CPU 102 is the main processor of the computer system 100 and controls and coordinates the operations of other system components. In particular, the CPU 102 issues instructions that control the operation of the PPU 202. In some embodiments, the CPU 102 writes a stream of instructions for the PPU 202 into a data structure (neither in Fig. 1 still in Fig. 2 (explicitly shown), which may be located in system memory 104, PP memory 204, or another memory location accessible to both CPU 102 and PPU 202. Additionally or alternatively, processors and / or accelerators other than CPU 102 may write one or more instruction streams for PPU 202 into a data structure. A pointer to the data structure is written to a push buffer to initiate the processing of the instruction stream in the data structure. PPU 202 reads the instruction streams from the push buffer and then executes the instructions asynchronously to the operation of CPU 102. In embodiments where multiple push buffers are created, execution priorities for each push buffer can be set by an application program via device driver 103 to control the timing of the different push buffers.

[0025] As also shown, the PPU 202 has an I / O (input / output) unit 205, which communicates with the rest of the computer system 100 via the communication path 113 and the memory bridge 105. The I / O unit 205 generates packets (or other signals) for transmission on the communication path 113 and also receives all incoming packets (or other signals) from the communication path 113 and forwards the incoming packets to the appropriate components of the PPU 202. For example, commands relating to processing tasks can be directed to a host interface 206, while commands relating to memory operations (e.g., reading from or writing to the PP memory 204) can be directed to a crossbar unit 210. The host interface 206 reads each push buffer and transmits the instruction stream stored in the push buffer to a front end 212.

[0026] As above in connection with Fig. As mentioned in Figure 1, the connection of the PPU 202 to the rest of the computer system 100 can be varied. In some embodiments, the accelerator processing subsystem 112, which includes at least one PPU 202, is implemented as an add-in card that can be inserted into an expansion slot of the computer system 100. In other embodiments, the PPU 202 can be integrated on a single chip with a bus bridge, such as the memory bridge 105 or the I / O bridge 107. In still other embodiments, some or all elements of the PPU 202 can be present together with the CPU 102 in a single integrated circuit or system of chips (SoC).

[0027] During operation, the front end 212 transmits the processing tasks received from the host interface 206 to a workload distribution unit (not shown) within the task / work unit 207. The workload distribution unit receives pointers to processing tasks, which are encoded as task metadata (TMD) and stored in memory. The pointers to TMDs are contained in an instruction stream, which is stored as a push buffer and received by the front end 212 from the host interface 206. Processing tasks, which can be encoded as TMDs, have indices associated with the data to be processed, as well as state parameters and instructions that define how the data is to be processed. The state parameters and instructions could, for example, define the program to be executed on the data.The task / work unit 207 receives tasks from the front end 212 and ensures that the GPCs 208 are configured to a valid state before initiating the processing tasks specified by the individual TMDs. A priority can be specified for each TMD, which is used to schedule the execution of the processing task. Processing tasks can also be received from the processing cluster array 230. Optionally, the TMDs can have a parameter that controls whether the TMD is added to the head or the end of a list of processing tasks (or to a list of pointers to the processing tasks), thus providing another layer of control over the execution priority.

[0028] The PPU 202 advantageously implements a highly parallel processing architecture based on a processing cluster arrangement 230, which has a set of C general processing clusters (GPCs) 208, where C ≥ 1. Each GPC 208 is capable of executing a large number (e.g., hundreds or thousands) of threads concurrently, with each thread being an instance of a program. In different applications, different GPCs 208 can be allocated for processing different types of programs or for performing different types of computations. The allocation of GPCs 208 can vary depending on the workload associated with each type of program or computation.

[0029] The memory interface 214 has a set of D partition units 215, where D ≥ 1. Each partition unit 215 is coupled to one or more dynamic random-access memory (DRAM) units 220 located in the PP memory 204. In one embodiment, the number of partition units 215 is equal to the number of DRAM units 220, and each partition unit 215 is coupled to a different DRAM unit 220. In other embodiments, the number of partition units 215 may differ from the number of DRAM units 220. It is understood that a DRAM unit 220 can be replaced by any other technically suitable arrangement. During operation, various rendering targets, such as texture maps and frame buffers, can be stored across DRAMs 220, allowing the partition units 215 to write sections of each rendering target in parallel to efficiently utilize the available bandwidth of the PP memory 204.

[0030] A specific GPC 208 can process data to be written to one of the DRAMs 220 within the PP memory 204. The crossbar unit 210 is configured to forward the output of each GPC 208 to the input of any partition unit 215 or to another GPC 208 for further processing. The GPCs 208 communicate with the memory interface 214 via the crossbar unit 210 to read from or write to various DRAMs 220. In one embodiment, the crossbar unit 210 has a connection to the I / O unit 205, in addition to a connection to the PP memory 204 via the memory interface 214, enabling the processing cores within the various GPCs 208 to communicate with the system memory 104 or other memory not belonging to the PPU 202. In the embodiment of the Fig. In 2, the crossbar unit 210 is directly connected to the I / O unit 205. In various embodiments, the crossbar unit 210 can use virtual channels to separate traffic flows between the GPCs 208 and the partition units 215.

[0031] Here too, the GPCs 208 can be programmed to perform processing tasks related to a variety of applications, including but not limited to linear and nonlinear data transformations, filtering of video and / or audio data, modeling operations (e.g., applying physical laws to determine the position, velocity, and other attributes of objects), image rendering operations (e.g., tessellation shader, vertex shader, geometry shader, and / or pixel / fragment shader programs), general arithmetic operations, etc. In operation, the PPU 202 is configured to transfer data from system memory 104 and / or PP memory 204 to one or more on-chip memory units, process the data, and write the resulting data back to system memory 104 and / or PP memory 204.The resulting data can then be accessed by other system components that have a CPU 102, another PPU 202 within the accelerator processing subsystem 112, or another accelerator processing subsystem 112 within the computer system 100.

[0032] As mentioned above, any number of PPUs 202 can be present in an accelerator processing subsystem 112. For example, multiple PPUs 202 can be located on a single add-in card, or multiple add-in cards can be connected via the communication path 113, or one or more of the PPUs 202 can be integrated into a bridge chip. The PPUs 202 in a multi-PPU system can be identical or different from one another. For example, different PPUs 202 can have a different number of processor cores and / or different amounts of PP memory 204. In implementations where multiple PPUs 202 are present, these PPUs can be operated in parallel to process data at a higher throughput than is possible with a single PPU 202.Systems containing one or more PPUs 202 can be implemented in a variety of configurations and form factors, including without limitation desktops, laptops, portable personal computers or other portable devices, servers, workstations, game consoles, embedded systems and the like.

[0033] Fig. Figure 3 is a block diagram of a general processing cluster (GPC) 208, which is located in the parallel processing unit (PPU) 202 of Fig. 2 according to various embodiments. In operation, the GPC 208 can be configured to execute a large number of threads in parallel to perform graphics, general processing, and / or computational operations. As used herein, a "thread" refers to an instance of a particular program running with a specific set of input data. Some embodiments employ Single-Instruction, Multiple-Data (SIMD) instruction output methods to support the parallel execution of a large number of threads without providing multiple independent instruction units.In other implementations, SIMT (Single-Instruction, Multiple-Thread) methods are used to support the parallel execution of a large number of generally synchronized threads, employing a common instruction unit configured to issue instructions to a number of processing engines within GPC 208. Unlike a SIMD execution regime, where all processing engines typically execute identical instructions, SIMT execution allows different threads to more easily follow different execution paths through a given program. A person skilled in the art understands that a SIMD processing regime is a functional subset of a SIMT processing regime.

[0034] The operation of the GPC 208 is controlled by a pipeline manager 305, which distributes the processing tasks received from a work distribution unit (not shown) within the task / work unit 207 to one or more streaming multiprocessors (SMs) 310. The pipeline manager 305 can also be configured to control a work distribution crossbar 330 by specifying destinations for processed data output by the SMs 310.

[0035] In one embodiment, the GPC 208 has a set of M SMs 310, where M ≥ 1. Each SM 310 also has a set of functional execution units (not shown), such as execution units and load memory units. Processing operations specific to each of the functional execution units can be grouped in a pipeline so that a new instruction can be issued for execution before a previous instruction has completed its execution. Any combination of functional execution units within a given SM 310 can be provided. In various embodiments, the functional execution units can be configured to support a variety of different operations, including integer and floating-point arithmetic (e.g., addition and multiplication), comparison operations, Boolean operations (e.g.,AND, OR, XOR), bit shifting and the calculation of various algebraic functions (e.g. planar interpolation and trigonometric, exponential and logarithmic functions, etc.). Advantageously, one and the same functional execution unit can be designed to perform different operations.

[0036] In operation, each SM 310 is configured to process one or more thread groups. As used herein, a "thread group" or "warp" refers to a group of threads that simultaneously execute the same program with different input data, with one thread of the group being assigned to a different execution unit within an SM 310. A thread group can have fewer threads than the number of execution units within the SM 310; in this case, part of the execution may be idle during the cycles in which the thread group is processed. A thread group can also have more threads than the number of execution units within the SM 310; in this case, processing may occur in successive clock cycles. Since each SM 310 can support up to G thread groups concurrently, it follows that up to G*M thread groups can be executed in a GPC 208 at any given time.

[0037] Furthermore, a large number of related thread groups can be active simultaneously in an SM 310 (at different stages of execution). This collection of thread groups is referred to here as a "cooperative thread array" ("CTA") or "thread array." The size of a given CTA is equal to m*k, where k is the number of concurrently executing threads in a thread group, which is typically an integer multiple of the number of execution units within the SM 310, and m is the number of concurrently active thread groups within the SM 310. In various implementations, a software application written in the CUDA (Compute Unified Device Architecture) programming language describes the behavior and operation of threads running on the GPC 208, including the behaviors and operations described above.A given processing task can be specified in a CUDA program, so that the SM 310 can be designed to perform and / or manage multi-purpose computing operations.

[0038] Although in Fig. Not shown in Figure 3, each SM 310 contains a Level 1 cache (L1 cache) or uses space in a corresponding L1 cache outside the SM 310 to support, among other things, load and store operations performed by the execution units. Each SM 310 also has access to Level 2 caches (L2) (not shown) shared by all GPCs 208 in the PPU 202. The L2 caches can be used to transfer data between threads. Finally, the SMs 310 also have access to the "global" memory outside the chip, which may include PP memory 204 and / or system memory 104. It is understood that any memory outside the PPU 202 can be used as global memory. Additionally, as shown in Figure 3, the SMs 310 can also be used as global memory. Fig. Figure 3 shows an L1.5 cache 335 present in the GPC 208 and configured to receive and hold data requested from memory by the SM 310 via the memory interface 214. Such data can include instructions, uniform data, and constant data. In embodiments with multiple SMs 310 within the GPC 208, the SMs 310 can advantageously utilize common instructions and data cached in the L1.5 cache 335.

[0039] Each GPC 208 can have an associated memory management unit (MMU) 320, configured to map virtual addresses to physical addresses. In various embodiments, the MMU 320 can be located either within the GPC 208 or in the memory interface 214. The MMU 320 includes a set of page table entries (PTEs) used to map a virtual address to a physical address of a tile or memory page, and optionally to a cache row index. The MMU 320 can include address translation lookaside buffers (TLBs) or caches, which may be located in SMs 310, in one or more L1 caches, or within the GPC 208.

[0040] In graphics and computing applications, the GPC 208 can be configured so that each SM 310 is coupled with a texture unit 315 to perform texture mapping operations, such as determining texture sampling positions, reading texture data, and filtering texture data.

[0041] In operation, each SM 310 transfers a processed task to the work-distribution crossbar 330 to make the processed task available to another GPC 208 for further processing or to store the processed task in an L2 cache (not shown), a PP memory 204, or a system memory 104 via the crossbar unit 210. In addition, a pre-raster operations (preROP) unit 325 is configured to receive data from the SM 310, forward data to one or more raster operations (ROP) units within the partition units 215, perform color mixing optimizations, organize pixel color data, and perform address translations.

[0042] It is clear that the core architecture described here is illustrative and that variations and modifications are possible. Among other things, any number of processing units, such as SMs 310, texture units 315, or preROP units 325, can be included in the GPC 208. Furthermore, the PPU 202, as described above in conjunction with Fig. 2 described, comprising any number of GPCs 208 configured to be functionally similar to one another, such that the execution behavior does not depend on which GPC 208 receives a particular processing task. Furthermore, each GPC 208 operates independently of the other GPCs 208 in the PPU 202 to execute tasks for one or more application programs. In light of the foregoing, the person skilled in the art understands that the Fig. The architecture described in 1-3 does not in any way limit the scope of the various embodiments of the present disclosure.

[0043] It should be noted that references to shared memory, as used here, may include one or more technically possible memories, including, without limitation, local memory shared by one or more SMs 310, or memory accessible via memory interface 214, such as cache memory, PP memory 204, or system memory 104. It should also be noted that references to cache memory, as used here, may include one or more technically feasible memories, including, but not limited to, an L1 cache, an L1.5 cache, and the L2 caches. Video encoder architecture for multi-line parallel encoding

[0044] Various embodiments include methods for the parallel encoding of multiple block lines by a video encoder contained in a computer system. As described here, an improved video encoder processes block lines of a media image (e.g., video, audio, and / or the like) as parallel units. The video encoder assigns a specific task to each hardware control unit in a set of multiple control units and schedules them to encode a different block line. In a video encoder with N control units, the line control units can encode N block lines simultaneously. The video encoder operates such that a preceding block line is encoded before a subsequent block line at least to the point where the block currently being encoded can access the necessary information for adjacent pixels contained in blocks of the preceding block line for the subsequent block line.This approach allows the row control unit for the subsequent block row to use the encoding information from the upper neighboring block row, which is being encoded by a different row control unit. Within a given block, the row control unit can sequentially encode the different partitions of the block. In this way, the video encoder enables the row control units to access information from neighboring partitions, thereby achieving high video encoding quality.

[0045] The line control units, which perform parallel encoding of block lines, share other hardware computational resources of the video encoder, including motion estimation, intra-search, rate distortion optimization, reconstruction, and filtering. Therefore, if one line control unit is waiting internally due to partition dependencies and thus does not utilize other hardware computational resources, other line control units encoding partitions for other block lines can utilize these resources. This parallel processing across multiple block lines allows for better utilization of these other hardware resources, thereby improving the utilization and efficiency of the video encoder's hardware resources. Furthermore, the performance of this approach can be enhanced by adding more line control units to enable parallel encoding of more block lines.In this way, the performance increase with the disclosed methods is scalable as a function of the number of line control units. At the same time, parallel coding of multiple block lines can achieve almost the same quality as sequential coding, but with significantly higher performance.

[0046] In this approach, each line control unit (LCU) within a hardware video encoder is responsible for encoding a different block line of a media frame contained in a video stream. The multiple LCUs manage the parallelism and synchronization between the concurrently encoded block lines. Furthermore, the LCUs manage access to various other hardware computational resources within the video encoder. These hardware computational resources are capable of handling computational requests from different LCUs encoding partitions across different block lines.

[0047] Fig. Figure 4 is a block diagram of a video encoder 400, designed to process multiple block lines in parallel for the computer system 100. Fig. 1-3 according to various embodiments. As shown, the video encoder 400, without limitation, comprises a line control unit assembly 410, a link 420, a motion estimation unit 425, an intra-search unit 430, a rate distortion optimization (RDO) unit 435, a reconstruction (Recon) unit 440, a filter 445, an engine storage system 450, a frame buffer interface (FB I / F) 470, and an entropy encoder 475. The line control unit assembly 410 comprises, without limitation, N line control units 415(0), 415(1), 415(2), ..., 415(N-1), also referred to as line controllers or simply controllers. The Engine Memory System 450 features various memory subsystems, including, but not limited to, Direct Memory Access Engines 455, Shared Memory 460, and Cache Memory 465.The components of the Engine Memory System 450 can access the image buffer memory (not shown) via the image buffer interface 470. The image buffer memory can be a special memory for storing image data or a section of another memory, including but not limited to PP memory 204, system memory 104, and / or the like.

[0048] Each line control unit 415(0), 415(1), 415(2), ..., 415(N-1) contained in the line control unit arrangement 410 encodes a different block line in a group of N block lines. For example, line control unit 415(0) can encode a first block line, line control unit 415(1) can encode a second block line, line control unit 415(2) can encode a third block line, and so on, so that line control unit 415(N-1) can encode an nth block line. Each line control unit 415(0), 415(1), 415(2), ... 415(N-1) has simultaneous access to various functional units of the encoder 400 via the connection 420. These other functional units include, among others, the motion estimation unit 425, the intra-search unit 430, the rate distortion optimization unit 435, the reconstruction unit 440, and the filter 445. The line control units 415(0), 415(1), 415(2), ... contained in the line control unit assembly 410The 415(N-1) units also communicate via one or more communication channels with the direct memory access engines 455, the shared memory 460, and the cache memory 465, which are contained in the engine memory system 450. To access the various functional units of the video encoder 400, the line control units 415(0), 415(1), 415(2), ..., 415(N-1) transmit commands and data relating to the currently encoded block via connection 420 to the corresponding functional unit.

[0049] With this simultaneous access to these functional units, the delay caused by a particular line control unit 415 has no significant effect on the access to functional units by other line control units 415. For example, when encoding a block for a block row, line control unit 415(0) may be prevented from accessing another functional unit, such as the motion estimator 425 or the intra-search unit 430, until the required input data is received from line control unit 415(0). While line control unit 415(0) is delayed until the input data is received, access to the motion estimator 425 and the intra-search unit 430 by other line control units 415(1), 415(2), ... 415(N-1), which are encoding other blocks for other block rows, is not blocked. Instead, other line control units 415(1), 415(2), ..., 415(N-1) can access the motion estimation unit 425, the intra-search unit 430 and / or other functional units as needed.

[0050] The connection 420 can be any suitable connection bus, any net, any network and / or the like to transfer data between line control units 415(0), 415(1), 415(2), ..., 415(N-1) and other functional units, including but not limited to the motion estimation unit 425, the intra-search unit 430, the rate distortion optimization unit 435, the reconstruction unit 440 and the filter 445.

[0051] The motion estimator 425 performs motion estimation and / or motion compensation for the blocks contained in the block lines encoded by the line control units 415(0), 415(1), 415(2), ... 415(N-1). More specifically, the motion estimator 425 performs motion estimation and / or motion compensation in response to receiving a request from a line control unit 415 to process a particular block. The motion estimator 425 performs motion estimation and / or motion compensation to generate an inter-frame candidate for the specified block based on the temporal redundancy between media frames. The motion estimator 425 generates a motion vector for the current block that predicts the block of the current media frame from the corresponding block of a reference media frame. The video encoder 400 can use the motion vector as an inter-frame candidate.Based on the motion vector, the motion estimator 425 generates motion-compensated pixels for the inter-image candidate. The motion estimator 425 then transmits these motion-compensated pixels to the requesting line control unit 415.

[0052] The intra-search unit 430 generates an intra-image candidate for the blocks contained in the block lines encoded by the line control units 415(0), 415(1), 415(2), ..., 415(N-1). More precisely, the intra-search unit 430 performs an intra-estimation and / or intra-prediction in response to receiving a request from a line control unit 415 to process a specific block. The intra-search unit 430 performs an intra-estimation and / or intra-prediction to generate an intra-image candidate for the specific block based on spatial redundancy within a media image.

[0053] To perform intra-estimation, the Intra-Search Unit 430 selects an intra-prediction mode based on the current pixels in the current media image and the neighboring pixels of the reconstructed current media image. In some embodiments, the Intra-Search Unit 430 can select the intra-prediction mode that best predicts the pixels of the current block. The Intra-Search Unit 430 can select the intra-prediction mode that yields the lowest rate-distortion cost based on the sum of squared errors (SSE) for the current block, as determined by the Rate Distortion Optimization Unit 435. The number and type of available prediction modes can vary depending on the block size. For example, the number and type of available prediction modes may differ for 4×4 pixel blocks, 8×8 pixel blocks, 6×16 pixel blocks, 32×32 pixel blocks and / or the like.In this respect, the Intra-Search Unit 430 can select different prediction modes for each of the possible block sizes, based on which prediction mode results in the lowest rate-distortion cost value determined by the Rate-Distortion Optimization Unit 435 for that block size. Furthermore, the Intra-Search Unit 430 can select different prediction modes for the luma values ​​in the block compared to the chroma sample values ​​in the block. In some embodiments, the prediction mode determines the order in which the pixels in the current block are sampled to generate the predicted intra-image candidate. For example, the prediction mode can specify vertical sampling, horizontal sampling, diagonal sampling down and to the left, diagonal sampling down and to the right, vertical sampling to the left, vertical sampling to the right, horizontal sampling down and to the top, and / or similar sequences.

[0054] To perform an intra-prediction, the intra-search unit 430 generates an intra-image candidate based on the selected intra-prediction mode. The intra-search unit 430 scans the pixel values ​​in the current block in the sequence specified by the selected intra-prediction mode. For each scanned pixel, the intra-search unit 430 determines a predicted pixel value based on the differences between that pixel value and the pixel values ​​of adjacent pixels. From these predicted pixel values, the intra-search unit 430 generates the intra-image candidate. The intra-search unit 430 then transmits the intra-image candidate to the requesting line control unit 415.

[0055] The Rate Distortion Optimizer 435 performs rate distortion optimization on the blocks contained in the block rows encoded by the Row Control Units 415(0), 415(1), 415(2), ... 415(N-1). More precisely, the Rate Distortion Optimizer 435 performs rate distortion optimization in response to receiving a request from a Row Control Unit 415 to process a specific block. The Rate Distortion Optimizer 435 selects a winning candidate for a block from between the inter-image candidate generated for that block by the Motion Estimator 425 and the intra-image candidate generated for that block by the Intra-Search Unit 430. The rate distortion optimization unit 435 receives the inter-image candidate and the intra-image candidate from the requesting line control unit 415.The rate distortion optimization unit 435 also receives the reconstructed pixels of the block in the reconstructed current media image from the requesting line control unit 415, as generated by the reconstruction unit 440. Based on the reconstructed pixels of the block in the reconstructed current media image, the rate distortion optimization unit 435 determines a rate distortion cost value based on the sum of squared errors (SSE) for the current block. The rate distortion optimization unit 435 selects the winning candidate, at least in part, based on the rate distortion cost value for the current block as determined by the rate distortion optimization unit 435. In some embodiments, the rate distortion optimization unit 435 also performs a transformation operation and / or a quantization operation on the block as part of the encoding procedure.The rate distortion optimization unit 435 transfers the winning candidate to the requesting line control unit 415. The rate distortion optimization unit 435 can also transfer the transformed and / or quantized block of the winning candidate to the requesting line control unit 415.

[0056] Reconstruction Unit 440 performs image reconstruction for the blocks contained in the block lines encoded by Line Control Units 415(0), 415(1), 415(2), ... 415(N-1). More precisely, Reconstruction Unit 440 performs image reconstruction in response to receiving a request from Line Control Unit 415 to process a specific block. Reconstruction Unit 440 performs image reconstruction on frequency coefficients that were previously transformed and quantized during the encoding process. Reconstruction Unit 440 executes an inverse quantization function to reverse the quantization previously performed on the block. Reconstruction Unit 440 also performs an inverse transformation function to reverse the transformation previously performed on the block. In doing so, Reconstruction Unit 440 produces a reconstructed remainder.The Reconstruction Unit 440 sums the reconstructed remainder with the winning candidate generated by the Rate Distortion Optimization Unit and received by the requesting Line Control Unit 415 to produce the Reconstructed Current Frame Block. The Reconstructed Current Frame Block is a proxy of the corresponding block of the media image generated by a video decoder when decoding the video stream produced by the video encoder 400. The Reconstruction Unit 440 transmits the reconstructed block to the requesting Line Control Unit 415.

[0057] Filter 445 performs one or more filtering operations on the blocks contained in the block lines encoded by the line control units 415(0), 415(1), 415(2), ... 415(N-1). More precisely, Filter 445 performs one or more filtering operations in response to receiving a request from a line control unit 415 to process a specific block. The one or more filtering operations may include deblocking filtering, sample adaptive offset filtering, sample matching filtering, and / or similar operations.In deblocking filtering, filter 445 improves the visual quality of the reconstructed current block of the media image by smoothing the sharp edges resulting from the transformation and / or quantization performed by the rate distortion optimization unit 435 during encoding, followed by the inverse quantization and / or inverse transformation performed by the reconstruction unit 440 during reconstruction. In sample adaptive offset filtering, filter 445 further filters the reconstructed current block of the media image by selectively adding offsets to the pixel values ​​of the reconstructed current block of the media image based on the pixel value of a given pixel and / or the pixel values ​​of one or more neighboring pixels. Filter 445 then passes the filtered block to the requesting line control unit 415.

[0058] The engine memory system 450, which includes direct-access memory engines 455, a common memory 460, and a cache memory 465, can communicate with other components of the video encoder 400, including, but not limited to, the line control units 415(0), 415(1), 415(2), ... 415(N-1), the motion estimator 425, the intra-search unit 430, the rate distortion optimization unit 435, the reconstruction unit 440, and the filter 445. The common memory 460 can store data and / or commands for use by the line control units 415(0), 415(1), 415(2), ... 415(N-1), the motion estimator 425, the intra-search unit 430, the rate distortion optimization unit 435, the Reconstruction unit 440 and / or filter 445 store data. Cache memory 465 can store short-term data and / or commands recently executed by line control units 415(0), 415(1), 415(2), ...415(N-1), the motion estimation unit 425, the intra-search unit 430, the rate distortion optimization unit 435, the reconstruction unit 440, and / or the filter 445 have been accessed or are predicted to be accessed by these units shortly. The data and / or instructions stored in the cache memory 465 may be a copy of data and / or instructions stored in other memory, including but not limited to shared memory 460, PP memory 204, system memory 104, and / or the like. Typically, the access times for loading data from and / or storing data in the cache memory 465 are shorter than the times for loading data from and / or storing data in these other memories. DMA engines 455 can perform block copies of data and / or instructions from one memory location to another memory location.More specifically, DMA engines 455 can copy a block of data and / or instructions within a specific memory location or between one memory location and another. Therefore, DMA engines 455 can copy a block of data and / or instructions within or between one or more shared memory locations 460, PP memory locations 204, system memory locations 104, and / or the like.

[0059] In some embodiments, line control units 415 can exchange data and / or instructions with each other and / or with other functional units of the encoder 400 via a shared memory 460. In such embodiments, a line control unit 415 can store motion vector data, neighbor pixel data, reconstruction data, instructions, and / or the like. The line control unit 415 can transmit a trigger instruction, along with the address in the shared memory where the corresponding data and / or instructions are stored, to a suitable functional unit. After executing one or more operations in response to the trigger instruction, the suitable functional unit can transmit the address in the shared memory where the corresponding result data are stored to the requesting line control unit 415.

[0060] When the line control units 415(0), 415(1), 415(2), ... 415(N-1) have completed encoding the blocks in each block line, they store the encoded blocks at a suitable location in the frame buffer memory via the frame buffer interface 470. An entropy encoder 475 monitors the frame buffer memory via the frame buffer interface 470 to determine when each encoded block has been stored in the frame buffer memory. From these encoded blocks stored in the frame buffer memory, the entropy encoder 475 generates the final encoded bitstream for the video encoder 400. In some embodiments, the entropy encoder 475 generates the final encoded bitstream, i.e., the output video stream, using a lossless compression method. Additionally or alternatively, the entropy encoder 475 generates the final encoded bitstream using a lossy compression method.To facilitate sequential block entropy coding, the line control units 415(0), 415(1), 415(2), ... 415(N-1) store the final winning candidate data in the frame buffer. The entropy encoder 475 encodes the blocks of a media image sequentially in raster scan order. The entropy encoder 475 waits for the final winning candidate data for each sequential block to be stored in the frame buffer before encoding the bitstream for that block. In this way, the entropy encoder 475 encodes the blocks of each block row of the image sequentially and individually in raster scan order. In raster scan order, the entropy encoder 475 encodes the blocks in each block row of the media image from left to right and encodes the block rows of the media image from top to bottom.

[0061] As described here, the line control units 415(0), 415(1), 415(2), ... 415(N-1) contained in the line control unit assembly 410 can encode up to N block lines of the media image simultaneously. If the media image contains a total of N block lines, the line control units 415(0), 415(1), 415(2), ... 415(N-1) can encode the media image in a single pass. More often, however, the number of block lines in the media image is greater than the number of line control units 415. In such cases, the video encoder 400 divides the media image into several groups of N block lines each. A first group of block lines contains the top N lines of the media image. A second group of block lines contains the second-to-top N lines of the media image. A third group of block lines contains the third-highest N lines of the media image, etc. The line control units are 415(0), 415(1), 415(2), ..., 415(N-1) encode the media image in groups, so that the line control units 415(0), 415(1), 415(2), ..., 415(N-1) encode the first group of block lines simultaneously, followed by the second group of block lines, followed by the third group of block lines simultaneously, and so on.

[0062] When each line control unit 415 has completed encoding the blocks for the corresponding block row of the first group of N block rows, the line control unit 415 can begin encoding a block row of a second group of N block rows. For example, when line control unit 415(0) has completed encoding the blocks for an assigned row in the first group of N rows, line control unit 415(0) can begin encoding the blocks for a corresponding row in the second group of N rows. Likewise, when each of the line control units 415(1), 415(2), ... 415(N-1) has completed encoding the blocks for the assigned block row in the first group of N block rows, line control units 415(1), 415(2), ... 415(N-1) can each begin encoding the blocks for a corresponding block row in the second group of N block rows.This process continues for each group of N lines until all lines for the current media image are encoded. In some embodiments, the total number of lines in the current media image may not be divisible by N. In such embodiments, the last group of lines to be encoded for the media image may contain fewer than N lines. Therefore, fewer than all N line control units 415 may be in operation while encoding the last group of block lines.

[0063] In some embodiments, the video encoder 400 can include feedback loops from a later stage to an earlier stage. For example, the visual quality of the output video stream can be improved by a feedback loop from the rate distortion optimization unit 435 to the motion estimation unit 425. With such a feedback loop, the motion estimation unit 425 can generate a motion vector for the current block based on pixel data from the current block as well as the motion vector from the previous block, resulting in improved motion estimation. Such a feedback loop can introduce a delay in which a line control unit 415 waits for the rate distortion optimization unit 435 to complete processing of the current block before the motion estimation unit 425 can perform motion estimation for the next block.With an operational line control unit 415, this delay through the feedback loop can lead to a reduction in performance and utilization, since one or more functional units are not accessed by the operational line control unit 415.

[0064] In contrast, if multiple line control units 415 are operational, a particular line control unit 415 may experience a delay, such as the delay resulting from the feedback loop delay from the rate distortion optimization unit 435 to the motion estimation unit 425. However, if the particular line control unit 415 is unable to access one or more functional units due to the delay, other line control units 415, which are encoding blocks for other block lines, can access that one or more functional units. This increases the utilization of the one or more functional units, resulting in improved performance.

[0065] Fig. Figure 5 shows a functional view of a video encoder 500, which produces a media image for the computer system 100. Fig. 1-4 according to various embodiments. The Video Encoder 500 can encode a video stream compatible with the HEVC (High Efficiency Video Coding) standard, also known as H.265 or MPEG-H (Motion Picture Experts Group High Efficiency) Part 2. Additionally or alternatively, the Video Encoder 500, as is and / or with minor modifications, can encode a video stream compatible with any other technically feasible video coding standard. Such additional and / or alternative video coding standards may include, but are not limited to, H.264, H.266, Video Compression Format 9 (VP9), Alliance for Open Media (AOMedia) Video 1 (AV1), and / or similar standards.

[0066] As shown, the Video Encoder 500 receives an input media image to be encoded. This received media image is used as the current media image (F n ) 505. The current media image (F n) 505 and other media images processed by the Video Encoder 500 are divided into multiple blocks, referred to as macroblocks, coding tree units (CTUs), and / or the like. Each block contains a group of adjacent pixels, for example, an 8x8 pixel block, a 16x16 pixel block, and / or the like. Each block is further divided into partitions, with each partition containing luminance pixels (luma pixels) and / or chrominance pixels (chroma pixels). Luma pixels contain the luma or Y pixel values ​​for the pixels in the block. Chroma pixels contain the chroma pixel values ​​for the pixels in the block. Chroma pixel values ​​are typically color difference values ​​and can be one of two types: (1) red color difference pixel values ​​(U or C) r ) and (2) blue color difference pixel values ​​(V or C) b ).

[0067] The Video Encoder 500 also features a reconstructed media image based on the previously received and encoded media image. This reconstructed media image is called the reference media image (F'). n-1 ) 510. Based on the current pixels in the current media image (F n ) 505 and on the reference pixels in the reference media image (F' n-1The Motion Estimator (ME) 510 generates a motion vector for the current block, predicting the block of the current media frame from the corresponding block of the reference media frame. The Video Encoder 500 can use the motion vector as an inter-frame candidate. The Motion Estimator 515 passes the inter-frame candidate to the Motion Compensation Unit (MC) 520. The Motion Compensation Unit 520 generates motion-compensated pixels for the inter-frame candidate. The Motion Compensation Unit 520 passes the motion-compensated pixels for the inter-frame candidate to the "Inter" input of the Selector 525.

[0068] Furthermore, the Intra estimation unit selects 570 based on the current pixels in the current media image (F n ) 505 and the adjacent pixels of the reconstructed current media image uF' n,The data received by a summing unit 565 selects an intra-prediction mode. In some embodiments, the intra-estimating unit 570 can select the intra-prediction mode that best predicts the pixels of the current block. The intra-estimating unit 570 can select the intra-prediction mode that yields the lowest rate-distortion cost value based on the sum of squared errors (SSE) of distortion for the current block, as determined by the rate-distortion optimization unit of the selector 525. The number and type of available prediction modes can vary depending on the block size. For example, the number and type of available prediction modes may differ for 4×4 pixel blocks, 8×8 pixel blocks, 16×16 pixel blocks, 32×32 pixel blocks, and / or the like.In this respect, the Intra Estimator 570 can select different prediction modes for each of the possible block sizes, based on which prediction mode results in the lowest rate-distortion cost value, as determined by the rate-distortion optimization unit for that block size. Furthermore, the Intra Estimator 570 can select different prediction modes for the luma values ​​in the block compared to the chroma sample values ​​in the block. In some embodiments, the prediction mode determines the order in which the pixels in the current block are sampled to generate the predicted intra-image candidate. For example, the prediction mode can specify vertical sampling, horizontal sampling, diagonal sampling down-left, diagonal sampling down-right, vertical sampling left, vertical sampling right, horizontal sampling down, horizontal sampling up, and / or the like.

[0069] Based on the selected intra-prediction mode, the intra-prediction unit 575 generates an intra-image candidate. The intra-prediction unit 575 scans the pixel values ​​in the current block in the order specified by the selected intra-prediction mode. For each scanned pixel, the intra-prediction unit 575 determines a predicted pixel value based on the differences between that pixel value and the pixel values ​​of neighboring pixels. From these predicted pixel values, the intra-prediction unit 575 generates the intra-image candidate. The intra-prediction unit 575 transmits the intra-image candidate to the "Intra" input of the selector 525.

[0070] Selector 525 determines whether to select the compensated pixels for the inter-frame candidate received by the motion compensation unit 520 or the intra-frame candidate received by the intra-prediction unit 575. The selection of the inter-frame candidate or the intra-frame candidate can be made at any level of granularity, including but not limited to block-wise, frame-by-frame, and / or similar methods. The procedure for determining whether to select the inter-frame candidate or the intra-frame candidate can be relatively simple or relatively complex. Typically, the more complex the procedure for determining whether to select the inter-frame candidate or the intra-frame candidate, the higher the video quality of the resulting encoded stream. The selected candidate between the inter-frame candidate and the intra-frame candidate is called the winning candidate.In some embodiments, the selector 525 determines the winner solely based on luma pixel values. In other embodiments, the selector 525 determines the winner based on both luma and chroma pixel values. Generally, selection based on luma and chroma pixel values ​​can be more accurate and therefore result in higher visual quality than selection based on luma pixel values ​​alone.

[0071] In some embodiments, the selector 525 can also perform rate distortion optimization (RDO) when selecting the winning candidate. The rate distortion optimization unit (in Fig. (5 not shown) from selector 525 receives the inter-image candidate and the intra-image candidate. The rate distortion optimization unit also receives the reconstructed pixels of the reconstructed current media image uF'. n,which are received by the inverse quantization unit 555, the inverse transformation unit 560, and the summing unit 565. Based on the reconstructed pixels of the reconstructed current media image uF', the rate distortion optimization unit determines a rate distortion cost value based on the sum of squared errors (SSE) of the distortion for the current block. The selector 525 selects the winning candidate, at least in part, based on the rate distortion cost value for the current block as determined by the rate distortion optimization unit. The selector 525 passes the winning candidate to the summing unit 530 and the summing unit 565.

[0072] The summing unit 530 inverts the winning candidate received from the selector 525 before combining the winning candidate with the current media image (F n) 505 combined. Consequently, the summator 530 determines the difference resulting from subtracting the winning candidate from the current media image (F n ) 505. This difference is called the residual pixel or, more generally, the residual D. n The summing mixer 530 carries over the remainder D. n to the transformation unit (T) 535.

[0073] The transformation unit 535 converts the remainder D received from the summing unit 530. ninto a series of frequency coefficients representing the image segment contained in each block. The transformation unit 535 transmits the frequency coefficients to the quantization unit (Q) 540. The quantization unit 540 reduces the total number of unique frequency coefficients received from the transformation unit 535 by quantizing the frequency coefficients according to defined frequency ranges or classes. The quantization unit 540 transmits the quantized frequency coefficients X to the rearrangement unit 545. The rearrangement unit 545 sorts the quantized frequency coefficients X in descending order, so that all coefficients with a value of zero ("0") are placed at the end of the set of frequency coefficients. The rearrangement unit 545 transmits the sorted quantized frequency coefficients to the entropy encoder 550.The entropy encoder 550 generates the final encoded bitstream for the video encoder 500. In some embodiments, the entropy encoder 550 generates the final encoded bitstream, i.e., the output video stream, using a lossless compression method. Additionally or alternatively, the entropy encoder 550 generates the final encoded bitstream using a lossy compression method. The final encoded bitstream generated by the video encoder 500 can then be decoded by a suitable video decoder (not shown).

[0074] In addition to transferring the quantized frequency coefficients X to the rearrangement unit 545, the quantization unit 540 transfers the quantized frequency coefficients X to the inverse quantization unit (Q). -1) 555. The inverse quantization unit 555 performs an inverse quantization function to reverse or invert the quantization performed by the quantization unit 540. The inverse quantization unit 555 transfers the inversely quantized frequency coefficients to the inverse transformation unit (T). -1 ) 560. The inverse transformation unit 560 performs an inverse transformation function to reverse or invert the transformation performed by the transformation unit 535. In doing so, the inverse transformation unit 560 generates the reconstructed remainder D'. n The inverse transformation unit 560 transfers the reconstructed remainder D' n to the summing unit 565.

[0075] The summing unit 565 adds the reconstructed remainder D' to the winning candidate generated by the selector 525. n in addition, to the reconstructed current media image uF' nto generate. The reconstructed current media image uF' n is a proxy of the media image that a video decoder generates when decoding the video stream produced by the video encoder 500. As described here, summing unit 565 transmits the reconstructed current media image uF'. n to the intra-estimation unit 570 to generate the intra-image candidate in conjunction with the intra-prediction unit 575. Additionally, the summing unit 565 transmits the reconstructed current media image uF'. n to the filter 580. In some embodiments, the filter 580 is a deblocking filter that improves the visual quality of the reconstructed actual media image uF' nImproved. Filter 580 improves visual quality by smoothing the sharp edges resulting from the transformation performed by the Transform Unit 535 and / or the quantization performed by the Quantization Unit 540, followed by the inverse quantization performed by the Inverse Quantization Unit 555 and / or the inverse transformation performed by the Inverse Transform Unit 560. Filter 580 passes the filtered image to the Sample Adaptive Offset (SAO) Filter 585. The Sample Adaptive Offset Filter 585 further filters the reconstructed actual media image uF'. n, by selectively adding offsets to the pixel values ​​of the reconstructed current media image uF' nbased on the pixel value of a given pixel and / or the pixel values ​​of one or more neighboring pixels. The Sample Adaptive Offset Filter 585 stores the SAO-filtered image as the final reconstructed current media image (F' n ) 590.

[0076] After the video encoder 500 has processed the current media image (F n Once ) 505 has completed, the video encoder 500 receives the next input media image, which then becomes the new current media image (F n ) 505 will be. In addition, the reconstructed current media image (F' n ) 590 to the new reference media image (F' n-1 ) 510. The 500 video encoder uses this new reference media image (F' n-1 ) 510, to select the Inter candidate for the new current media image (F n ) to generate 505.

[0077] In some embodiments, the visual quality of the output video stream can be further improved by a feedback loop (not shown) from the selector 525 to the motion estimator 515. After selecting the winning candidate, the selector 525 determines the final motion vector for the current block. The selector 525 transmits the final motion vector for the current block to the motion estimator 515. The motion estimator 515 can use this final motion vector for the current block to generate the motion vector for the next block. In this way, the motion estimator 515 can generate a motion vector for the current block based on pixel data from the current block as well as the motion vector from the previous block, resulting in improved motion estimation.This improved motion estimation can in turn lead to improved motion compensation, as performed by the motion compensation unit 520, and improved selection accuracy, as performed by the selector 525.

[0078] In some embodiments, a given block can contain multiple subblocks or partitions. The subblocks can be of different sizes. For example, a 16x16-pixel block can contain 8x16-pixel subblocks, 16x8-pixel subblocks, 8x8-pixel subblocks, and / or the like in any combination. In such embodiments, the motion estimation unit 515 can generate a motion vector for each subblock and combine the motion vectors from the different subblocks to generate a final motion vector for the block.

[0079] In some embodiments, the Video Encoder 500 can be combined with the architecture of the Video Encoder 400. Fig. 4 can be implemented. In such embodiments, media images, including but not limited to the current media image (F), can be implemented. n ) 505, the reference media image (F' n-1 ) 510 and the reconstructed current media image (F' n ) 590, in any technically possible storage medium. In particular, these media images can be stored in a shared storage medium 460, a cache storage medium 465, an image buffer storage medium, and / or the like. The motion estimation unit 515 and / or the motion compensation unit 520 can, without restriction, replace the motion estimation unit 425 of Fig. 4. The intra-estimation unit 570 and / or the forecast unit 575 can, without restriction, replace the intra-search unit 430 of Fig. 4. One or more of the selector 525 (including the rate distortion optimization unit of the selector 525), the summer 530, the transformation unit 535 and / or the quantization unit 540 can, without restriction, modify the rate distortion optimization unit 435 of Fig. 4. The inverse quantization unit 555, the inverse transformation unit 560 and / or the summing unit 565 can, without restriction, replace the reconstruction unit 440 of the Fig. 4.

[0080] Fig. Figure 6 is a block diagram of a video encoder 600, which is designed for sequential video coding and is implemented in the computer system 100 according to various embodiments. Fig. 1 can be implemented. As shown, the video encoder 600 without restriction includes a unit 610 for deciding on the previous block, a motion estimation unit 615, a reconstruction unit 620, an intra-search unit 625, and a rate distortion optimization (RDO) unit 630.

[0081] The previous block decision unit 610 can, without restriction, provide the inputs for generating an inter-image candidate and an intra-image candidate for the current block based on the output of the previous block's encoding. These inputs include the reconstructed media image based on the previously received and encoded media image. This reconstructed media image is used as the reference media image to generate the inter-image candidate for the current block. These inputs also include the reconstructed media image based on the current image, which provides the neighbor pixel data used to generate the intra-image candidate for the current block.

[0082] The motion estimation unit 615 can replace the motion estimation unit 425 without restriction. Fig. 4, the combination of motion estimation unit 515 and motion compensation unit 520 of Fig. 5 and / or the like. The motion estimator 615 generates an inter-image candidate for the current block, which is encoded by the video encoder 600. The motion estimator 615 transmits the inter-image candidate to the rate distortion optimization unit 630.

[0083] The reconstruction unit 620 can replace the reconstruction unit 440 without restriction. Fig. 4, the combination of the inverse quantization unit 555, the inverse transformation unit 560 and the summing unit 565 of the Fig. 5 and / or the like. The Reconstruction Unit 620 generates a reconstructed media image based on the current image block, which is being encoded. The reconstructed media image provides data from neighboring pixels to generate an intra-image candidate. The Reconstruction Unit 620 transmits the reconstructed media image to the Intra-Search Unit 625.

[0084] The intra-search unit 625 can, without restriction, replace the intra-search unit 430. Fig. 4, the combination of the intra-estimation unit 570 and the intra-prediction unit 575 of the Fig. 5 and / or the like. The Intra-Search Unit 625 generates an intra-image candidate for the current block, which is encoded by the Video Encoder 600, based on the reconstructed media image received by the Reconstruction Unit 620. The Intra-Search Unit 625 transmits the intra-image candidate to the Rate Distortion Optimization Unit 630.

[0085] The Rate Distortion Optimization Unit 630 can replace the Rate Distortion Optimization Unit 435 without restriction. Fig. 4, the selector 525 (including the rate distortion optimization unit of the selector 525) of Fig. 5 and / or the like. The Rate Distortion Optimization Unit 630 receives the inter-image candidate from the Motion Estimation Unit 615 and the intra-image candidate from the Intra-Search Unit 625. The Rate Distortion Optimization Unit 630 also receives the reconstructed pixels of the reconstructed current media image from the Reconstruction Unit 620. Based on the reconstructed pixels of the reconstructed current media image, the Rate Distortion Optimization Unit 630 determines a Rate Distortion Cost value based on the sum of squared errors (SSE) of the distortion for the current block. The Rate Distortion Optimization Unit 630 selects the winning candidate, at least in part, based on the Rate Distortion Cost value for the current block as determined by the Rate Distortion Optimization Unit.

[0086] The inputs to unit 610 for deciding on the previous block depend on a feedback loop 640 from the output of the rate distortion optimization unit 630 to the input of unit 610 for deciding on the previous block. This feedback loop 640 enables improved visual quality in generating the inter-image candidate and / or the intra-image candidate and in selecting between the inter-image candidate and the intra-image candidate. However, this feedback loop 640 can also introduce a delay when a current block is encoded based on data generated during the encoding of the current block and / or the previous block.This delay is represented by the RDO idle cycle 650, during which the rate distortion optimization unit 630 waits for the inter-frame candidate from the motion estimator 615 and / or the intra-frame candidate from the intra-search unit 625. After the rate distortion optimization unit 630 receives the inter-frame candidate from the motion estimator 615 and / or the intra-frame candidate from the intra-search unit 625, it performs the functions described here during the RDO busy cycle 655. The delay represented by the RDO idle cycle 650 can be eliminated by removing the feedback loop 640, but removing the feedback loop 640 may result in an undesirable loss of visual quality in the generated video stream.

[0087] In some embodiments, the video encoders 400, 500, 600 of the Fig. 4-6 other feedback loops and delay sources. In general, these feedback loops and other delay sources can lead to underutilization and reduced performance of various functional units of the video encoders 400, 500, 600 when a line control unit 415 is operating. However, when multiple line control units 415 are operational, delays caused by feedback loops and / or other sources maintained by one line control unit 415 can be mitigated by the operation of other line control units 415 that are not currently delayed. In this way, each line control unit 415 can access the various functional units of the video encoders 400, 500, 600 without regard to delays caused by one or more other line control units 415.

[0088] The Fig. Figures 7A-B illustrate how blocks of a media image are processed by the 400, 500, and 600 series video encoders. Fig. 4-6 are coded according to various embodiments. As in Fig. As shown in Figure 7A, a video encoder can sequentially encode the block lines 710 of a media image 700, for example, when a single line control unit 415 of the video encoder 400 is in operation. During sequential encoding of block lines 710, the video encoder encodes the blocks of each block line 710 individually in raster scan order. In raster scan order, the video encoder encodes blocks in each block line 710 of the media image 700 from left to right and encodes the block lines 710 of the media image 700 from top to bottom. The video encoder first encodes line 710(0) by encoding the leftmost block, then the second block from the left, and so on, until the rightmost block is completely encoded.After encoding all blocks in line 710(0), the video encoder proceeds to encode line 710(1) by encoding the leftmost block, followed by the second block from the left, and continues encoding the blocks sequentially until the encoding of the rightmost block is complete. After encoding all blocks in block line 710(1), the video encoder proceeds to encode block line 710(2) and the remaining block lines 710 in the same manner until all block lines 710 of the media image 700 are encoded.

[0089] As in Fig. As shown in Figure 7B, a video encoder can encode the block lines 760 of a media image 750 in parallel, for example, when multiple line control units 415 of the video encoder 400 are in operation. When encoding block lines 760 in parallel, a video encoder containing N line control units 415 can encode the blocks of up to N block lines 760 in parallel. Specifically, the video encoder can encode the blocks of block line 760(0), the blocks of block line 760(1), the blocks of block line 760(2), and so on, up to the blocks of block line 760(N-1) in parallel. When encoding the blocks of a particular line 760, the video encoder can include data from an adjacent block in the same line 760 and / or from the previous line 760.For example, to encode the rightmost shaded block in line 760(1), the video encoder can include data, such as pixel data, motion vector data, and / or the like, from the adjacent block to the left of the current block in the same line 760(1). Additionally or alternatively, the video encoder can include data, such as pixel data, motion vector data, and / or the like, from one or more adjacent blocks of the current block in the previous block line 760(0), such as the block directly above it, the block above and to its left, and / or the block above and to the right of the current block. Therefore, when the line control unit 415(1) encodes a block in block line 760(1), it waits until the line control unit 415(0) has encoded the relevant adjacent blocks in block line 760(0).Similarly, when the line control unit 415(2) encodes a block in block line 760(2), it waits until the line control unit 415(1) has encoded the corresponding neighboring blocks in block line 760(1), and so on. In this way, multiple line control units 415 can encode blocks at different positions from left to right in their respective block lines 760, even though multiple line control units 415 can encode blocks for multiple block lines 760 in parallel.

[0090] When each line control unit 415 has completed encoding the blocks for the corresponding block line 760 of the first group of N block lines, the line control unit 415 can begin encoding a block line 760 for a second group of N block lines. For example, when line control unit 415(0) has completed encoding the blocks for block line 760(0), line control unit 415(0) can begin encoding the blocks for block line 760(N) (not shown). Likewise, line control unit 415(1) can begin encoding the blocks for block line 760(N+1) (not shown) when line control unit 415(1) has completed encoding the blocks for block line 760(1).Similarly, the line control unit 415(2) can begin encoding the blocks for block line 760(N+2) (not shown) when the line control unit 415(2) has completed encoding the blocks for block line 760(2), and so on. This process continues for each group of N lines 760 until all lines 760 for media images 750 have been encoded. In some embodiments, the total number of block lines in a media image 750 may not be divisible by N. In such embodiments, the last group of block lines 760 to be encoded for a media image 750 may contain fewer than N block lines 760. As a result, fewer than all N line control units 415 may be operational while encoding the last group of block lines 760.

[0091] It is understood that the system shown here serves only for illustration and that variations and modifications are possible. The procedures described here can be executed by one or more alternative accelerators, including but not limited to CPUs, GPUs, video encoders, DMA units, IPUs, NPUs, TPUs, NNPs, DPUs, VPUs, ASICs, FPGAs, and / or the like, in any combination. In general, the procedures described here can be applied to any CPU 102, PPU 202, video encoder, and / or any other processing unit in any combination.

[0092] Fig. Figure 8 is a flowchart of the process steps for the parallel coding of multiple block lines in the computer system of the Fig. 1-7B according to various embodiments. Additionally or alternatively, the process steps can be performed by one or more alternative accelerators, including, but not limited to, CPUs, GPUs, DMA units, IPUs, NPUs, TPUs, NNPs, DPUs, VPUs, ASICs, FPGAs and / or the like, in any combination. Although the process steps in conjunction with the systems of Fig. As described in 1-7B, the person skilled in the art understands that any system designed to carry out the process steps in any order falls within the scope of this disclosure.

[0093] As shown, a procedure 800 begins at step 802, in which a first control, such as a first of the control units 415 of Fig. 4, blocks are encoded in a first block row of a media image. In step 804, a second controller, such as a second of the 415 control units, encodes. Fig.4. Blocks in a second block row of the media image. The first controller encodes the first set of blocks simultaneously with the second controller, which encodes the second set of blocks. Depending on the video encoding standard, the blocks contained in the first and second block rows can be macroblocks, coding tree units (CTUs), and / or the like. More generally, the first and second controllers are two controllers in a set of N controllers contained in a row control unit array. Each of the N controllers encodes a different block row in a group of N block rows of a media image.

[0094] In step 806, the first controller or the second controller determines that a coding resource is available to perform a coding function on a first block, where the first block is contained in the first block row or the second block row, respectively. The coding resource is one of several functional units contained within the video encoder, which also includes the first controller and the second controller. Each of the controllers has concurrent access to these various functional units of the video encoder. These other functional units include, among others, a motion estimation unit, an intra-search unit, a rate distortion optimization unit, a reconstruction unit, and a filter.

[0095] To access the various functional units of the video encoder, the first or second controller in step 808 stores commands and / or data relating to the first block to be coded in a shared memory. This shared memory can be any memory that can be accessed by the controllers contained in the line control unit arrangement, as well as various other functional units of the video encoder.

[0096] To access the various functional units of the video encoder, the first controller or the second controller in step 810 sends a command to the encoding resource, causing the encoding resource to execute an encoding function on the first block. In response to receiving the command, the encoding resource executes an encoding function. The encoding resource and the encoding function can be any resource and function that perform one or more steps to encode the first block.

[0097] In this respect, the encoding resource can be a motion estimation unit. The encoding function can include generating a motion vector for a first block contained within the first plurality of blocks. The motion vector can be an inter-image candidate for the first block. Furthermore, the encoding function can include generating motion-compensated pixels for the inter-image candidate based on the motion vector.

[0098] Additionally or alternatively, the coding resource can be an intra-search unit. The coding function can include selecting an intra-prediction mode based on pixel data contained in a first block within the first plurality of blocks, and pixel data from neighboring pixels contained in a reconstructed media image of the media image. Furthermore, the coding function can include generating an intra-image candidate based on the selected intra-prediction mode.

[0099] Additionally or alternatively, the coding resource can be a rate-distortion optimization unit. The coding function can include selecting a winning candidate for a first block contained in the first plurality of blocks from between an inter-image candidate for the first block generated by a motion estimation unit and an intra-image candidate for the first block generated by an intra-search unit. The coding function for selecting the winning candidate for the first block can include: determining a rate-distortion cost value based on a sum of squared error (SSE) distortion for the first block; and selecting the winning candidate at least partially based on the rate-distortion cost value.

[0100] Additionally or alternatively, the encoding resource can be a reconstruction unit. The encoding function can include generating frequency coefficients by performing an inverse quantization function to undo a quantization previously performed on a first block contained within the first plurality of blocks. Furthermore, the encoding function can include generating reconstructed residual data by performing an inverse transformation function to undo a transformation previously performed on the first block. Finally, the encoding function can include summing the reconstructed residual data with an inter-image candidate for the first block or an intra-image candidate for the first block to generate a reconstructed block of the first block.

[0101] Additionally or alternatively, the encoding resource can be a filter unit. The encoding function can include filtering the first block using at least one deblocking filter, a sample adaptive offset filter, and / or the like.

[0102] In step 812, the first controller or the second controller determines whether the encoding of the first block is complete. The encoding of the first block is not complete if at least one encoding function has not yet been executed by the corresponding encoding resource. If the encoding of the first block is not complete, procedure 800 returns to step 806 as described above. The encoding of the first block is complete when the various encoding functions executed by the different encoding resources have completed and the first block is ready for entropy encoding. If the encoding of the first block is complete, procedure 800 proceeds to step 814.

[0103] In step 814, the first controller or the second controller stores the coded first block in memory. The memory can be any suitable memory, including but not limited to a frame buffer, PP memory 204, system memory 104, and / or the like. After storing the coded first block in shared memory, an entropy encoder generates a bitstream from the coded first block. The entropy encoder encodes the blocks of a media image sequentially in raster-scan order. The entropy encoder waits for the final winner-candidate data for each sequential block to be stored in memory before encoding the bitstream for that block. In this way, the entropy encoder encodes the blocks of each block row of the image sequentially and individually in raster-scan order.In raster scan order, the entropy encoder encodes blocks in each line of the media image from left to right and encodes the lines of the media image from top to bottom. Procedure 800 then terminates. Alternatively, procedure 800 continues with step 802, described above, to process additional blocks in the lines and additional lines in the media image.

[0104] In summary, various embodiments include methods for the parallel encoding of multiple block lines by a video encoder contained in a computer system. As described here, an improved video encoder processes block lines in a media image as parallel units. The video encoder assigns and schedules the encoding of a different block line to each hardware control unit in a set of multiple control units. In a video encoder with N control units, the line control units can encode N block lines concurrently. The video encoder operates such that a preceding block line is encoded before a subsequent block line, at least until the point at which the block currently encoded for the subsequent block line can access the necessary information for adjacent pixels contained in blocks of the preceding block line.This approach allows the row control unit for the subsequent block row to use the encoding information from the upper neighboring block row, which is being encoded by a different row control unit. Within a given block, the row control unit can sequentially encode the different partitions of the block. In this way, the video encoder enables the row control units to access information from neighboring partitions, thereby achieving high video encoding quality.

[0105] The line control units, which perform parallel encoding of block lines, share other hardware computational resources of the video encoder, including motion estimation, intra-search, rate distortion optimization, reconstruction, and filtering. Therefore, if a line control unit is waiting internally due to partition dependencies and thus does not utilize other hardware computational resources, other line control units encoding partitions for other block lines can utilize these resources. This parallel processing across multiple block lines allows for better utilization of these other hardware resources, thereby improving the utilization and efficiency of the video encoder's hardware resources. Furthermore, the performance of this approach can be enhanced by adding more line control units to enable parallel encoding of more block lines.In this way, the performance increase with the disclosed methods is scalable as a function of the number of line control units. At the same time, parallel encoding of multiple block lines can achieve almost the same quality as sequential encoding, but with significantly higher performance.

[0106] In this approach, each line control unit (LCU) within a hardware video encoder is responsible for encoding a different block line of a media frame contained in a video stream. The multiple LCUs manage the parallelism and synchronization between the concurrently encoded block lines. Furthermore, the LCUs manage access to various other hardware computing resources within the video encoder. These hardware computing resources are capable of handling computational requests from different LCUs encoding partitions across different block lines.

[0107] A technical advantage of the disclosed methods over the prior art is that a video encoder using the disclosed methods has multiple line control units that encode multiple lines of partitions in the media image in parallel. Furthermore, various other hardware computing resources included in the video encoder are available in parallel to the line control units. Therefore, each line control unit can access all free functional units of the video encoder without having to wait for other line control units to reach a specific stage of the video encoding process. This improves the parallel processing of block lines and the utilization of other functional units of the video encoder compared to previous conventional approaches. These advantages represent one or more technological improvements over prior art approaches.

[0108] All combinations of claim elements listed in the claims and / or of elements described in this application fall in any way within the considered scope of the present disclosure and protection.

[0109] The descriptions of the various embodiments serve for illustrative purposes but do not claim to be exhaustive or limited to the disclosed embodiments. Many modifications and variations are obvious to the person skilled in the art without affecting the scope and spirit of the described embodiments.

[0110] Aspects of the present embodiments may be implemented as a system, method, or computer program product. Accordingly, aspects of the present disclosure may take the form of a purely hardware implementation, a purely software implementation (including firmware, resident software, microcode, etc.), or an embodiment that combines software and hardware aspects, all of which may be generally referred to herein as a "module" or "system." Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer-readable media containing computer-readable program code.

[0111] Any combination of one or more computer-readable media can be used. The computer-readable medium can be a computer-readable signaling medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not exclusively, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.More specific examples (a non-exhaustive list) of computer-readable storage medium would include: an electrical connection with one or more wires, a portable computer disk, a hard disk, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only storage device (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. For the purposes of this document, computer-readable storage medium can be any tangible medium capable of containing or storing a program for use by or in conjunction with a command-execution system, device, or apparatus.

[0112] Aspects of the present disclosure are described above with reference to flowchart diagrams and / or block diagrams of processes, devices (systems), and computer program products according to embodiments of the disclosure. It is understood that each block of the flowchart diagrams and / or block diagrams, and combinations of blocks in the flowchart diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to create a machine such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in the flowchart and / or in a block or blocks of a block diagram.Such processors can be, without restriction, general-purpose processors, special-purpose processors, application-specific processors, or field-programmable gate arrays.

[0113] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, procedures, and computer program products according to various embodiments of the present disclosure. In this respect, each block in the flowchart or block diagrams can represent a module, segment, or section of code comprising one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions specified in the blocks may occur in a different order than that shown in the figures. For example, two blocks shown consecutively may in reality be executed essentially simultaneously, or the blocks may sometimes be executed in reverse order, depending on the specific functionality.It should also be noted that each block of the block diagrams and / or flowchart representation, as well as combinations of blocks in the block diagrams and / or flowchart representation, can be implemented by special hardware-based systems that perform the specified functions or actions, or by combinations of special hardware and computer instructions.

[0114] While the foregoing relates to embodiments of the present disclosure, other and further embodiments of the disclosure may be developed without affecting the fundamental scope of the disclosure, and the scope of the disclosure is determined by the following claims.

Claims

[1] Computer-implemented method for parallel encoding of multiple block lines in a media image, wherein the method comprises: Encoding an initial multitude of blocks contained in a first line of the media image by means of an initial control; Encoding a second set of blocks contained in a second line of the media image by a second controller in parallel to the first controller that encodes the first set of blocks; and Accessing a first hardware computing resource by the second controller to execute a first video encoding function, where the first hardware computing resource is simultaneously accessible by the first controller and the second controller. [2] Computer-implemented method according to claim 1, wherein the first controller encodes the first plurality of blocks simultaneously with the second controller encoding the second plurality of blocks. [3] Computer-implemented method according to claim 1 or 2, wherein the first hardware computing resource comprises a motion estimation unit, and wherein the first video coding function comprises generating a motion vector for a first block contained in the first plurality of blocks, and wherein the motion vector comprises an inter-image candidate for the first block. [4] Computer-implemented method according to claim 3, wherein the first video coding function further comprises generating motion-compensated pixels for the inter-image candidate based on the motion vector. [5] Computer-implemented method according to any of the preceding claims, wherein the first hardware computing resource comprises an intra-search unit, and wherein the first video coding function comprises selecting an intra-prediction mode based on pixel data contained in a first block that is contained in the first plurality of blocks, and pixel data of neighboring pixels contained in a reconstructed media image of the media image. [6] Computer-implemented method according to claim 5, wherein the first video coding function further comprises generating an intra-image candidate based on the selected intra-prediction mode. [7] Computer-implemented method according to any of the preceding claims, wherein the first hardware computing resource comprises a rate distortion optimization unit, and wherein the first video coding function comprises selecting a winning candidate for a first block contained in the first plurality of blocks between an inter-image candidate for the first block generated by a motion estimation unit and an intra-image candidate for the first block generated by an intra-search unit. [8] Computer-implemented method according to claim 7, wherein selecting the winning candidate for the first block comprises: Determining a rate-distortion cost value based on a sum of squared errors with respect to a distortion for the first block; and Selecting the winning candidate at least partially based on the rate distortion cost value. [9] Computer-implemented method according to any of the preceding claims, wherein the first hardware computing resource comprises a reconstruction unit, and wherein the first video coding function comprises: Generating frequency coefficients by performing an inverse quantization function to reverse a quantization previously performed on a first block contained in the first plurality of blocks; and Generating reconstructed residual data by performing an inverse transformation function to reverse a transformation previously performed on the first block. [10] Computer-implemented method according to claim 9, wherein the first video coding function further comprises: Summing the reconstructed residual data with an inter-image candidate for the first block or an intra-image candidate for the first block to generate a reconstructed block of the first block. [11] Computer-implemented method according to any of the preceding claims, wherein the first hardware computing resource comprises a filter unit, and wherein the first video coding function comprises filtering a first block contained in the first plurality of blocks using a deblocking filter and / or a sample adaptive offset filter. [12] Computer-implemented method according to one of the preceding claims, further comprising, after encoding a first block contained in the first plurality of blocks, storing the encoded first block in a shared memory, wherein an entropy encoder generates a bitstream from the encoded first block. [13] Computer-implemented method according to any of the preceding claims, wherein the first plurality of blocks and the second plurality of blocks comprise macroblocks and / or coding tree units or CTUs. [14] Computer-implemented method according to any of the preceding claims, wherein the first controller and the second controller are contained in a plurality of encoders encoding a first group of lines of the media image, the first group of lines comprising the first line and the second line, and further comprising: Determine, by means of the first control, that the encoding of the first multitude of blocks contained in the first line of the media image is complete; and Encoding by first controlling a third multitude of blocks contained in a third line of the media image, where the third line is contained in a second group of lines of the media image. [15] Computer-implemented method according to claim 14, further comprising: Determine, by means of the second control, that the encoding of the second set of blocks contained in the second line of the media image is complete; and Encoding a fourth set of blocks contained in a fourth line of the media image, by the second control, where the fourth line is contained in the second group of lines of the media image. [16] Computer-implemented method according to one of the preceding claims, wherein the first plurality of blocks and / or the second plurality of blocks is encoded according to one or more of High Efficiency Video Coding, HEVC, 264, H.264, H.265, H.266, Video Compression Format 9, VP9, ​​or Alliance for Open Media, AOMedia, Video 1, AV1. [17] Completing computer system: a first control: which encodes an initial multitude of blocks contained in a first line of a media image; and a second control system: which encodes a second multitude of blocks contained in a second line of the media image, and which accesses an initial hardware computing resource to perform an initial video encoding function, where the first hardware computing resource is simultaneously accessible by the first controller and the second controller. [18] Computer system according to claim 17, wherein the first hardware computing resource comprises a motion estimation unit, and wherein the first video coding function comprises: Generating a motion vector for a first block contained in the first plurality of blocks, wherein the motion vector includes an inter-image candidate for the first block; and Generate motion-compensated pixels for the inter-image candidate based on the motion vector. [19] Computer system according to claim 17 or 18, wherein the first hardware computing resource comprises an intra-search unit, and wherein the first video coding function comprises: Selecting an intra-prediction mode based on pixel data contained in a first block, which is contained in the first plurality of blocks, and pixel data from neighboring pixels contained in a reconstructed media image of the media image; and Generating an intra-image candidate based on the selected intra-prediction mode. [20] Computer system according to any one of claims 17 to 19, wherein: The computer system also includes a motion estimation unit and an intra-search unit, the first hardware computing resource includes a rate distortion optimization unit, The first video coding function includes selecting a winning candidate for a first block contained in the first plurality of blocks, between an inter-image candidate for the first block generated by the motion estimation unit and an intra-image candidate for the first block generated by the intra-search unit, by: Determining a rate-distortion cost value based on a sum of squared errors with respect to a distortion for the first block; and Selecting the winning candidate at least partially based on the rate distortion cost value.

Images