Method for uniform parallel processing of independently coded image segments

Abstract

An apparatus and method for efficiently processing video data for data transport. In various implementations, a computing system includes a video processing circuit sending a video stream to a display device. The video processing circuit divides a video frame into partitions such as slices, each with corresponding multiple rows. The video processing circuit includes multiple slice processing circuits, each for performing a compression algorithm on a corresponding slice of the video frame. The slice processing circuits have access to a shared buffer that stores context data of the compression algorithm. The shared buffer can also store reconstructed pixels of a row that was processed (compressed) earlier. By providing access, via the shared buffer, to the different types of data previously generated by the slice processing circuits, the video processing circuit can select any available slice processing circuit to process any row of any slice of the video frame.

Description

METHOD FOR UNIFORM PARALLEL PROCESSING OF INDEPENDENTLY CODED IMAGE SEGMENTSCROSS REFERENCE TO RELATED APPLICATION

[0001] The present application claims priority to U.S. Patent Application Serial No. 18 / 618722 filed March 27, 2024, entitled METHOD FOR UNIFORM PARALLEL PROCESSING OF INDEPENDENTLY CODED IMAGE SEGMENTS, the entire contents of which are incorporated herein by reference.BACKGROUNDDescription of the Relevant Art

[0002] Video processing algorithms are complex and include many different functions. Computing systems use advanced processors to satisfy the high computation demands. The video processing complexity increases as display resolution increases and refresh rates increase. Additionally, high-definition video encoding applications are growing rapidly in the consumer market space and video processing applications requiring processing in real-time also increases. These multiple instances of increasing demands of video processing systems cause the data bandwidth of video data transport to increase. The video data transport can be across a display link to a display device. The video data transport can also be across a bus or interconnects of a communication fabric to transfer the video data between two memories. Many times, the communication protocols used for transporting video data are unable to satisfy the growing video bandwidth demands.

[0003] In view of the above, efficient methods and apparatuses for efficiently processing video data for data transport are desired.BRIEF DESCRIPTION OF THE DRAWINGS

[0004] FIG. 1 is a generalized diagram of an apparatus that efficiently processes video data for transport.

[0005] FIG. 2 is a generalized block diagram of a method for efficiently processing video data for data transport.

[0006] FIG. 3 is a generalized block diagram of a method for efficiently processing video data for data transport.

[0007] FIG. 4 is a generalized diagram of a computing system that efficiently processes video data for transport.

[0008] FIG. 5 is a generalized diagram of a timing diagram that efficiently processes video data for transport.

[0009] FIG. 6 is a generalized diagram of a timing diagram that efficiently processes video data for transport.

[0010] FIG. 7 is a generalized diagram of an apparatus that efficiently processes video data for transport.

[0011] FIG. 8 is a generalized diagram of a timing diagram that efficiently processes video data for transport.

[0012] While the invention is susceptible to various modifications and alternative forms, specific implementations are shown by way of example in the drawings and are herein described in detail. It should be understood, however, that drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the invention is to cover all modifications, equivalents and alternatives falling within the scope of the present invention as defined by the appended claims.DETAILED DESCRIPTION

[0013] In the following description, numerous specific details are set forth to provide a thorough understanding of the present invention. However, one having ordinary skill in the art should recognize that the invention might be practiced without these specific details. In some instances, well-known circuits, structures, and techniques have not been shown in detail to avoid obscuring the present invention. Further, it will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements are exaggerated relative to other elements.

[0014] Apparatuses and methods for efficiently processing video data for data transport are contemplated. In various implementations, a computing system includes a video processing circuit sending a video stream to a display device. In an implementation, the video processing circuit receives rendered video data, compresses the video data, and sends the compressed version of the video data to a display device via a display link. In some implementations, thedisplay link is a video cable. The video processing circuit is in a user’s computing device such as one of a desktop computer, a laptop computer, a server computer, a tablet computer, a smartphone, a gaming device, a smartwatch, and so on. Each of the computing device, the display link, and the display device supports a chosen communication protocol. Examples of the communication protocol are DisplayPort, HDMI (High-Definition Multimedia Interface), and so forth.

[0015] Although video data transport can be across a display link to a display device, the video data transport can also be across a bus or interconnects of a communication fabric to transfer the video data between two memories. The memory controllers of the two memories support one of a variety of memory communication protocols. To support display devices that can achieve high resolution and high refresh rate, but also reduce the video data transmission bandwidth, the video processing circuit performs one of a variety of compression algorithms. An example of the compression algorithm is Display Stream Compression (DSC).

[0016] To perform the steps of the compression algorithm, the video processing circuit divides an image, such as a video frame, into multiple partitions. In some implementations, each partition is a slice, which is processed in raster scan order. As used herein, the term “raster scan order” refers to an ordering of pixels of a video frame by rows such as ordering from left to right across the video frame and then moving down to a subsequent row of pixels. A “row” of a video frame can also be referred to as a “horizontal line” or “line.” When a video frame is divided into slices, the rows (or lines) traverse only across the slice, rather than across the entire video frame. The video processing circuit includes multiple slice processing circuits, each for performing the compression algorithm on a corresponding slice of the video frame. In some implementations, each of the slice processing circuits performs the DSC algorithm on an assigned row of a slice at a time.

[0017] The multiple slice processing circuits have access to a shared buffer that stores context data of the compression algorithm. In some implementations, the compression algorithm processes pixels in groups, such as a group of three pixels, as the compression algorithm traverses across the row. When executing the compression algorithm, the slice processing circuit generates context data for each group of pixels. The slice processing circuit uses the context data of a first group when performing compression on a contiguous, subsequent group of pixels in the row. Rather than each slice processing circuit stores its own context data, the slice processing circuits share the context data. Additionally, in an implementation, the shared buffer stores reconstructed pixels of a row that was processed (compressed) earlier. Therefore, in an implementation, when a slice processing circuit processes a fourth row of a second slice,this slice processing circuit has access to a copy of the reconstructed pixels of the third row of the second slice although another slice processing circuit processed the third row. Additionally, the slice processing circuit has access to context data corresponding to the last group of pixels of the third row of the second slice, which the slice processing circuit uses when processing the first group of pixels of the fourth row of the second slice. It is noted that for each row of a slice except the first row, slice processing circuits access the shared buffer to obtain a copy of the reconstructed pixels of a previous row. There is no previous row for the first row of each slice.

[0018] By providing access, via the shared buffer, to the different types of data previously generated by the slice processing circuits, the video processing circuit can select any available slice processing circuit to process (compress) any row of any slice of the video frame. It is unnecessary for the video processing circuit to assign rows of a particular slice to only the slice processing circuit that has been processing the particular slice . Assigning rows of the particular slice to any of the slice processing circuits both reduces the number of instantiations of the slice processing circuit and reduces the operating clock frequency of the slice processing circuits. Therefore, on-die area reduces, power consumption reduces, but performance increases as scheduling becomes less complicated and data processing begins sooner. Further details of these techniques to efficiently generate memory access requests for executing machine learning data models are provided in the following description of Figures 1-8.

[0019] Referring to FIG. 1, a generalized diagram is shown of an apparatus 100 for efficiently processing video data for data transport. In the illustrated implementation, apparatus 100 includes pixel processing circuit 110 that receives video data 150 and video data 151 and generates compressed video data 160 and compressed video data 161. To do so, pixel processing circuit 110 uses multiple slice processing circuits 120A-120B that access shared buffers. The share buffers include slice context buffer 130 and shared line buffer 140. In some implementations, the shared slice context buffer 130 and shared line buffer 140 are combined in a single, shared buffer, in other implementations, the shared slice context buffer 130 and shared line buffer 140 are separate buffers as shown.

[0020] In various implementations, external de-rasterizer circuitry generates video data 150 and video data 151 as multiple image data slices. The image is a video frame to be transported across a display link and displayed on a display device. Data transport of compressed video data 160 and compressed video data 161 occurs across the display link. Data transport of compressed video data 160 and compressed video data 161 can also occur between two memories of a computing device. In an implementation, apparatus 100 receives rendered videodata and compresses the received video data using the slice processing circuits 120A-120B. For example, in an implementation, the slice processing circuits 120A-120B receive the video data 150 and video data 151 and generate the compressed video data 160 and the compressed video data 161. The external de-rasterizer circuitry divides the rendered video frame into partitions such as slices, which are processed in raster scan order.

[0021] When the rendered video frame is divided into slices, the rows (or lines) traverse only across the slice, rather than across the entire video frame. An example of three slices labeled Slice 0, Slice 1 and Slice 2 is shown. Each of the Slices 0 to 2 includes three lines labeled Line 0, Line 1, and Line 2. The horizontal arrows of the Slices 0 to 2 illustrate the Lines 0 to 2. In some implementations, the compression algorithm used by pixel compressing circuit 110 processes pixels in groups, such as a group of three pixels, as the compression algorithm traverses across the row. The compression algorithm generates context data for each group of pixels, and the compression algorithm uses the context data for a subsequent neighboring group of pixels. The diagonal arrows of the Slices 0 to 2 illustrate the raster scan order. Additionally, the diagonal arrows illustrate the data dependency between the last group of pixels of a particular line, such as Line 0, and the first group of pixels in the subsequent line such as Line 1 that uses the context data generated during the compression of the last group of pixels of Line 0.

[0022] Each of the raster scan order and the compression dependency order that is based on the generation of context data are shown in the bottom right box. The video data 150 and video data 151 arrive at the pixel compressing circuit 110 following the raster scan order. In various implementations, video data 150 and video data 151 arrive simultaneously in parallel at the pixel processing circuit 110. In a similar manner, compressed video data 160 and compressed video data 161 exit simultaneously in parallel from the pixel processing circuit 110. Crossbar circuitry is not shown for ease of illustration. In other implementations, video data 150 and video data 151 arrive with a non-zero delay between them, but the processing of video data 150 and video data 151 by pixel compressing circuit 110 overlaps.

[0023] Each of the slice processing circuits 120A-120B performs compression based on a selected algorithm. In some implementations, slice processing circuits 120A-120B use the Display Stream Compression (DSC) algorithm. Although two slice processing circuits are shown, pixel compressing circuit 110 can include another number of slice processing circuits. However, to reduce on-die area of pixel compressing circuit 110, the number of slice processing circuits can be limited and yet still achieve high performance based on scheduling the slice processing circuits 120A-120B in a manner to maintain high throughput.

[0024] Although the rendered video frame is shown as divided into three slices, another number of slices can be used based on design requirements. However, in various implementations, the number of slice processing circuits used by pixel compressing circuit 110 does not match the number of slices of the input rendered video frame. In the illustrated implementation, there is an even number of slice processing circuits used by pixel compressing circuit 110, whereas the input rendered video frame includes an odd number of slices. In other implementations, there is an odd number of slice processing circuits used by pixel compressing circuit 110, whereas the input rendered video frame includes an even number of slices. Further details of such an implementation are provided in the description of apparatus 700 (of FIG. 7).

[0025] It is noted that whether pixel compressing circuit 110 has an even number of slice processing circuits or an odd number of slice processing circuits, scheduling the slice processing circuits 120A-120B in a manner to maintain high throughput occurs when the number of slices of the video frame is greater than or equal to the number of slice processing circuits utilized in pixel compressing circuit 110. When the number of slices of the video frame is greater than or equal to the number of slice processing circuits utilized in pixel compressing circuit 110, the operating clock frequency of slice processing circuits 120A-120B can be a clock frequency obtained by dividing an operating clock frequency of an external de-rasterizer circuit by the number of slice processing circuits utilized in pixel compressing circuit 110. Further details of these cases and setting the operating clock frequencies of the slice processing circuits 120A-120B are provided in the description of timing diagrams 500-600 and 800 (of Figures 5-6 and 8).

[0026] In some implementations, each of slice processing circuits 120A-120B can perform the DSC algorithm on an assigned row of a slice at a time. When executing the compression algorithm, such as the DSC algorithm, each of slice processing circuits 120A-120B generates context data for each group of pixels. Slice processing circuits 120A-120B store the generated context data of the group of pixels in the shared slice context buffer 130. The shared slice context buffer 130 and shared line buffer 140 can use one of a variety of data storage arrangements for storing shared data such as flip-flop circuits, latches, registers, one of a variety of types of random-access memory (RAM), content addressable memory (CAM), and so forth.

[0027] Slice processing circuits 120A-120B use the context data stored in shared slice context buffer 130. In an implementation, pixel compressing circuit 110 receives Line 0 of Slice 0 as video data 150 and assigns video data 150 to slice processing circuit 120A. In some implementations, in parallel, pixel compressing circuit 110 receives Line 0 of Slice 1 as video data 151 and assigns video data 151 to slice processing circuit 120B. In other implementations,pixel compressing circuit 110 receives Line 0 of Slice 1 as video data 151 a non-zero delay after pixel compressing circuit 110 receives Line 0 of Slice 0 as video data 150. Further details of the timing of reception of video data are provided in the description of timing diagrams 500 and 800 (of Figures 5 and 8). Regardless of the actual timing of reception of video data, slice processing circuit 120A has not yet completed compressing Line 0 of Slice 0 as video data 150 when pixel compressing circuit 110 assigns Line 0 of Slice 1 as video data 151 to slice processing circuit 120B. Pixel compressing circuit 110 continues to receive video data in raster scan order.

[0028] At a later time, as pixel compressing circuit 110 continues to receive video data in raster scan order, pixel compressing circuit 110 receives Line 1 of Slice 2 as video data 151 and assigns video data 151 to slice processing circuit 120A. To process (compress) the first group of pixels of Line 1 of Slice 2, slice processing circuit 120A accesses shared slice context buffer 130 to retrieve context data of the last group of pixels of Line 0 of Slice 2. In some implementations, slice processing circuit 120B processed Line 0 of Slice 2 at an earlier time, generated this context data of the last group of pixels of Line 0 of Slice 2, and stored this context data in shared slice context buffer 130. Therefore, it is unnecessary for slice processing circuit 120A to have processed Line 0 of Slice 2 earlier in order to now process Line 1 of Slice 2. In other words, slice processing circuits 120A-120B do not rely on internally stored context data for processing video data 150.

[0029] Additionally, in various implementations, slice processing circuits 120A-120B access shared line buffer 140 for reconstructed pixels of a row that was processed (compressed) earlier. Therefore, in an implementation, pixel compressing circuit 110 receives Line 2 of Slice 0 as video data 150 and assigns this video data to slice processing circuit 120B. To process (compress) the pixel data of Line 2 of Slice 0, slice processing circuit 120B accesses shared line buffer 140 for reconstructed pixel data of Line 1 of Slice 0. It is possible and contemplated that slice processing circuit 120A processed the pixel data of Line 1 of Slice 0 at an earlier time. Therefore, it is unnecessary for slice processing circuit 120B to have processed Line 1 of Slice 0 earlier in order to now process Line 2 of Slice 0. In other words, slice processing circuits 120A-120B do not rely on internally stored reconstructed pixel data for processing video data 150. It is noted that for each row of a slice except the first row, slice processing circuits 120A- 120B access the shared line buffer 140 to obtain a copy of the reconstructed pixels of a previous row. There is no previous row for the first row of each slice.

[0030] By providing access, via the shared slice context buffer 130 and shared line buffer 140, to the different types of data previously generated during data compression of pixel data of avideo frame, pixel compressing circuit 110 can select any available one of slice processing circuits 120A-120B to process (compress) any row of any slice of the video frame. It is unnecessary for the pixel compressing circuit 110 to assign rows of a particular slice to only the slice processing circuit that has been processing the same particular slice. Selecting any available one of slice processing circuits 120A-120B to process (compress) any row of any slice of the video frame reduces the number of instantiations of the slice processing circuits 120A-120B and reduces the operating clock frequencies of the slice processing circuits 120A- 120B. Therefore, on-die area reduces, power consumption reduces, but performance increases as scheduling becomes less complicated and data processing begins sooner.

[0031] In some implementations, apparatus 100 is in a user’s computing device such as one of a desktop computer, a laptop computer, a server computer, a tablet computer, a smartphone, a gaming device, a smartwatch, and so on. In some implementations, pixel compressing circuit 110 sends compressed video data 160 and compressed video data 161 to a bitstream construction circuitry, which sends output data to re -rasterizing circuitry. In an implementation, apparatus 100 is used to send the generated compressed video data to a display link. In some implementations, the display link is a video cable. Each of the user’s computing device, the display link, and the display device supports a chosen communication protocol such as DisplayPort, HDMI (High-Definition Multimedia Interface), and so forth. In another implementation, apparatus 100 is also used to send the generated compressed video data between two memories of the computing device.

[0032] Referring to FIG. 2, a generalized diagram is shown of a method 200 for efficiently processing video data for data transport. For purposes of discussion, the steps in this implementation (as well as Figure 3) are shown in sequential order. However, in other implementations some steps occur in a different order than shown, some steps are performed concurrently, some steps are combined with other steps, and some steps are absent.

[0033] A pixel compressing circuit assigns the first row (or first line) of a first partition to a first processing circuit (block 202). The first processing circuit processes the first row of the first partition (block 204). The pixel compressing circuit assigns the first row of a second partition to a second processing circuit (block 206). In various implementations, each of the first processing circuit and the second processing circuit is a slice processing circuit of a pixel compressing circuit. Therefore, each of the first processing circuit and the second processing circuit processes a row of video data based on a selected compression algorithm. In some implementations, the compression algorithm is the DSC algorithm, each of the partitions is a slice of a video frame. The second processing circuit processes the first row of the secondpartition (block 208). During the processing of the first row of the first partition, in various implementations, the first processing circuit generates context data for each group of pixels (e.g., each group of 3 pixels) and stores this context data in a shared context buffer. In a similar manner, during the processing of the first row of the second partition, the second processing circuit generates context data for each group of pixels and stores this context data in the shared context buffer. Additionally, the first processing circuit and the second processing circuit generate reconstructed pixels of a corresponding row and stores this data in a shared line buffer.

[0034] The pixel compressing circuit assigns the first row of a third partition to the first processing circuit (block 210). The first processing circuit processes the first row of the third partition by (block 212). During the processing of the first row of the third partition, in various implementations, the first processing circuit generates context data for each group of pixels and stores this context data in the shared context buffer. The first processing circuit generates reconstructed pixels of the first row of the third partition and stores this data in the shared line buffer. It is noted that for the first row of any partition, such as the third partition, the first processing circuit does not access the shared line buffer to obtain a copy of the reconstructed pixels of a previous row. There is no previous row for the first row of any partition. The pixel compressing circuit assigns a second row of the first partition to the second processing circuit (block 214). The second processing circuit accesses the shared context buffer to retrieve context data of the first row of the first partition generated by the first processing circuit (block 216).

[0035] The second processing circuit accesses the shared line buffer to retrieve reconstructed image data of the first row of the first partition previously generated by the first processing circuit, rather than previously generated by the second processing circuit (block 218). The second processing circuit processes the second row of the first partition using the retrieved context data (block 220). With the use of the shared context buffer and the shared line buffer, it is unnecessary for the second slice processing circuit to have processed rows of the first partition at an earlier time in order to now process the second row of the first partition. In other words, the first slice processing circuit and the second slice processing circuit do not rely on internally stored context data or internally stored reconstructed image data for processing video data. During the processing of the second row of the first partition, the second processing circuit generates context data for each group of pixels and stores this context data in the shared context buffer. The second processing circuit generates reconstructed pixels of the second row of the first partition and stores this data in the shared line buffer.

[0036] Selecting any available one of the first slice processing circuit and the second slice processing circuit to process (compress) any row of any partition of the video frame reduces the number of instantiations of the slice processing circuits and reduces the operating clock frequencies of the slice processing circuits. Therefore, on-die area reduces, power consumption reduces, but performance increases as scheduling becomes less complicated and data processing begins sooner. Although method 200 uses two slice processing circuits, in various implementations, the number of slice processing circuits does not match the number of partitions of the input rendered video frame and there can be another even number of slice processing circuits, whereas the input rendered video frame includes an odd number of partitions. In other implementations, there is an odd number of slice processing circuits and the video frame includes an even number of partitions.

[0037] Referring to FIG. 3, a generalized diagram is shown of a method 300 for efficiently processing video data for data transport. A video processing circuit includes multiple components such as at least de-rasterizer circuitry and a pixel compressing circuit. The de- rasterizer circuitry generates multiple image data slices (block 302). The video processing circuit selects a slice of multiple slices and a row of the slice for data compression by the pixel compressing circuit (block 304). The pixel compressing circuit includes multiple slice processing circuits. In various implementations, the number of slice processing circuits does not match the number of slices of the input image data. It is noted that whether the pixel compressing circuit has an even number of slice processing circuits or an odd number of slice processing circuits, scheduling the slice processing circuits in a manner to maintain high throughput occurs when the number of slices of the video frame is greater than or equal to the number of slice processing circuits utilized in the pixel compressing circuit.

[0038] In an implementation, the pixel compressing circuit includes two slice processing circuits and the input image data uses three slices. If no slice processing circuit of the multiple slice processing circuits of the pixel compressing circuit is available (“no” branch of the conditional block 306), then the video processing circuit waits for an available slice processing circuit (block 308). However, by setting the operating clock frequency of the slice processing circuits to a clock frequency obtained by dividing an operating clock frequency of the external de-rasterizer circuit by the number of slice processing circuits utilized in pixel compressing circuit, at least one slice processing circuit will be available each time a row of a slice is ready to be scheduled. Further details of these cases and setting the operating clock frequencies of the slice processing circuits 120A-120B are provided in the description of timing diagrams 500-600 and 800 (of Figures 5-6 and 8).

[0039] If any of multiple slice processing circuits of the pixel compressing circuit are available (“yes” branch of the conditional block 306), then the pixel compressing circuit selects the available slice processing circuit different from a slice processing circuit used earlier for processing image data of the row of the slice (block 310). The pixel compressing circuit assigns the slice to the available slice processing circuit (block 312). The available (and assigned) slice processing circuit processes (compresses) the slice by accessing information previously generated during earlier processing of image data of the slice (block 314).

[0040] Turning now to FIG. 4, a generalized diagram is shown of a computing system 400 that efficiently processes video data for transport. In an implementation, the computing system 400 includes at least processing circuits 402 and 410, input / output (I / O) interfaces 420, bus 425, network interface 435, memory controllers 430, memory devices 440, display controller 460, and display 465. In other implementations, computing system 400 includes other components and / or computing system 400 is arranged differently. For example, power management circuitry, and phased locked loops (PLLs) or other clock generating circuitry are not shown for ease of illustration. In various implementations, the components of the computing system 400 are on the same die such as a system-on-a-chip (SOC). In other implementations, the components are individual dies in a system-in-package (SiP) or a multi -chip module (MCM). A variety of computing devices use the computing system 400 such as a desktop computer, a laptop computer, a server computer, a tablet computer, a smartphone, a gaming device, a smartwatch, and so on.

[0041] Processing circuits 402 and 410 are representative of any number of processing circuits which are included in computing system 400. In an implementation, processing circuit 410 is a general -purpose central processing unit (CPU). In one implementation, processing circuit 402 is a parallel data processing circuit with a highly parallel data microarchitecture, such as a GPU. The processing circuit 402 can be a discrete device, such as a dedicated GPU (dGPU), or the processing circuit 402 can be integrated (an iGPU) in the same package as another processing circuit. Other parallel data processing circuits that can be included in computing system 400 include digital signal processing circuits (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and so forth.

[0042] In various implementations, processing circuit 402 includes multiple, replicated compute circuits 404A-404N, each including similar circuitry and components such as single- instruction-multiple-data (SIMD) circuits, one or more caches, and so forth. The hardware of scheduler 405 assigns wavefronts to be dispatched to the compute circuits 404A-404N. In an implementation, scheduler 405 is a command processing circuit of a GPU. In someimplementations, the application 446 stored on the memory devices 440 and its copy (application 416) stored on the memory 412 are a highly parallel data application that provide multiple kernels (function calls) to be executed on the compute circuits 404A-404N. The high parallelism offered by the hardware of the compute circuits 404A-404N is used for real-time data processing. Examples of real-time data processing are rendering multiple pixels, image blending, pixel shading, vertex shading, and geometry shading. In such cases, each of the data items of a wavefront is a pixel of an image. The compute circuits 404A-404N can also be used to execute other threads that require operating simultaneously with a relatively high number of different data elements (or data items). Examples of these threads are threads for scientific, medical, finance and encryption / decryption computations.

[0043] Memory 412 represents a local hierarchical cache memory subsystem. Memory 412 stores source data, intermediate results data, results data, and copies of data and instructions stored in memory devices 440. Processing circuit 410 is coupled to bus 425 via interface 409. Processing circuit 410 receives, via interface 409, copies of various data and instructions, such as the operating system 442, one or more device drivers, one or more applications such as application 446, and / or other data and instructions. The hardware, such as circuitry 418, of processing circuit 410 retrieves a copy of the application 446 from the memory devices 440, and circuitry 418 stores this copy as application 416 in memory 412.

[0044] In some implementations, computing system 400 utilizes a communication fabric (“fabric”), rather than the bus 425, for transferring requests, responses, and messages between the processing circuits 402 and 410, the I / O interfaces 420, the memory controllers 430, the network interface 435, and the display controller 460. When messages include requests for obtaining targeted data, the circuitry of interfaces within the components of computing system 400 translates target addresses of requested data. In some implementations, the bus 425, or a fabric, includes circuitry for supporting communication, data transmission, network protocols, address formats, interface signals and synchronous / asynchronous clock domain usage for routing data.

[0045] Memory controllers 430 are representative of any number and type of memory controllers accessible by processing circuits 402 and 410. While memory controllers 430 are shown as being separate from processing circuits 402 and 410, it should be understood that this merely represents one possible implementation. In other implementations, one of memory controllers 430 is embedded within one or more of processing circuits 402 and 410 or it is located on the same semiconductor die as one or more of processing circuits 402 and 410. Memory controllers 430 are coupled to any number and type of memory devices 440.

[0046] Memory devices 440 are representative of any number and type of memory devices. For example, the type of memory in memory devices 440 includes Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), NAND Flash memory, NOR flash memory, Ferroelectric Random Access Memory (FeRAM), or otherwise. Memory devices 440 store at least instructions of an operating system 442, one or more device drivers, and application 446. In some implementations, application 446 is a highly parallel data application such as a video graphics application, a shader application, or other. Copies of these instructions can be stored in a memory or cache device local to processing circuit 410 and / or processing circuit 402.

[0047] I / O interfaces 420 are representative of any number and type of I / O interfaces (e.g., peripheral component interconnect (PCI) bus, PCI-Extended (PCI-X), PCIE (PCI Express) bus, gigabit Ethernet (GBE) bus, universal serial bus (USB). Various types of peripheral devices (not shown) are coupled to I / O interfaces 420. Such peripheral devices include (but are not limited to) displays, keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, and so forth. Network interface 135 receives and sends network messages across a network. In some implementations, display controller 460 receives rendered pixel data from processing circuit 402 via bus or fabric 425. In various implementations, pixel compressing circuit 462 of display controller 460 includes an apparatus with the same functionality as apparatus 100 (of FIG. 1).

[0048] Display 465 supports high resolution and high refresh rates, which increases the data bandwidth of display link 464. In some implementations, the data bandwidth for transporting video data also increases on bus or fabric 425. However, pixel compressing circuit 462 includes slice processing circuits with the same functionality as slice processing circuits 120A-120B (of FIG. 1). Therefore, pixel compressing circuit 462 has the same functionality as pixel compressing circuit 110 (of FIG. 1), which reduces on-die area, reduces power consumption, but increases performance as scheduling becomes less complicated and data processing begins sooner. Each of pixel compressing circuit 110 (of FIG. 1) and pixel compressing circuit 462 can be used to compress (encode) video data while reducing power consumption and maintaining high throughput as shown in timing diagrams 500, 600 and 800 (of Figures 5, 6 and 8). In some implementations, one or more of processing circuit 402, processing circuit 410, and I / O interfaces 420 includes an instantiated copy of the hardware, such as circuitry, of pixel compressing circuit 462.

[0049] In various implementations, display 465 includes decoder 466 that performs the reverse operation of the data compression operation performed by pixel compressing circuit 462. Insome implementations, decoder 466 performs data decompression in a similar manner as the slice processing circuits 120A-120B perform data compression. For example, decoder 466 includes separate slice processing circuits that perform data decompression and access a shared context buffer and a shared line buffer during data decompression. Assigning compressed video data to these slice processing circuits of decoder 466 occurs in a similar manner as assignments of the slice processing circuits 120A-120B.

[0050] It is noted that in some implementations, one or more of the components of computing system 400, such as pixel compressing circuit 462 and other components, is implemented on a chiplet. As used herein, a “chiplet” is also referred to as an “intellectual property block” (or IP block). However, a “chiplet” is a semiconductor die (or die) fabricated separately from other dies, and then interconnected with these other dies in a single integrated circuit in the MCM. On a single silicon wafer, only multiple chiplets are fabricated as multiple instantiated copies of particular integrated circuitry, rather than fabricated with other functional blocks that do not use an instantiated copy of the particular integrated circuitry. For example, the chiplets are not fabricated on a silicon wafer with various other functional blocks and processors on a larger semiconductor die such as an SoC. A first silicon wafer (or first wafer) is fabricated with multiple instantiated copies of integrated circuitry a first chiplet, and this first wafer is diced using laser cutting techniques to separate the multiple copies of the first chiplet. A second silicon wafer (or second wafer) is fabricated with multiple instantiated copies of integrated circuitry of a second chiplet, and this second wafer is diced using laser cutting techniques to separate the multiple copies of the second chiplet.

[0051] Turning now to FIG. 5, a generalized diagram is shown of a timing diagram 500 for efficiently processing video data for data transport. Timing diagram 500 includes different types of data and respective transitions over time. De-rasterizer input 510 is rendered video data that arrives from a parallel data processing circuit at a de-rasterizer circuit. The parallel data processing circuit can be one of a GPU, a DSP, a FPGA, an ASIC, or other. The de- rasterizer circuit can be one of a variety of de-rasterizer circuits used to divide a video frame into slices. Based on de-rasterizer input 510 and received control signals (not shown), the de- rasterizer circuit generates each of de-rasterizer output 520 and de-rasterizer output 530 to send to pixel compressing circuitry such as pixel compressing circuit 110 (of FIG. 1) and pixel compressing circuit 462 (of FIG. 4). In an implementation, the de-rasterizer circuit receives control signals indicating that the video frame is to be divided into two slices.

[0052] At the point in time tO (or time tO), the de-rasterizer circuit receives a first line labeled “Line 0” of a rendered video frame. This first line includes rendered pixel data that can bedivided into two slices. Although no delay is shown, the de-rasterizer circuit generates de- rasterizer output 520, which corresponds to the first slice (SO) of Line 0, a short delay after time tO. The de-rasterizer circuit begins generating de-rasterizer output 530, which corresponds to the second slice (SI) of Line 0, at time tl. As shown, in various implementations, the de- rasterizer circuit generates de-rasterizer output 520 and de-rasterizer output 530 with an output frequency that is smaller than the input frequency of de-rasterizer input 510. In an implementation, the output frequency is the input frequency divided by the number of slice processing circuits used by the pixel compressing circuit.

[0053] When the pixel compressing circuit includes two slice processing circuits, the output frequency of de-rasterizer output 520 and de-rasterizer output 530 is one half of the input frequency of de-rasterizer input 510. The operating clock frequency can be reduced, since there are two slice processing circuits performing compression. Therefore, power consumption can be reduced while performance remains at a level achieved with the input frequency of de- rasterizer input 510. An overlap of data processing (data compression) occurs beginning at time tl. In an implementation, a first slice processing circuit is compressing rendered video data of Line 0 and Slice 0 (L0, SO), and at time tl, a second slice processing circuit begins compressing rendered video data of Line 0 and Slice 1 (L0, SI). At time t2, the first slice processing circuit is compressing rendered video data of Line 1 and Slice 0 (LI, SO). Therefore, the first slice processing circuit continues to compress the first slice (SO) of the lines (L0, LI, L2).

[0054] Although no delay is shown, a re-rasterizer circuit generates re-rasterizer output 540, which corresponds to the compressed video data of the first slice (SO) of Line 0, a short delay after time t2. In other implementations, the re-rasterizer output 540 is provided a short delay after time tl as soon as the corresponding video data is compressed. The re-rasterizer circuit performs the opposite steps of de-rasterizer circuit and reassembles the video data so as not to include any divided slices. Any of a variety of re-rasterizer circuits can be used to generate re- rasterizer output 540. However, the video data is compressed. The re-rasterizer circuit generates re-rasterizer output 540 with a frequency equal to the input frequency of de-rasterizer input 510. Therefore, the dividing of the video frame into slices and reassembling the slices occurs at a higher operating clock frequency than the operating clock frequency of the slice processing circuits performing data compression.

[0055] It is noted that scheduling the slice processing circuits in a manner to maintain high throughput occurs when the number of slices of the video frame is greater than or equal to the number of slice processing circuits utilized in the pixel compressing circuit. When the number of slices of the video frame is greater than or equal to the number of slice processing circuitsutilized in pixel compressing circuit, the operating clock frequency of slice processing circuits can be a clock frequency obtained by dividing an operating clock frequency of an external de- rasterizer circuit by the number of slice processing circuits utilized in pixel compressing circuit. In the illustrated implementation, there are two slice processing circuits receiving the video data of de-rasterizer output 520 and de-rasterizer output 530. Therefore, the operating clock frequencies of the two slice processing circuits are obtained by dividing the operating clock frequency of the external de-rasterizer circuits by two.

[0056] Turning now to FIG. 6, a generalized diagram is shown of a timing diagram 600 for efficiently processing video data for data transport. Timing diagram 600 includes different types of data and respective transitions over time. De-rasterizer input 610 is rendered video data that arrives from a parallel data processing circuit at a de-rasterizer circuit. Based on de- rasterizer input 610 and received control signals (not shown), the de-rasterizer circuit generates each of de-rasterizer output 620 and de-rasterizer output 630 to send to pixel compressing circuitry such as pixel compressing circuit 110 (of FIG. 1) and pixel compressing circuit 462 (of FIG. 4). In an implementation, the de-rasterizer circuit receives control signals indicating that the video frame is to be divided into three slices, rather than two slices.

[0057] At the point in time tO (or time tO), the de-rasterizer circuit receives a first line labeled “Line 0” of a rendered video frame. This first line includes rendered pixel data that can be divided into three slices. Although no delay is shown, the de-rasterizer circuit generates de- rasterizer output 620, which corresponds to the first slice (SO) of Line 0, a short delay after time tO. The de-rasterizer circuit begins generating de-rasterizer output 630, which corresponds to compressed video data of the second slice (SI) of Line 0, at time tl. In other implementations, the re-rasterizer output 640 is provided a short delay after time tl as soon as the corresponding video data is compressed. As shown, in various implementations, the de- rasterizer circuit generates de-rasterizer output 620 and de-rasterizer output 630 with an output frequency that is smaller than the input frequency of de-rasterizer input 610. In an implementation, the output frequency is the input frequency divided by the number of slice processing circuits used by the pixel compressing circuit.

[0058] When the pixel compressing circuit includes two slice processing circuits, the output frequency of de-rasterizer output 620 and de-rasterizer output 630 is one half of the input frequency of de-rasterizer input 610 although the video frame is divided into three slices. The operating clock frequency can be reduced, since there are two slice processing circuits performing compression. Therefore, power consumption can be reduced while performance remains at a level achieved with the input frequency of de-rasterizer input 610. An overlap ofdata processing (data compression) occurs beginning at time tl. In an implementation, a first slice processing circuit is compressing rendered video data of Line 0 and Slice 0 (LO, SO), and at time tl, a second slice processing circuit begins compressing rendered video data of Line 0 and Slice 1 (LO, SI).

[0059] At time t2, the first slice processing circuit is compressing rendered video data of Line 0 and Slice 2 (LO, S2). Therefore, the first slice processing circuit continues to compress video data of the first line “Line 0.” At time t3, the second slice processing circuit is compressing rendered video data of Line 1 and Slice 0 (LI, SO). Therefore, the second slice processing circuit has begun compressing the first slice (SO) of a line (“Line 1” or LI, in this case). This assignment works, because the pixel compressing circuit uses shared buffers to store context data for each group of pixels and reconstructed pixels of a row that was processed (compressed) earlier. For example, the pixel compressing circuit uses shared buffers similar to buffers 130 and 140 (of FIG. 1).

[0060] Although no delay is shown, a re-rasterizer circuit generates re-rasterizer output 640 beginning with the first slice (SO) of Line 0, a short delay after time tl and completing the corresponding re-rasterizer output for the first slice (SO) of Line 0 a short delay after time t2. The re-rasterizer circuit performs the opposite steps of de-rasterizer circuit and reassembles the video data so as not to include any divided slices. The re-rasterizer circuit generates re- rasterizer output 640 with a frequency equal to the input frequency of de-rasterizer input 610. Therefore, the dividing of the video frame into slices and reassembling the slices occurs at a higher operating clock frequency than the operating clock frequency of the slice processing circuits performing data compression.

[0061] Referring to FIG. 7, a generalized diagram is shown of an apparatus 700 for efficiently processing video data for data transport. Circuitry and components described earlier are numbered identically. In the illustrated implementation, apparatus 700 includes pixel processing circuit 710 with encoders 720, slice context buffer 130, and shared line buffer 140. Encoders 720 includes multiple slice processing circuits 120A-120C that access slice context buffer 130 and shared line buffer 140. Slice processing circuit 120C includes an instantiated copy of the circuitry of slice processing circuits 120A-120B. Encoders 720 receives video data 150, 151, and 752 and generates compressed video data 160, 161, and 762. In the illustrated implementation, external de-rasterizer circuitry divides a rendered video frame into four slices labeled Slice 0, Slice 1, Slice 2, and Slice 3, which are processed in raster scan order.

[0062] The number of slice processing circuits used by pixel compressing circuit 710 does not match the number of slices of the input rendered video frame. It is noted that scheduling theslice processing circuits 120A- 120C in a manner to maintain high throughput occurs when the number of slices of the video frame is greater than or equal to the number of slice processing circuits 120A-120C utilized in the pixel compressing circuit 710. When the number of slices of the video frame is greater than or equal to the number of slice processing circuits 120A- 120C utilized in pixel compressing circuit 710, the operating clock frequency of slice processing circuits 120A-120C can be a clock frequency obtained by dividing an operating clock frequency of an external de-rasterizer circuit by the number of slice processing circuits 120A-120C utilized in pixel compressing circuit 710.

[0063] In the illustrated implementation, there is an odd number of slice processing circuits used by pixel compressing circuit 710, whereas the input rendered video frame includes an even number of slices. When processing (compressing) slices of the video frame received as video data 150, 151 and 752, the slice processing circuits 120A-120C access slice context buffer 130 and shared line buffer 140. Therefore, it is unnecessary for slice processing circuits 120A-120C to have processed lines of a particular slice ahead of time before processing currently received lines of the particular slice. Pixel compressing circuit 710 can select any available one of slice processing circuits 120A-120C to process (compress) any row of any slice of the video frame. It is unnecessary for the pixel compressing circuit 110 to assign rows of a particular slice to only the slice processing circuit that has been processing the same particular slice. Selecting any available one of slice processing circuits 120A-120C to process (compress) any row of any slice of the video frame reduces the number of instantiations of the slice processing circuits 120A-120C and reduces the operating clock frequencies of the slice processing circuits 120A-120C. Therefore, on-die area reduces, power consumption reduces, but performance increases as scheduling becomes less complicated and data processing begins sooner.

[0064] Turning now to FIG. 8, a generalized diagram is shown of a timing diagram 800 for efficiently processing video data for data transport. Timing diagram 800 includes different types of data and respective transitions over time. Circuitry, components, and data types described earlier are numbered identically. In an implementation, the de-rasterizer circuit receives control signals indicating that the video frame is to be divided into four slices. Based on de-rasterizer input 610 and received control signals (not shown), the de-rasterizer circuit generates each of de-rasterizer output 620, de-rasterizer output 630 and de-rasterizer output 830 to send to pixel compressing circuitry such as pixel compressing circuit 110 (of FIG. 1), pixel compressing circuit 462 (of FIG. 4), and pixel compressing circuit 710 (of FIG. 7). In an implementation, the de-rasterizer circuit sends de-rasterizer output 620, de-rasterizer output630 and de-rasterizer output 830 to three slice processing circuits such as slice processing circuits 120A-120C of pixel compressing circuit 710.

[0065] At the point in time tO (or time tO), the de-rasterizer circuit receives a first line labeled “Line 0” of a rendered video frame. This first line includes rendered pixel data that can be divided into two slices. Although no delay is shown, the de-rasterizer circuit generates de- rasterizer output 620, which corresponds to the first slice (SO) of Line 0, a short delay after time tO. The de-rasterizer circuit begins generating de-rasterizer output 630, which corresponds to the compressed second slice (S 1) of Line 0, at time tl . As shown, in various implementations, the de-rasterizer circuit generates de-rasterizer output 620 and de-rasterizer output 630 with an output frequency that is smaller than the input frequency of de-rasterizer input 610. In an implementation, the output frequency is the input frequency divided by the number of slice processing circuits used by the pixel compressing circuit. Although the clock frequency is shown as halved in the illustrated implementation, the clock frequency can be divided by three due to the pixel compressing circuit using three slice processing circuits.

[0066] At time t2, the first slice processing circuit is compressing rendered video data of Line 1 and Slice 1 (LI, SI). Therefore, the first slice processing circuit compresses video data of a slice that it has not yet compressed before. This assignment works, because the pixel compressing circuit uses shared buffers to store context data for each group of pixels and reconstructed pixels of a row that was processed (compressed) earlier. For example, the pixel compressing circuit uses shared buffers similar to buffers 130 and 140 (of FIG. 1). The re- rasterizer circuit generates re-rasterizer output 840 with a frequency equal to the input frequency of de-rasterizer input 610.

[0067] It is noted that one or more of the above-described implementations include software. In such implementations, the program instructions that implement the methods and / or mechanisms are conveyed or stored on a computer readable medium. Numerous types of media which are configured to store program instructions are available and include hard disks, floppy disks, CD-ROM, DVD, flash memory, Programmable ROMs (PROM), random access memory (RAM), and various other forms of volatile or non-volatile storage. Generally speaking, a computer accessible storage medium includes any storage media accessible by a computer during use to provide instructions and / or data to the computer. For example, a computer accessible storage medium includes storage media such as magnetic or optical media, e g., disk (fixed or removable), tape, CD-ROM, or DVD-ROM, CD-R, CD-RW, DVD-R, DVD-RW, or Blu-Ray. Storage media further includes volatile or non-volatile memory media such as RAM (e.g., synchronous dynamic RAM (SDRAM), double data rate (DDR, DDR2,DDR3, etc.) SDRAM, low-power DDR (LPDDR2, etc.) SDRAM, Rambus DRAM (RDRAM), static RAM (SRAM), etc.), ROM, Flash memory, non-volatile memory (e.g., Flash memory) accessible via a peripheral interface such as the Universal Serial Bus (USB) interface, etc. Storage media includes microelectromechanical systems (MEMS), as well as storage media accessible via a communication medium such as a network and / or a wireless link.

[0068] Additionally, in various implementations, program instructions include behavioral- level descriptions or register-transfer level (RTE) descriptions of the hardware functionality in a high-level programming language such as C, or a design language (HDL) such as Verilog, VHDL, or database format such as GDS II stream format (GDSII). In some cases, the description is read by a synthesis tool, which synthesizes the description to produce a netlist including a list of gates from a synthesis library. The netlist includes a set of gates, which also represent the functionality of the hardware including the system. The netlist is then placed and routed to produce a data set describing geometric shapes to be applied to masks. The masks are then used in various semiconductor fabrication steps to produce a semiconductor circuit or circuits corresponding to the system. Alternatively, the instructions on the computer accessible storage medium are the netlist (with or without the synthesis library) or the data set, as desired. Additionally, the instructions are utilized for purposes of emulation by a hardware -based type emulator from such vendors as Cadence®, EVE®, and Mentor Graphics®.

[0069] Although the implementations above have been described in considerable detail, numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such variations and modifications.

Claims

WHAT IS CLAIMED IS1. An apparatus comprising: circuitry configured to: compress, by a first processing circuit of a plurality of processing circuits, first video data of a first partition of a plurality of partitions of a video frame; and compress, by a second processing circuit of a plurality of processing circuits, second video data of the first partition of the plurality of partitions of the video frame, wherein the second processing circuit has access to information previously generated by the first processing circuit.

2. The apparatus as recited in claim 1, wherein the circuitry is further configured to convey, via a display link, the first video data to a display device from the first processing circuit, responsive to the first processing circuit having completed compressing the first video data.

3. The apparatus as recited in claim 1, wherein the circuitry is further configured to store, in a shared buffer, the information previously generated by the first processing circuit comprising context data of a compression algorithm.

4. The apparatus as recited in claim 3, wherein the circuitry is further configured to store, in the shared buffer, reconstructed pixels of the first video data.

5. The apparatus as recited in claim 1 , wherein the circuitry is further configured to assign the second processing circuit for compressing the second video data of the first partition based on availability.

6. The apparatus as recited in claim 1, wherein the circuitry is further configured to divide the video frame into the plurality of partitions, each comprising a slice of the video frame.

7. The apparatus as recited in claim 6, wherein the circuitry is further configured to insert a single line of a first slice of the video frame into the first video data based on a raster scan order.

8. A method, comprising: storing, by circuitry of a memory, a video frame; compressing, by a first processing circuit of a plurality of processing circuits, first video data of a first partition of a plurality of partitions of a video frame; and compressing, by a second processing circuit of a plurality of processing circuits, second video data of the first partition of the plurality of partitions of the video frame, wherein the second processing circuit has access to information previously generated by the first processing circuit.

9. The method as recited in claim 8, further comprising conveying the first video data to a display device from the first processing circuit, responsive to the first processing circuit has completed compressing the first video data.

10. The method as recited in claim 8, further comprising storing, in a shared buffer, the information previously generated by the first processing circuit comprising context data of a compression algorithm.

11. The method as recited in claim 10, further comprising storing, in the shared buffer, reconstructed pixels of the second video data.

12. The method as recited in claim 8, further comprising assigning the second processing circuit for compressing the second video data of the first partition based on availability.

13. The method as recited in claim 8, further comprising dividing the video frame into the plurality of partitions, each comprising a slice of the video frame.

14. The method as recited in claim 13, further comprising inserting a single line of a first slice of the video frame into the first video data based on a raster scan order.

15. A computing system comprising: a memory comprising circuitry configured to store a video frame; and a video processing circuit comprising circuitry configured to: compress, by a first processing circuit of a plurality of processing circuits, first video data of a first partition of a plurality of partitions of the video frame; and compress, by a second processing circuit of a plurality of processing circuits, second video data of the first partition of the plurality of partitions of the video frame, wherein the second processing circuit has access to information previously generated by the first processing circuit.

16. The computing system as recited in claim 15, wherein the circuitry is further configured to convey, via a display link, the first video data to a display device from the first processing circuit, responsive to the first processing circuit has completed compressing the first video data.

17. The computing system as recited in claim 15, wherein the circuitry is further configured to store, in a shared buffer, the information previously generated by the first processing circuit comprising context data of a compression algorithm.

18. The computing system as recited in claim 17, wherein the circuitry is further configured to store, in the shared buffer, reconstructed pixels of the second video data.

19. The computing system as recited in claim 15, wherein the circuitry is further configured to assign the second processing circuit for compressing second video data of the first partition based on availability.

20. The computing system as recited in claim 15, wherein the circuitry is further configured to divide the video frame into the plurality of partitions, each comprising a slice of the video frame.

Images