Application programming interface for allocating memory to store compressed video.

Abstract

This disclosure relates to application programming interfaces for allocating memory to store compressed video, and specifically discloses apparatus, systems, and methods for modifying compressed video. In at least one embodiment, the processor includes circuitry for executing one or more instructions to store information and to store instructions for one or more decompression algorithms for decompressing the information.

Description

Technical Field

[0001] Apparatus, systems, and methods for modifying compressed video. In at least one embodiment, the processor includes circuitry for executing one or more instructions to store information and to store instructions for one or more decompression algorithms for decompressing the information. Background Technology

[0002] Current software for editing compressed video (e.g., adding motion blur to objects) typically involves decompressing video segments from CPU memory, transferring them to GPU memory for pixel-by-pixel modification, and then recompressing the video. This multi-step process increases the risk of errors and degrades performance. Attached Figure Description

[0003] Figure 1 A system for editing compressed video stored in GPU memory is shown according to at least one embodiment;

[0004] Figure 2A This is a flowchart illustrating, according to at least one embodiment, the process of modifying a compressed video while storing the video in GPU memory;

[0005] Figure 2B This is a block diagram illustrating an overview of a system for connecting multiple APIs for reading from and writing compressed video, according to at least one embodiment.

[0006] Figure 3 A block diagram illustrating a GPU memory allocation operation (operation) according to at least one embodiment is provided;

[0007] Figure 4 The process of allocating memory on the GPU to store compressed video is shown;

[0008] Figure 5 This is a block diagram illustrating a video texture object creation operation (“operation”) according to at least one embodiment;

[0009] Figure 6 It is a flowchart illustrating the process of creating a video texture object according to at least one embodiment;

[0010] Figure 7 This is a block diagram illustrating a video surface object creation operation (“operation”) according to at least one embodiment.

[0011] Figure 8 This is a flowchart of the process of creating a video surface object according to at least one embodiment;

[0012] Figure 9A block diagram illustrating a texture reading operation (“operation”) according to at least one embodiment is provided;

[0013] Figure 10 This is a flowchart of a process for obtaining pixel data from compressed frames in a compressed video, according to at least one embodiment;

[0014] Figure 11 This is a block diagram illustrating a surface reading operation (“operation”) according to at least one embodiment;

[0015] Figure 12 This is a flowchart illustrating a process for indicating the properties of one or more pixels in a compressed frame of a compressed video, according to at least one embodiment;

[0016] Figure 13 This is a block diagram illustrating a surface writing operation (“operation”) according to at least one embodiment;

[0017] Figure 14 This is a flowchart illustrating a process for writing pixel data into a specific frame of a compressed video, according to at least one embodiment;

[0018] Figure 15 A block diagram of a driver and / or runtime is shown according to at least one embodiment, the driver and / or runtime including one or more libraries for providing one or more application programming interfaces (APIs);

[0019] Figure 16 An example of a system 1600 according to at least one embodiment is shown. The system 1600 may include software and hardware for performing multiple APIs to access or modify one or more frames of a compressed video, or otherwise perform any of the operations described herein.

[0020] Figure 17 An example data center system according to at least one embodiment is shown;

[0021] Figure 18 A system-on-a-chip (SOC) according to at least one embodiment is shown;

[0022] Figure 19A A parallel processor according to at least one embodiment is shown;

[0023] Figure 19B A processing cluster according to at least one embodiment is shown;

[0024] Figure 19C A graphics multiprocessor according to at least one embodiment is shown;

[0025] Figure 20An accelerator processor according to at least one embodiment is shown;

[0026] Figure 21A A central processing unit according to at least one embodiment is shown;

[0027] Figure 21B The illustration shows an embodiment according to at least one of the embodiments. Figure 21A The core of the central processing unit;

[0028] Figure 22 Another accelerator processor according to at least one embodiment is shown;

[0029] Figure 23 A neuromorphic processor according to at least one embodiment is shown;

[0030] Figure 24 A supercomputer according to at least one embodiment is shown;

[0031] Figure 25 Another accelerator processor according to at least one embodiment is shown;

[0032] Figure 26 Another processor according to at least one embodiment is shown;

[0033] Figure 27 Another accelerator processor according to at least one embodiment is shown;

[0034] Figure 28 A tensor processing unit according to at least one embodiment is shown;

[0035] Figure 29 A RISC-V compatible processor according to at least one embodiment is shown;

[0036] Figure 30A and Figure 30B A language processing unit according to at least one embodiment is shown;

[0037] Figure 31 A software stack of a programming platform according to at least one embodiment is shown;

[0038] Figure 32 Software supported by a programming platform according to at least one embodiment is shown;

[0039] Figure 33 A method for using at least one embodiment is shown. Figure 32 Compiled code executed on the programming platform;

[0040] Figure 34 An example of an autonomous vehicle and its system architecture according to at least one embodiment is shown;

[0041] Figure 35A The inference and / or training logic according to at least one embodiment is illustrated;

[0042] Figure 35B The inference and / or training logic according to at least one embodiment is shown; and

[0043] Figure 35C Training and deployment of a neural network according to at least one embodiment are illustrated. Detailed Implementation

[0044] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of at least one embodiment. However, those skilled in the art will understand that the inventive concept can be practiced without one or more of these specific details.

[0045] Systems, methods, and apparatuses for executing APIs to read pixel data from one or more frames of a compressed video and to write pixel data to one or more frames of a compressed video, while the compressed video is stored in compressed form in GPU memory. In at least one embodiment, the compressed video is stored in GPU memory (e.g., global memory), and a specified range of compressed video frames are decompressed to temporary storage (e.g., L2 cache on the GPU) for reading and modification using multiple APIs. In at least one embodiment, modified frames are recompressed and written back to the GPU memory from the temporary storage.

[0046] In at least one embodiment, the processor includes one or more circuitry for executing an application programming interface (API) to store information and instructions for one or more decompression algorithms used to decompress the information. In at least one embodiment, the processor includes one or more circuitry for executing the API to store one or more identifiers of one or more videos and one or more instructions for one or more pixel interpolation algorithms. In at least one embodiment, the processor includes one or more circuitry for executing the API to indicate one or more decompression algorithms to be used to access one or more frames of one or more videos. In at least one embodiment, the processor includes one or more circuitry for executing the API to perform interpolation on two or more pixels of one or more video frames, at least in part, based on one or more frame identifiers of the one or more video frames. In at least one embodiment, the processor includes one or more circuitry for executing the API to indicate one or more properties of the one or more pixels, at least in part, based on one or more identifiers of one or more video frames containing one or more pixels. In at least one embodiment, the processor includes one or more circuits for executing an API to modify one or more properties of the one or more pixels, at least in part, based on one or more identifiers of one or more video frames containing one or more pixels.

[0047] Figure 1 A system 100 for editing compressed video 108 stored in GPU memory is illustrated according to at least one embodiment. In at least one embodiment, the system 100 shows a setup for executing a video host program 104 (e.g., video editing software) on a host 102. In at least one embodiment, the host 102 may be a central processing unit (CPU), for example, combined with... Figure 26 The CPU 2632 is mentioned above. In at least one embodiment, host program 104 is video editing software executed on host 102. In at least one embodiment, device 112 is a GPU, for example, combined with... Figure 19C The GPU mentioned above.

[0048] In at least one embodiment, host 102 performs overall control and logic of the video editing process, such as applying user commands and supervising workflows. In at least one embodiment, host 102 determines which video frames need to be processed and which operations should be applied, while device 112 performs the actual modifications to the video data. In at least one embodiment, host 102 sends instructions to device 112, which utilizes its high-throughput processing capabilities to perform video processing tasks. In at least one embodiment, this division of labor ensures that video editing tasks can be performed efficiently, as host 102 manages high-level operations and coordination, while device 112 performs computationally intensive video processing.

[0049] In at least one embodiment, system 100 improves upon existing software for editing compressed video 108. In at least one embodiment, the existing software involves several inefficient steps. In at least one embodiment, first, a portion of the compressed video 108 is decompressed from host memory 110. In at least one embodiment, next, the decompressed video is uploaded to GPU memory for pixel modification. In at least one embodiment, the video is recompressed after editing is applied. In at least one embodiment, this multi-step process increases the chance of errors, such as timing mismatches between the CPU and GPU, or quality loss due to multiple compressions and decompressions.

[0050] In at least one embodiment, system 100 allows editing of compressed video 118 stored in GPU memory. In at least one embodiment, this is achieved through multiple software components, such as a compressed video loader 106, a video reader 114, and a video writer 116.

[0051] In at least one embodiment, the compressed video loader 106 is a host function, instruction, or API that is executed to allocate memory on device 112 to store the compressed video in memory 120 of device 112. In at least one embodiment, memory 120 may include global memory, shared memory, or other types of memory.

[0052] In at least one embodiment, when a user interacts with the editing software and selects a frame or a range of frames in the compressed video 118 via a user interface, the user's action triggers a command in the editing software that specifies the compressed video and frame range to be edited. In at least one embodiment, this command is passed to device 112, where the device's CUDA driver controls the processing. In at least one embodiment, the CUDA driver corresponds to a combination of... Figure 33 The CUDA driver 3307 is described.

[0053] In at least one embodiment, the CUDA driver uses metadata associated with the compressed video 118 stored in memory 120 to identify the desired frame range. In at least one embodiment, the CUDA driver uses this metadata to retrieve only the specified compressed frames from memory 120. In at least one embodiment, these frames are subsequently decompressed on the GPU of device 112 using dedicated decompression hardware based on a decompression algorithm that is part of the metadata associated with the compressed video. In at least one embodiment, the decompressed frames (e.g., decompressed frame 117) are temporarily stored in cache 122 of device 112. In at least one embodiment, cache 122 may be one of a plurality of caches on device 112. In at least one embodiment, cache 122 is an L1 cache, such as in combination with... Figure 21B The L1 cache 2120 is described. In at least one embodiment, cache 122 is an L2 cache, for example, as combined with... Figure 21B The L2 cache described is 2148.

[0054] In at least one embodiment, the decompressed frames 117 remain transient, existing only in cache 122 for the duration during which these frames are edited. In at least one embodiment, the decompressed frames 117 in cache 122 can be rendered and displayed to the user in the interface of editing software. In at least one embodiment, the user can perform various modifications, such as adjusting brightness, contrast, or adding visual effects. In at least one embodiment, these modifications are applied directly to the decompressed frames 117 residing in the cache. In at least one embodiment, when device 112 decompresses a selected range of frames, it first decompresses one or more keyframes within the range. In at least one embodiment, the keyframes serve as reference points, enabling efficient decompression of predicted frames (e.g., P-frames and B-frames).

[0055] In at least one embodiment, both the video reader 114 and the video writer 116 are implemented as kernel programs that execute on the device 112 to provide core functionality for reading from and writing compressed video 118 stored in memory 120 via cache 122. In at least one embodiment, these kernel programs can interact directly with dedicated hardware (e.g., an NVDEC for decompression). In at least one embodiment, the video reader 114 is used to access the compressed video 118. In at least one embodiment, when invoked, the video reader 114 uses a specified frame range and utilizes this information to locate the required frames. In at least one embodiment, the video reader 114 can cause a CUDA driver to directly decompress the compressed video frames within the specified range into cache 122. In at least one embodiment, the video reader 114 uses one or more GPU-based video texture objects to access the decompressed frames.

[0056] In at least one embodiment, the video writer 116 facilitates modifications to video frames. In at least one embodiment, after selected frames are decompressed and stored in cache 122, the video writer 116 applies the desired changes directly to the decompressed frames, such as decompressed frame 117. In at least one embodiment, the video writer 116 uses one or more GPU-based video surface objects to perform these modifications, which allows pixel-level access to the decompressed frames. In at least one embodiment, once the modifications are complete, the video writer invokes the GPU's NVENC hardware encoder to recompress the modified frames, which are then written back to GPU memory (e.g., memory 120), replacing the original compressed data.

[0057] In at least one embodiment, combined Figure 1 At least one of the described features is the use of combination Figures 2A-35C This is achieved by at least one feature of any system and / or process shown and / or described. In at least one embodiment, in combination with Figure 2B At least one of the described features is used to achieve Figures 2A-34 At least a portion of any system and / or process shown and / or described.

[0058] Figure 2A This is a flowchart illustrating a process 200A of modifying compressed video while the compressed video is stored in GPU memory, according to one embodiment. In at least one embodiment, the processor (e.g., combined with...) Figure 1 The described device 112) executes process 200A.

[0059] In at least one embodiment, in step 202, the processor (e.g., combined with) Figure 1 The described host (102) receives compressed video. In at least one embodiment, before receiving the compressed video, the CPU calls an API to allocate memory on the processor to store the compressed video. In at least one embodiment, the API (e.g., cudaMallocVideoArray()) allocates a CUDA array in GPU memory using parameters such as video resolution, codec type, and frame range. In at least one embodiment, the frame range ensures that the memory allocation is aligned with the number of frames to be stored. In at least one embodiment, by allowing this granularity, the API facilitates efficient memory usage, which is crucial for processing large videos. In at least one embodiment, the codec type represents the compression algorithm of the compressed video. In at least one embodiment, this parameter is stored as part of the metadata of the compressed video and used by other APIs to decompress the compressed video. In at least one embodiment, the compressed video and its associated metadata (including the compression algorithm) are stored together in pre-allocated GPU memory.

[0060] In at least one embodiment, in step 204, the processor creates a textured video object for accessing a range of frames in the compressed video. In at least one embodiment, the API (e.g., cudaCreateVideoTextureObject()) is used to define the textured video object, which allows read-only access to the compressed video. In at least one embodiment, the API uses resource descriptors and texture descriptors linked to the allocated CUDA array to configure access parameters. In at least one embodiment, the API includes a frame range parameter that allows the creation of the textured video object to access a range of frames in the compressed video.

[0061] In at least one embodiment, in step 206, the processor accesses pixel data within frames of the range specified by the frame range parameter of the textured video object. In at least one embodiment, the processor calls an API (e.g., videoTex2D()) to retrieve pixel data from within video frames of the range. In at least one embodiment, the API call causes the processor to decompress the compressed frames of the range into a cache. In at least one embodiment, the API performs targeted access to pixels at specific locations within specific frames of the decompressed frames of the range. In at least one embodiment, the API triggers the GPU's decoding hardware to decompress only the specified frames and temporarily store the decompressed data in a cache.

[0062] In at least one embodiment, in step 208, the processor uses an API to create a surface video object to write pixel data into the range of decompressed frames in a cache. In at least one embodiment, the API (e.g., cudaCreateVideoSurfaceObject()) is used to define the surface video object, which enables write access to the decompressed frames. In at least one embodiment, the API uses a resource descriptor similar to the resource descriptor in the texture video object, but configures it for modifiable rather than read-only access. In at least one embodiment, the frame parameter in the resource descriptor defines a range of frames that are accessible for editing, thereby ensuring that modifications are limited to the selected portion of the video.

[0063] In at least one embodiment, in step 210, the processor uses an API to modify pixel data in the decompressed frame. In at least one embodiment, the API (e.g., videosurface2Dwrite()) is used in the kernel to write new pixel data to a specified frame. In at least one embodiment, the API accepts parameters (e.g., x, y, and frame index) that enable precise editing of individual pixels within a frame. In at least one embodiment, by using frame parameters, the API ensures that editing is applied only to the intended frame and does not affect other decompressed frames. In at least one embodiment, changes to one or more decompressed frames are temporarily stored in a cache.

[0064] In at least one embodiment, in step 212, the processor recompresses the range of modified frames and writes the recompressed modified frames back to GPU memory to overwrite the corresponding portion of the original compressed video.

[0065] In at least one embodiment, combined Figure 2A At least one of the described features is utilizing Figure 1 and Figure 3-Figure 35C This is achieved by at least one feature of any system and / or process shown and / or described. In at least one embodiment, it is combined with Figure 2B At least one of the described features is used to achieve Figure 1 and Figures 3-34 At least a portion of any system and / or process shown and / or described.

[0066] Figure 2B This is a block diagram illustrating an overview of a system 200B, according to at least one embodiment, that connects multiple APIs for reading from and writing compressed video. In at least one embodiment, Figure 2BThis demonstrates the relationship between the API, metadata, and GPU resources, which is crucial for efficient video processing and editing.

[0067] In at least one embodiment, system 200B includes a metadata table 214 that serves as a centralized repository for storing basic information about compressed video data. In at least one embodiment, the metadata table 214 contains entries such as a video identifier 216, a corresponding decompression algorithm 218, and a corresponding interpolation setting 220. In at least one embodiment, the video identifier 216 serves as a unique reference to a specific compressed video, thereby allowing access to its corresponding data. In at least one embodiment, the video identifier 216 may be automatically generated by the GPU, or it may be an address of GPU memory storing the compressed video data. In at least one embodiment, the decompression algorithm 218 specifies the method required to decompress video frames (e.g., H.264 or VP9), while the interpolation setting 220 defines the pixel sampling method to be used (e.g., bilinear or nearest neighbor).

[0068] In at least one embodiment, the metadata table 214 is created and populated by an API (e.g., a video surface or texture creation API) during initialization or resource allocation. In at least one embodiment, the metadata table 214 is managed by a GPU driver that ensures the table remains synchronized with the compressed video data stored in GPU memory 238. In at least one embodiment, the metadata table 214 resides in GPU-accessible memory, enabling fast lookups during video operations via APIs such as the texture read API 230, surface read API 232, or surface write API 234.

[0069] In at least one embodiment, video information 222 and 224 represent resource descriptors that provide metadata for identifying and accessing video content. In at least one embodiment, video information 222 is associated with video texture object 226, and video information 224 is associated with video surface object 228. In at least one embodiment, video texture object 226 is created by video texture object creation API 225, which binds GPU resources for processing video to video texture object 226. In at least one embodiment, texture reading API 230 uses video texture object 226 to efficiently acquire pixel data. In at least one embodiment, video surface object 228 is created by video surface object creation API 227, which binds GPU resources for processing video to video surface object 228.

[0070] In at least one embodiment, the video surface object 228 is used by the surface read API 232 to retrieve pixel data (e.g., pixel properties) and by the surface write API 234 to write the pixel data to a specific frame of the video. In at least one embodiment, the texture read API 230, surface read API 232, and surface write API 234 provide operations for interacting indirectly with the compressed video 240 stored in GPU memory 238 via cache 236. In at least one embodiment, GPU memory 238 is allocated by GPU memory allocation API 237, which specifies the number of frames in the compressed video to be stored and the byte size of the compressed video.

[0071] In at least one embodiment, a range of compressed frames in compressed video 240 is decompressed into cache 236 during API operations, thereby enabling access to individual frame or pixel data. In at least one embodiment, the cache 236 temporarily stores the decompressed frame data. In at least one embodiment, the range of frames to be decompressed is determined by range parameters associated with video texture object 226 and video surface object 228. In at least one embodiment, each of texture read API 230, surface read API 232, and surface write API 234 takes video texture object 226 or video surface object 228 as input parameters, thus enabling access to the range of frames associated with video texture object 226 or video surface object 228.

[0072] In at least one embodiment, when one of the texture read API 230, surface read API 232, and surface write API 234 accesses the compressed video 240, the GPU decompresses a range of frames in the compressed video 240, which is compressed using a keyframe- and inter-frame-based format. In at least one embodiment, the GPU driver dynamically manages cache 236, allocating memory for decompressed frames based on a specified range. In at least one embodiment, if a new range of frames is requested, previously decompressed frames can be removed from cache 236 to make room for the new decompressed data. In at least one embodiment, this dynamic management allows the APIs to handle operations on large or high-resolution video files without exceeding available memory limits, while keeping the compressed video 240 in GPU memory.

[0073] In at least one embodiment, the texture read API 230 is used to obtain interpolated pixel values ​​from the decompressed video frame based on input texture coordinates. In at least one embodiment, the surface read API 232 is used to retrieve pixel properties from a specified location in the decompressed frame using a video surface object 228, while the surface write API 234 is used to modify pixel data at a specified location in the frame.

[0074] In at least one embodiment, GPU core 241 utilizes these APIs to perform operations such as video editing, rendering, or processing. In at least one embodiment, GPU core 241 can invoke APIs 230, 232, and 234 with input parameters specifying pixel positions, frame indices, and / or pixel data for read or write operations. In at least one embodiment, system 200B utilizes the metadata table 214 to ensure that the APIs have access to appropriate decompression algorithms, interpolation methods, and resource descriptors, thereby enabling efficient video manipulation.

[0075] In at least one embodiment, combined Figure 2B At least one of the described features is utilizing Figure 1 and Figure 3-Figure 35C This is achieved by at least one feature of any system and / or process shown and / or described. In at least one embodiment, it is combined with Figure 2B At least one of the described features is used to achieve Figure 1 and Figures 3-34 At least a portion of any system and / or process shown and / or described.

[0076] Figure 3 This is a block diagram illustrating a GPU memory allocation operation 300 (“Operation 300”) according to at least one embodiment. In at least one embodiment, Operation 300 includes one or more computational operations that cause a processor (e.g., GPU or CPU) to execute an API to allocate memory in GPU memory to store compressed video and associated metadata.

[0077] In at least one embodiment, operation 300 includes a GPU memory allocation call 302 (“Call 302”) and a GPU memory allocation 304 (“Response 304”). In at least one embodiment, Call 302 is a function call initiated by one or more software programs (e.g., one or more kernels executed by the GPU). In at least one embodiment, the function call invokes instructions / APIs to allocate GPU memory to store compressed video data and metadata. In at least one embodiment, Call 302 specifies multiple parameters 306-314. In at least one embodiment, Call 302 specifies a pointer to array 306 as an input parameter. In at least one embodiment, this pointer represents a reference to an array storing compressed video and metadata. In at least one embodiment, Call 302 specifies a channel format descriptor 308 as an additional input parameter. In at least one embodiment, this channel format descriptor indicates the format of the video data, such as RGBA with 8-bit channels. In at least one embodiment, Call 302 specifies a video compression algorithm 310 as an additional input parameter. In at least one embodiment, the video compression algorithm 310 instructs a codec, such as H.264, H.265, or AV1, for compressing the video, thereby enabling the GPU to correctly interpret and decompress the compressed video. In at least one embodiment, call 302 specifies a video size descriptor 312 as an additional input parameter. In at least one embodiment, the video size descriptor 312 defines the width and height of the video frames, as well as the temporal range of the compressed video data (e.g., start and end frames). In at least one embodiment, call 302 specifies a compressed video size 314 as an additional input parameter. In at least one embodiment, the compressed video size represents the total memory size (in bytes) of the compressed video, thereby ensuring efficient memory allocation.

[0078] In at least one embodiment, response 304 includes a status 316 as output. In at least one embodiment, this status is a code or value indicating whether the memory allocation was successfully executed. In at least one embodiment, a value of 0 indicates that the memory allocation operation was successfully executed. In at least one embodiment, a non-zero value indicates an error, such as insufficient GPU memory or invalid input parameters.

[0079] In at least one embodiment, combined Figure 3 At least one of the described features uses Figure 1-2B and Figure 4-34 Implemented by at least one feature of any system and / or process described herein. In at least one embodiment, combined with Figure 3 At least one of the described features is used to achieve Figure 1-2B and Figures 4-34At least a portion of any system and / or process shown and / or described.

[0080] Figure 4 A process 400 for allocating memory on a GPU to store compressed video is illustrated. In at least one embodiment, process 400 is executed by a processor (e.g., a GPU).

[0081] In at least one embodiment, in step 402, the GPU receives a call to execute an API to allocate memory on the GPU to store the compressed video. In at least one embodiment, the call is received from a host program, for example, in conjunction with... Figure 1 The compressed video loader 106 described herein. In at least one embodiment, the call includes input parameters that specify at least a portion of the metadata of the compressed video. In at least one embodiment, the metadata includes a video ID, which is a unique identifier of the compressed video. In at least one embodiment, the video ID may be generated internally by the API or the GPU memory. In at least one embodiment, the video ID may be a reference, handle, or pointer to the compressed video, wherein the reference, handle, or pointer acts as a proxy for the video identifier. In at least one embodiment, the metadata also includes a compression algorithm, which is an indicator of a codec used for compression, which subsequent API calls may use to decompress the compressed video. In at least one embodiment, the metadata includes the width and height of each video frame, and the total number of frames in the compressed video.

[0082] In at least one embodiment, in step 404, the GPU allocates memory based on at least a portion of the metadata. In at least one embodiment, the allocation is performed in global memory on the GPU, which is used to store the compressed video. In at least one embodiment, the metadata portion includes attributes of the compressed video, such as video size, data format, and compressed size. In at least one embodiment, the size of the memory to be allocated is calculated using these attributes to ensure sufficient space is allocated to accommodate all frames of the compressed video. In at least one embodiment, the GPU's internal memory management mechanism initializes the allocated memory, clearing or setting default values ​​to ensure readiness for data storage. In at least one embodiment, the memory is mapped for efficient data transfer between the host (CPU) and the device (GPU), while taking alignment constraints into account and optimizing host-to-device communication. In at least one embodiment, the GPU partitions the allocated memory into chunks corresponding to individual video frames, thereby enabling efficient access during decompression or processing. In at least one embodiment, the memory layout is further optimized to process compressed data, thereby ensuring compatibility with decompression algorithms and compliance with GPU cache requirements.

[0083] In at least one embodiment, in step 406, the GPU stores the metadata of the compressed video in a metadata store. In at least one embodiment, the metadata store is a data structure on the GPU used to store and manage the metadata of the compressed video. In at least one embodiment, the metadata store links the metadata to allocated GPU memory, enabling subsequent APIs to access this information to perform operations such as decompressing the video stored in the allocated GPU memory and reassigning attributes. In at least one embodiment, the store may be implemented as a table in GPU memory or a dedicated register to ensure efficient metadata access, thereby minimizing latency and processing overhead. In at least one embodiment, the metadata table stores information (e.g., pointers) for identifying the allocated GPU memory and a decompression algorithm for decompressing the compressed video to be stored in the GPU memory. In at least one embodiment, the metadata is stored in the metadata store when the API is executed. In at least one embodiment, the metadata store is located in the allocated GPU memory.

[0084] In at least one embodiment, in step 408, the GPU prepares the allocated GPU memory for subsequent operations. In at least one embodiment, the preparation includes initializing the memory and optimizing it for video decompression and processing tasks. In at least one embodiment, the preparation includes clearing memory regions or setting default values ​​to avoid residual data artifacts. In at least one embodiment, the GPU optimizes the memory layout for efficient access during video decompression and processing, aligns the memory with hardware-specific requirements, and organizes it to facilitate parallel operation.

[0085] In at least one embodiment, in step 410, the CPU generates a response to the API call. The response includes a success or failure code and a pointer to the allocated memory. In at least one embodiment, the success code confirms that the memory was correctly allocated, while the failure code provides detailed information about the problem, such as invalid input parameters or insufficient GPU memory. In at least one embodiment, the response also includes a pointer to the allocated memory, which serves as a reference for subsequent operations. In at least one embodiment, this pointer allows a software program (e.g., a video editing program) to directly access the allocated GPU memory to perform tasks such as transferring compressed video data, decompressing frames, or rendering.

[0086] In at least one embodiment, the compressed video can be stored in the allocated GPU memory via a process utilizing a processor that includes one or more circuits for executing another API. In at least one embodiment, the compressed video data is transferred from the host system (e.g., CPU memory) to the allocated GPU memory using a direct memory transfer operation (e.g., cudaMemcpyToArray()). In at least one embodiment, the transfer operation ensures that the data is placed in the allocated memory with an optimized layout. In at least one embodiment, in addition to storing the compressed video data, the transfer API also stores metadata about the video in the allocated memory or a linked metadata repository.

[0087] In at least one embodiment, the processor stores an indication of the decompression algorithm in the metadata. In at least one embodiment, the indication (e.g., codec type (e.g., H.264 or VP9)) can be used to decompress the compressed video. In at least one embodiment, by associating the decompression algorithm with the stored video data, the processor eliminates the need for the programmer to re-specify this information in subsequent API calls.

[0088] In at least one embodiment, combined Figure 4 At least one of the described features is the use of Figure 1-3and Figure 5-34 This is achieved by at least one feature of any system and / or process shown and / or described. In at least one embodiment, it is combined with Figure 4 At least one of the described features is used to achieve Figure 1-3 and Figure 5-34 At least a portion of any system and / or process shown and / or described.

[0089] Figure 5 This is a block diagram illustrating a video texture object creation operation 500 (“Operation 500”) according to at least one embodiment. In at least one embodiment, Operation 500 includes one or more computational operations that cause a processor to create a texture object for accessing video data in GPU memory and configuring interpolation settings for pixel sampling.

[0090] In at least one embodiment, operation 500 includes a video texture object creation call 502 (“Call 502”) and a video texture object creation response 504 (“Response 504”). In at least one embodiment, Call 502 is a function call initiated by one or more software programs (e.g., kernel functions executed by the GPU). In at least one embodiment, the function call invokes instructions / APIs to configure and create texture objects for video data. In at least one embodiment, when executed, the instructions cause the processor to bind video memory, define filtering and addressing properties, and specify interpolation algorithms.

[0091] In at least one embodiment, the API binds video memory to a textured video object to be created by the API by linking an allocated memory region containing compressed video with a configuration of the textured video object. In at least one embodiment, the binding is performed via a resource descriptor specifying details such as memory type (e.g., a 2D array), memory address, and video data layout. In at least one embodiment, the GPU or another processor passes the descriptor to a GPU driver, which creates an internal mapping between the memory and the video texture object. In at least one embodiment, the binding process includes configuring the GPU driver to associate the specified memory region with the textured video object. In at least one embodiment, the GPU driver validates the resource descriptor and updates an internal metadata table to maintain the relationship between the memory and the video texture object, thereby ensuring that any subsequent operations involving the textured video object reference the correct memory resource without manual intervention. In at least one embodiment, the binding enables the GPU kernel to use the textured video object to access and process video data using GPU-based features such as sampling and interpolation.

[0092] In at least one embodiment, call 502 retrieves a plurality of input parameters 506-512. In at least one embodiment, the parameters include a pointer to a texture video object 506, which provides a reference to the texture video object to be created, thereby allowing subsequent API calls to interact with or use the object. In at least one embodiment, the parameters include a video memory resource descriptor 508, which defines the location and type of video memory associated with the texture video object (e.g., a 2D CUDA array) and ensures that the created texture video object is correctly mapped to GPU memory allocated for compressed video. In at least one embodiment, the parameters include a texture video object descriptor 510, which contains properties such as filtering mode, addressing mode, and coordinate normalization. In at least one embodiment, the filtering mode property specifies the interpolation algorithm (e.g., bilinear, nearest neighbor) used for pixel sampling. In at least one embodiment, the addressing mode property configures the behavior of out-of-bounds coordinates (e.g., clamping, wrapping). In at least one embodiment, the coordinate normalization property indicates whether the texture coordinates are normalized or unnormalized. In at least one embodiment, the parameters include a frame range 512 that defines a subset of video frames to be associated with the textured video object, including a start frame and an end frame, as well as spatial dimensions such as width and height.

[0093] In at least one embodiment, the range parameter enables precise and efficient access to video segments. In at least one embodiment, the compressed video uses temporal compression techniques that store information across frames in a manner that reduces redundancy, where not every frame is self-contained. For example, in at least one embodiment, keyframes (also known as I-frames) are fully encoded and contain all the information needed to reconstruct the image, while most other frames (e.g., predicted frames (P-frames) and bidirectional frames (B-frames)) depend on data from adjacent frames for complete decoding. In at least one embodiment, accessing a specific range of frames ensures that all necessary data (including dependent frames and their associated keyframes) is available for correct decoding. In at least one embodiment, the range parameter acts as a sliding window on the compressed video, allowing a program (e.g., a GPU kernel) to incrementally access manageable portions of the compressed video, moving the window as needed to load subsequent ranges. For example, in at least one embodiment, a video editing application can use the sliding window to load only the frames needed for a specific scene, thereby freeing up memory and allowing the program to run more efficiently. In at least one embodiment, a video editing application can access a range of frames sequentially, one at a time. However, in at least one embodiment, the video editing application can skip frames, allowing the application to jump directly to a specific scene or frame. Furthermore, in at least one embodiment, ranges can be accessed discontinuously or out of order.

[0094] In at least one embodiment, the API enables video identifiers and pixel interpolation algorithm indications to be stored and linked together. In at least one embodiment, pixel interpolation is crucial for tasks such as scaling, resizing, or transforming video frames to meet display or processing requirements. In at least one embodiment, the API automatically performs the process of associating specific interpolation algorithms with corresponding videos, thereby ensuring that the unique processing requirements of each video are met without manual intervention.

[0095] In at least one embodiment, although various interpolation methods exist, such as bilinear interpolation, nearest neighbor selection, and more advanced techniques, some APIs do not provide any information in their parameters to indicate the desired interpolation algorithm. This prevents these APIs from determining or enforcing a suitable interpolation method, resulting in inefficient and inconsistent video processing workflows. In at least one embodiment, Figure 5 The described API allows for the storage of both video identifiers and indications of the interpolation algorithm, and their association with each other, enabling access to the video using the indicated pixel interpolation method. In at least one embodiment, when another API is called to interpolate pixels of a frame, that API uses the video identifier lookup table and retrieves the appropriate interpolation method.

[0096] In at least one embodiment, response 504 includes state 516, which is a code or value indicating whether the instruction has been successfully executed. In at least one embodiment, a success state confirms that the texture object has been created and associated with GPU memory, while a failure state indicates an error, such as invalid parameters or insufficient GPU resources.

[0097] In at least one embodiment, combined Figure 5 At least one of the described features is the use of Figure 1-4 and Figure 6-34 It is implemented by at least one feature of any system and / or process described herein. In at least one embodiment, it is combined with Figure 5 At least one of the described features is used to achieve Figure 1-4 and Figure 6-34 At least a portion of any system and / or process shown and / or described.

[0098] Figure 6 This is a flowchart illustrating a process 600 for creating a video texture object according to at least one embodiment. In at least one embodiment, process 600 is executed at least partially on a computer system such as those described in other parts of this disclosure. In at least one embodiment, process 600 is executed by a GPU.

[0099] In at least one embodiment, in step 602, the processor receives a call to execute an API to create a video texture object. In at least one embodiment, the call can be initiated from within the GPU kernel, for example, by combining... Figure 1 The described video reader 114. In at least one embodiment, the call includes a video identifier that uniquely identifies the compressed video stored in GPU memory and an indication of the interpolation algorithm to be applied when processing the video frame as input parameters. In at least one embodiment, the interpolation algorithm may include techniques such as bilinear interpolation, nearest neighbor interpolation, or other pixel processing algorithms.

[0100] In at least one embodiment, in step 604, the processor stores the video identifier and the indication of the interpolation algorithm in a metadata table managed by the GPU driver. In at least one embodiment, the metadata table links the video identifier to the interpolation algorithm, enabling other APIs to retrieve this information to process video frames. In at least one embodiment, the textured video object to be created acts as a logical handle or reference that links the compressed video stored in GPU memory to the interpolation algorithm stored in the metadata table. In at least one embodiment, the textured video object allows other APIs to efficiently access and process the video using the correct interpolation algorithm. In at least one embodiment, when another API is involved, a reference, point, or handle to the video texture object is passed as an input parameter. In at least one embodiment, the GPU driver can retrieve the video identifier of the compressed video and the associated interpolation algorithm stored during the creation of the textured video object.

[0101] In at least one embodiment, in step 606, the processor creates the video texture object. In at least one embodiment, the video texture object is a GPU-managed structure capable of efficiently accessing video data stored in GPU memory. In at least one embodiment, the textured video object encapsulates video attributes such as size, data format, and memory layout, while utilizing GPU texture units for optimized sampling and interpolation. In at least one embodiment, the video texture object specifies the interpolation method for pixels, supports algorithms such as bilinear interpolation or nearest neighbor interpolation, and defines addressing modes for handling out-of-bounds coordinates, such as clamping or wrapping. In at least one embodiment, the textured video object simplifies access to video frames by linking compressed video memory to the CUDA kernel, thereby enabling operations such as frame sampling, scaling, and filtering. In at least one embodiment, the textured video object minimizes overhead by using parallel GPU hardware.

[0102] In at least one embodiment, in step 608, the processor generates a response indicating the result of the operation. In at least one embodiment, if the API successfully creates the video texture object, the response includes a success indication and provides a reference to the video texture object for use in subsequent API calls. In at least one embodiment, if the API fails, the response includes error information indicating the reason for the failure, such as invalid parameters or metadata storage problems.

[0103] In at least one embodiment, combined Figure 6 At least one of the described features is the use of Figure 1-5 and Figure 7-34It is implemented by at least one feature of any system and / or process described and / or described herein. In at least one embodiment, it is combined with Figure 6 At least one of the described features is used to achieve Figure 1-5 and Figure 7-34 At least a portion of any system and / or process described herein.

[0104] Figure 7 This is a block diagram illustrating a video surface object creation operation 700 (“Operation 700”) according to at least one embodiment. In at least one embodiment, Operation 700 includes one or more computational operations that cause a processor to create a video surface object that links compressed video data stored in GPU memory with specific metadata (e.g., frame range, memory descriptor, and decompression algorithm).

[0105] In at least one embodiment, operation 700 includes a video surface object creation call 702 (“Call 702”) and a video surface object creation response 704 (“Response 704”). In at least one embodiment, Call 702 is a function call initiated by one or more software programs (e.g., applications or kernel functions executing on a GPU). In at least one embodiment, the function call invokes instructions / APIs to create a video surface object. In at least one embodiment, the instructions, when executed, cause the processor to associate an identifier of the compressed video with a decompression algorithm indicated in a resource descriptor.

[0106] In at least one embodiment, call 702 includes a plurality of parameters 706-710. In at least one embodiment, the parameters include a pointer or handle 706 referencing a video surface object to be created. In at least one embodiment, this pointer will be returned for future access to and management of the video surface object. In at least one embodiment, the parameters include a resource descriptor 708 of video memory, which specifies detailed information about the GPU memory storage used to store the compressed video, such as its type, layout, or address. In at least one embodiment, the parameters include a frame range 710, which defines the start and end frames to be associated with the video surface object. In at least one embodiment, this parameter allows selective operation on specific segments of the compressed video stored in the GPU memory described by resource descriptor 708.

[0107] In at least one embodiment, response 704 includes a state 716 as output of the call. In at least one embodiment, state 716 is a code, value, or other information indicating whether the video surface object creation operation was successfully executed. For example, in at least one embodiment, a value of 0 or true may indicate successful creation, while a value of 1 or false may indicate an error or failure in creating the video surface object.

[0108] In at least one embodiment, combined Figure 7 At least one of the described features is utilizing Figure 1-6 and Figure 8-34 This is achieved by at least one feature of any system and / or process shown and / or described. In at least one embodiment, in combination with Figure 7 At least one of the described features is used to achieve Figure 1-6 and Figure 8-34 At least a portion of any system and / or process shown and / or described.

[0109] Figure 8 This is a flowchart illustrating a process 800 for creating a video surface object according to at least one embodiment. In at least one embodiment, process 800 is executed at least partially on a computer system such as those described elsewhere in this disclosure. In at least one embodiment, process 800 is executed by a GPU.

[0110] In at least one embodiment, in step 802, the processor receives a call to execute an API to create a video surface object. In at least one embodiment, the call can be initiated from within the GPU kernel, for example, by combining... Figure 1 The described video reader 114 and / or video writer 116. In at least one embodiment, the call includes input parameters, such as a video identifier that uniquely identifies the compressed video in GPU memory, and an indication of a decompression algorithm to be applied to the frames of the video. In at least one embodiment, the video identifier may be a reference, handle, or pointer to the compressed video. In at least one embodiment, the video identifier may be automatically generated by the GPU as a piece of information.

[0111] In at least one embodiment, in step 804, the processor stores the video identifier and the indication of the decompression algorithm in a metadata table managed by the GPU driver. In at least one embodiment, the metadata table links the video identifier to the decompression algorithm, enabling subsequent APIs to retrieve this information to decompress and process video frames.

[0112] In at least one embodiment, in step 806, the processor creates the video surface object. In at least one embodiment, the video surface object acts as a logical handle or reference that links the compressed video stored in GPU memory to metadata stored in a metadata table managed by the GPU driver. In at least one embodiment, the object enables one or more other APIs to efficiently access and process the video using a specified decompression algorithm. In at least one embodiment, when the API is called to create the video surface object, the call takes a resource descriptor as input, which acts as an indirect reference to the compressed video stored in GPU memory. In at least one embodiment, when another API accesses the compressed video using the video surface object, the other API treats the video surface object as an identifier of the compressed video (e.g., an address or pointer to the GPU memory storing the compressed video). In at least one embodiment, the other API uses the video surface object to query the metadata table to determine the decompression algorithm associated with the compressed video, and then uses that decompression algorithm to decompress the video.

[0113] In at least one embodiment, in step 808, the processor generates a response indicating the result of the API call. In at least one embodiment, the response includes a status indicator. For example, a status of 0 or True may indicate successful creation of the video surface object, while a status of 1 or False may indicate an error or failure during object creation.

[0114] In at least one embodiment, combined Figure 8 At least one of the described features is the use of Figure 1-7 and Figure 9-34 It is implemented by at least one feature of any system and / or process described herein. In at least one embodiment, it is combined with Figure 8 At least one of the described features is used to achieve Figure 1-7 and Figure 9-34 At least a portion of any system and / or process described herein.

[0115] Figure 9 This is a block diagram illustrating a texture reading operation 900 (“Operation 900”) according to at least one embodiment. In at least one embodiment, Operation 900 includes one or more computational operations that cause a processor (e.g., a GPU) to acquire pixel data from a specified video frame using specified texture coordinates and perform interpolation on two or more pixels in the pixel data.

[0116] In at least one embodiment, operation 900 includes a texture read API call 902 (“Call 902”) and a texture read API response 904 (“Response 904”). In at least one embodiment, Call 902 is a function call initiated by one or more software programs (e.g., kernel functions executed by a GPU or other parallel processing unit). In at least one embodiment, the function call invokes instructions to acquire pixel data from a selected video frame. In at least one embodiment, when executed, the instructions cause a processor to access pixel data at specified coordinates within a particular video frame and perform interpolation on two or more pixels of the selected video frame.

[0117] In at least one embodiment, call 902 retrieves multiple parameters 906-910. In at least one embodiment, the parameters include a pointer to a video texture object 906, which is a reference to the video texture object. In at least one embodiment, the video texture object contains metadata and resource mappings necessary for locating the video frame data in GPU memory. In at least one embodiment, the video texture object serves as a logical abstraction that encapsulates the metadata and resource mappings necessary for efficiently accessing and processing the video frame data stored in GPU memory. In at least one embodiment, this metadata includes details such as a unique video identifier, a range of frames associated with the video texture object, a decompression algorithm for the compressed video format, and interpolation settings such as bilinear or nearest neighbor methods for pixel sampling. In at least one embodiment, the metadata also describes the video format, such as RGBA or YUV, to ensure the correct interpretation of pixel values. In at least one embodiment, the video texture object contains resource mappings that define how the video data is organized and accessed. In at least one embodiment, the resource mappings include a memory layout, the exact memory address storing the compressed video, and links to resource descriptors that specify memory types and accessibility settings.

[0118] In at least one embodiment, the parameters of call 902 include a horizontal texture coordinate 908, which specifies the x-coordinate of the selected frame from which pixel data is to be obtained. In at least one embodiment, this value may be normalized (e.g., in the range [0,1]) or unnormalized (e.g., specified in texels). In at least one embodiment, the parameters of call 902 include a vertical texture coordinate 910, which specifies the y-coordinate of the selected frame from which pixel data is to be obtained. In at least one embodiment, similar to the horizontal coordinate, the vertical coordinate may also be normalized or unnormalized. In at least one embodiment, the parameters of call 902 include a frame index (optional): although not explicitly shown in the figures, some implementations of the API may include a frame index 912 for indicating a specific frame in a video batch from which pixel data is to be obtained.

[0119] In at least one embodiment, the response 904 includes a pixel value 916 read from a location specified by horizontal texture coordinates 908 and vertical texture coordinates 910 on the frame indicated by frame index 912. In at least one embodiment, the returned pixel value 916 includes information about the color and intensity of one or more pixels, enabling a video editing or processing application to analyze or manipulate that portion of the video frame.

[0120] In at least one embodiment, combined Figure 9 At least one of the described features is the use of Figure 1-8 and Figure 10-34 It is implemented by at least one feature of any system and / or process described herein. In at least one embodiment, it is combined with Figure 9 At least one of the described features is used to achieve Figure 1-8 and Figure 10-34 At least a portion of any system and / or process described herein.

[0121] Figure 10 This is a flowchart illustrating a process 1000 for acquiring pixel data from compressed frames in a compressed video, according to at least one embodiment. In at least one embodiment, process 1000 is executed at least partially on a computer system such as those described elsewhere in this disclosure. In at least one embodiment, process 1000 is executed by a processor such as a GPU.

[0122] In at least one embodiment, in step 1002, the processor receives a call to execute an API to obtain pixel data from compressed frames in a compressed video. In at least one embodiment, the call includes input parameters such as a pointer to a video texture object, horizontal and vertical texture coordinates, and a frame index specifying a frame of the compressed video.

[0123] In at least one embodiment, in step 1004, the processor identifies the compressed video and a range of frames associated with the video texture object. In at least one embodiment, the video texture object includes metadata and a resource map that enables the processor to locate video data stored in GPU memory and determine a range of frames accessible via the API.

[0124] In at least one embodiment, in step 1006, the processor decompresses the identified range of frames into a cache. In at least one embodiment, a decompression algorithm specified in the video texture object is applied to the compressed video data to convert each frame within each of the frame range into a usable decompressed format.

[0125] In at least one embodiment, in step 1008, the processor determines the position within the decompressed frame based on the horizontal and vertical texture coordinates provided in the call. In at least one embodiment, these coordinates specify a precise pixel or region of interest within the decompressed frame.

[0126] In at least one embodiment, in step 1010, if the texture coordinates are not precisely aligned with the texel center, for example, when the specified texture coordinates fall between discrete texels (texture elements) in the decompressed frame, the processor performs interpolation on two or more pixels in the decompressed frame. In at least one embodiment, a texel is a pixel in a different context. In at least one embodiment, a texel is an individual element of a texture stored in memory, while a pixel is the smallest individual color point in a digital image displayed on the screen. In at least one embodiment, the interpolation algorithm (e.g., bilinear interpolation or nearest neighbor interpolation) specified during the creation of the video texture object is applied to compute the interpolated pixel value. In at least one embodiment, the interpolation algorithm to be used is determined based on stored metadata matching the video texture object. In at least one embodiment, if the metadata does not specify an interpolation algorithm, a default pixel value sampling method can be used. In at least one embodiment, the interpolation process is performed directly on the decompressed frame data stored in the GPU cache or shared memory. In at least one embodiment, the processor uses decompressed pixel data as input for the interpolation computation.

[0127] In at least one embodiment, in step 1012, the processor obtains a pixel value from the calculated position in the decompressed frame. In at least one embodiment, the obtained pixel value is then returned as the output of the API for further processing by the GPU kernel.

[0128] In at least one embodiment, combined Figure 10 At least one characteristic described is utilizing Figure 1-9 and Figure 11-34 This is achieved by at least one feature of any system and / or process shown and / or described. In at least one embodiment, it is combined with Figure 10 At least one feature described is used to achieve Figure 1-9 and Figure 11-34 At least a portion of any system and / or process shown and / or described.

[0129] Figure 11 This is a block diagram illustrating a surface read operation 1100 (“Operation 1100”) according to at least one embodiment. In at least one embodiment, Operation 1100 includes one or more computational operations that cause a processor (e.g., a GPU) to read pixel data from a specified video frame stored in GPU memory and return the result to a memory specified by the caller.

[0130] In at least one embodiment, operation 1100 includes a surface readout API call 1102 (“Call 1102”) and a surface readout API response 1118 (“Response 1118”). In at least one embodiment, call 1102 is a function call initiated by a software program (e.g., a video editing or rendering application) to access pixel data at specified coordinates in a selected frame of a video. In at least one embodiment, the function call invokes the surface readout API to retrieve properties of one or more pixels at specified coordinates on the selected frame from a range of frames of a compressed video stored in GPU memory. In at least one embodiment, the retrieved properties describe visual characteristics of one or more pixels, such as their color value, intensity, or other relevant attributes. In at least one embodiment, this information is used to perform video editing tasks, such as adjusting the color or brightness of a particular video frame. In at least one embodiment, retrieving pixel properties is also referred to as indicating pixel properties.

[0131] In at least one embodiment, a call to 1102 retrieves a plurality of parameters 1103-1112. In at least one embodiment, the parameters include a pointer 1103 to memory storing the retrieved data, which is a reference to the memory location where the retrieved property data will be stored. In at least one embodiment, the parameters include a pointer 1106 to a video surface object, which is a reference to an object linking video data stored in GPU memory to metadata and decompression algorithms required for processing. In at least one embodiment, the parameters include a horizontal texture coordinate 1108, which specifies the x-coordinate in a selected frame from which property data of pixels to be retrieved. In at least one embodiment, this value may be normalized (e.g., in the range [0,1]) or unnormalized (e.g., in texels). In at least one embodiment, the parameters include a vertical texture coordinate 1110, which specifies the y-coordinate in the selected frame. In at least one embodiment, similar to the horizontal coordinate, the vertical coordinate may also be normalized or unnormalized. In at least one embodiment, the parameters include a frame index 1112, which indicates the specific frame in the video to be accessed.

[0132] In at least one embodiment, response 1118 includes state 1116 as output. In at least one embodiment, state 1116 is a code, value, or other information indicating whether the surface read operation was successfully performed. For example, in at least one embodiment, a state value of 0 may indicate success, while a non-zero state value may indicate an error, such as an invalid parameter or inaccessible memory.

[0133] In at least one embodiment, combined Figure 11 At least one of the described features is utilizing Figure 1-10 and Figure 12-34 It is implemented by at least one feature of any system and / or process described herein. In at least one embodiment, it is combined with Figure 11 At least one of the described features is used to achieve Figure 1-10 and Figure 12-34 At least a portion of any system and / or process described herein.

[0134] Figure 12 This is a flowchart illustrating a process 1200 for indicating the properties of one or more pixels in a compressed frame of a compressed video, according to at least one embodiment. In at least one embodiment, process 1200 is executed at least partially on a computer system such as those described elsewhere in this disclosure. In at least one embodiment, process 1200 is executed by a processor such as a GPU.

[0135] In at least one embodiment, in step 1202, the processor receives a call to execute an API to indicate the properties of one or more pixels in a compressed frame of the compressed video. In at least one embodiment, the call includes input parameters such as a pointer to a video surface object, horizontal texture coordinates and vertical texture coordinates, and a frame index specifying the frame of the compressed video.

[0136] In at least one embodiment, in step 1204, the processor identifies the compressed video and a range of frames associated with the video surface object. In at least one embodiment, the video surface object includes metadata and resource maps that allow the processor to locate the video data stored in GPU memory and determine the range of frames accessible via the API.

[0137] In at least one embodiment, in step 1206, the processor decompresses the identified range of frames into a cache. In at least one embodiment, a decompression algorithm specified in the video surface object is applied to the compressed video data to convert the range of frames into a usable decompressed format.

[0138] In at least one embodiment, in step 1208, the processor determines the position within the decompressed frame based on the horizontal and vertical texture coordinates provided in the call. In at least one embodiment, these coordinates specify a precise pixel or region of interest within the decompressed frame.

[0139] In at least one embodiment, in step 1210, the processor retrieves the properties of one or more pixels from the decompressed frame. In at least one embodiment, these properties include attributes such as color, intensity, or transparency, which describe the visual characteristics of the one or more pixels in the decompressed frame.

[0140] In at least one embodiment, in step 1212, the processor stores the properties of the one or more pixels in a designated memory location. In at least one embodiment, this memory location is provided as an input parameter in the API call, thereby enabling efficient transfer of the pixel properties for further processing or analysis.

[0141] In at least one embodiment, combined Figure 12 At least one of the described features is utilizing Figure 1-11 and Figure 13-34 This is achieved by at least one feature of any system and / or process shown and / or described. In at least one embodiment, it is combined with Figure 12 At least one of the described features is used to achieve Figure 1-11 and Figure 13-34 At least a portion of any system and / or process shown and / or described.

[0142] Figure 13 A block diagram illustrating a surface write operation 1300 (“Operation 1300”) according to at least one embodiment is shown. In at least one embodiment, Operation 1300 includes one or more computational operations that cause a processor (e.g., a GPU) to write pixel data to a specified location in a selected video frame stored in GPU memory.

[0143] In at least one embodiment, operation 1300 includes a surface write API call 1302 (“Call 1302”) and a surface write API response 1318 (“Response 1318”). In at least one embodiment, call 1302 is a function call initiated by one or more software programs (e.g., rendering or editing applications) for writing pixel data to a specified location in a selected video frame. In at least one embodiment, the function call invokes an instruction to locate the video surface object, access the corresponding video frame, and write the pixel data to the specified coordinates on the selected frame.

[0144] In at least one embodiment, a call to 1302 retrieves a plurality of parameters 1303-1312. In at least one embodiment, the parameters include pixel data 1303 to be written, which specifies the pixel value (e.g., color, intensity) to be written to the video frame. In at least one embodiment, the parameters include a pointer to a video surface object 1306, which is a reference to an object that provides access to the compressed video data stored in GPU memory and links it to associated metadata and decompression algorithms. In at least one embodiment, the parameters include a horizontal texture coordinate 1308, which specifies the x-coordinate in the selected video frame where the pixel data will be written. In at least one embodiment, this value may be normalized (e.g., in the range [0,1]) or unnormalized (e.g., specified in texels). In at least one embodiment, the parameters include a vertical texture coordinate 1310, which specifies the y-coordinate in the selected video frame where the pixel data will be written. In at least one embodiment, similar to the horizontal coordinate, the vertical coordinate may also be normalized or unnormalized. In at least one embodiment, the parameter includes a frame index 1312, which indicates the specific frame in the video in which the pixel data will be written.

[0145] In at least one embodiment, response 1318 includes state 1316 as output. In at least one embodiment, state 1316 is a code, value, or other information indicating whether the surface write operation was successfully performed. For example, a state value of 0 may indicate success, while a non-zero state value may indicate an error, such as invalid texture coordinates, an inaccessible video surface object, or a memory conflict.

[0146] In at least one embodiment, combined Figure 13 At least one of the described features is utilizing Figure 1-12 and Figure 14-34 This is achieved by at least one feature of any system and / or process shown and / or described. In at least one embodiment, it is combined with Figure 13 At least one of the described features is used to achieve Figure 1-12 and Figure 14-34 At least a portion of any system and / or process shown and / or described.

[0147] Figure 14 This is a flowchart illustrating a process 1400 for writing pixel data into a specific frame of a compressed video, according to at least one embodiment. In at least one embodiment, process 1400 is executed at least partially on a computer system such as those described elsewhere in this disclosure. In at least one embodiment, process 1400 is executed by a processor (e.g., a GPU).

[0148] In at least one embodiment, in step 1402, the processor receives a call to execute an API to write pixel data to a specific frame in the compressed video. In at least one embodiment, the call includes input parameters such as pixel data to be written, a pointer to a video texture object, horizontal texture coordinates and vertical texture coordinates, and a frame index specifying the frame in the compressed video.

[0149] In at least one embodiment, in step 1404, the processor identifies the compressed video and a range of frames associated with the video texture object. In at least one embodiment, the video texture object includes metadata and a resource map that enables the processor to locate the video data stored in GPU memory and determine the range of frames accessible via the API.

[0150] In at least one embodiment, in step 1406, the processor decompresses the identified range of frames into a cache. In at least one embodiment, a decompression algorithm specified in the video texture object is applied to the compressed video data to convert the range of frames into a usable decompressed format.

[0151] In at least one embodiment, in step 1408, the processor determines the position within the decompressed frame based on the horizontal and vertical texture coordinates provided in the call. In at least one embodiment, the coordinates specify the precise pixel or region of interest within the decompressed frame.

[0152] In at least one embodiment, in step 1410, the processor writes the provided pixel data to the determined location within the decompressed frame. In at least one embodiment, the step modifies the pixel value at the specified coordinates to reflect the data provided in the API call.

[0153] In at least one embodiment, combined Figure 14 At least one of the described features is utilizing Figure 1-13 and Figure 15-34 This is achieved by at least one feature of any system and / or process shown and / or described. In at least one embodiment, it is combined with Figure 14 At least one of the described features is used to achieve Figure 1-13 and Figure 15-34 At least a portion of any system and / or process shown and / or described.

[0154] Figure 15 A block diagram 1500 of a driver and / or runtime is shown according to at least one embodiment, the driver and / or runtime including one or more libraries for providing one or more application programming interfaces (APIs). In at least one embodiment, the software program 1502 is a software module stored on a processor, such as... Figure 1 The processor described herein. In at least one embodiment, the software program 1502 includes one or more software modules. In at least one embodiment, the software program 1502 is a collection of software code, commands, instructions, or other text sequences for instructing a computing device to perform one or more computational operations and / or to invoke one or more other instruction sets (e.g., one or more APIs 1510 or one or more API functions 1512) to be executed. In at least one embodiment, the software program is a compiler.

[0155] In at least one embodiment, one or more APIs 1510 are software instruction sets that, when executed, cause one or more processors to perform one or more computational operations. In at least one embodiment, one or more APIs 1510 are distributed or otherwise provided as part of one or more libraries 1506, runtimes 1504, drivers, and / or any other grouping of software and / or executable code further described herein. In at least one embodiment, one or more APIs 1510 perform one or more computational operations in response to a call to software program 1502. In at least one embodiment, the functionality provided by one or more APIs 1510 includes one or more software functions 1512, such as software functions that can be used to accelerate one or more portions of software program 1502 using one or more parallel processing units (PPUs) (e.g., graphics processing units (GPUs)).

[0156] In at least one embodiment, one or more APIs 1510 are hardware interfaces for one or more circuits that perform one or more computational operations. In at least one embodiment, the one or more APIs 1510 described herein are implemented as one or more circuits that perform combined... Figure 1-14 The described one or more techniques. In at least one embodiment, the software program 1502 includes instructions that, if executed, cause one or more hardware devices and / or circuits to perform a combination Figure 1-20 The above-described one or more technologies.

[0157] In at least one embodiment, software program 1502 (e.g., user-implemented software program) utilizes one or more APIs 1510 to perform various computational operations, such as memory reservation, matrix multiplication, arithmetic operations, or any computational operation performed by a parallel processing unit (PPU) (e.g., a graphics processing unit (GPU)), as further described herein. In at least one embodiment, one or more APIs 1510 provide a set of callable functions 1512 (referred to herein as APIs, API functions, and / or functions), each of which performs one or more computational operations, such as computational operations related to parallel computing.

[0158] In at least one embodiment, software program 1502 interacts with or otherwise communicates with one or more APIs 1510 to perform one or more computational operations using one or more PPUs (e.g., GPUs). In at least one embodiment, the computational operations include at least one or more sets of computational operations that are accelerated by being performed at least partially by the PPUs. In at least one embodiment, software program 1502 interacts with one or more APIs 1510 to facilitate parallel computing using remote or local interfaces.

[0159] In at least one embodiment, the interface is software instructions that, if executed, provide access to one or more functions 1512 provided by one or more APIs 1510. In at least one embodiment, the software program 1502 uses a native interface when a software developer compiles the software program 1502 in conjunction with one or more libraries 1506 that include one or more APIs 1510 or otherwise provide access to one or more APIs 1510. In at least one embodiment, the software program 1502 is statically compiled in conjunction with pre-compiled libraries 1506 or uncompiled source code including instructions for executing one or more APIs 1510. In at least one embodiment, the software program 1502 is dynamically compiled, and the one or more software programs are linked to one or more pre-compiled libraries 1506 that include one or more APIs 1510 using a linker.

[0160] In at least one embodiment, when a software developer executes a software program that communicates with a library 1506 including one or more APIs 1510, or otherwise via a network or other remote communication medium, the software program 1502 uses a remote interface. In at least one embodiment, the library 1506 including one or more APIs 1510 is executed by a remote computing service (e.g., a computing resource service provider). In another embodiment, the library 1506 including one or more APIs 1510 is executed by any other computing host that provides the one or more APIs 1510 to the software program 1502.

[0161] In at least one embodiment, software program 1502 utilizes one or more APIs 1510 to allocate and otherwise manage memory to be used by software program 1502. In at least one embodiment, software program 1502 utilizes one or more APIs 1510 to allocate and otherwise manage memory to be used by one or more portions of software program 1502, which will be accelerated using one or more PPUs (e.g., GPUs or any other accelerators or processors further described herein). In at least one embodiment, software program 1502 selects one or more portions of one or more neural networks to be deactivated during the training of one or more neural networks, based at least in part on whether one or more portions of one or more neural networks will still be used after training the one or more neural networks.

[0162] In at least one embodiment, each of one or more APIs 1510 is an API for facilitating parallel computing. In at least one embodiment, each of one or more APIs 1510 is any other API further described herein. In at least one embodiment, each of one or more APIs 1510 is provided by a driver and / or runtime 1504. In at least one embodiment, each of one or more APIs 1510 is provided by a CUDA user-mode driver. In at least one embodiment, each of one or more APIs 1510 is provided by a CUDA runtime. In at least one embodiment, the driver is a data value and software instructions that, if executed, perform or otherwise facilitate the operation of one or more functions 1512 of one or more APIs 1510 during the loading and execution of one or more portions of the software program 1502. In at least one embodiment, runtime 1504 is a data value and software instructions that, if executed, perform one or more functions 1512 of one or more APIs 1510 or otherwise facilitate the operation of one or more functions 1512 of one or more APIs 1510 during the execution of software program 1502. In at least one embodiment, software program 1502 utilizes one or more APIs 1510 implemented or otherwise provided by the driver and / or runtime 1504 to perform combinatorial arithmetic operations by software program 1502 during execution by one or more PPUs (e.g., GPUs).

[0163] In at least one embodiment, software program 1502 utilizes one or more APIs 1510 provided by a driver and / or runtime 1504 to perform combinatorial arithmetic operations on one or more PPUs (e.g., GPUs). In at least one embodiment, one or more APIs 1510 provide combinatorial arithmetic operations via a driver and / or runtime 1504, as described above. In at least one embodiment, software program 1502 utilizes one or more APIs 1510 provided by a driver and / or runtime 1504 to allocate or otherwise reserve one or more blocks of memory 1514 for one or more PPUs (e.g., GPUs). In at least one embodiment, software program 1502 utilizes one or more APIs 1510 provided by a driver and / or runtime 1504 to allocate or otherwise reserve blocks of memory. In at least one embodiment, one or more APIs 1510 are used to perform combinatorial arithmetic operations, as incorporated herein by reference. Figure 1-14 Any of the descriptions.

[0164] In at least one embodiment, block diagram 1500 illustrates a processor including one or more circuitry for executing one or more software programs to combine two or more application programming interfaces (APIs) into a single API. In at least one embodiment, block diagram 1500 illustrates a system including one or more processors for executing one or more software programs to combine two or more application programming interfaces (APIs) into a single API. In at least one embodiment, the API is used to identify one or more expected software outputs for comparison with one or more other software outputs to be generated by the software.

[0165] In at least one embodiment, some or all of the processes described herein (or any other described processes, or variations and / or combinations thereof) may be executed under the control of one or more computer systems configured with executable instructions and / or other data, and may be implemented as executable instructions that execute jointly on processor 100 or a combination of processors 100. In at least one embodiment, the executable instructions and / or other data may be stored on a non-transitory computer-readable storage medium (e.g., a computer program persistently stored on a magnetic, optical, or flash memory medium).

[0166] Figure 16An example of a system 1600 according to at least one embodiment is shown. The system 1600 may include software and hardware for performing multiple APIs to access or modify one or more frames of compressed video or otherwise perform any of the operations described herein. The system 1600 may include a storage device 1602 and a processor 1608. The storage device 1602 may include, for example, memory, a cache, or other storage devices as further described below. The storage device 1602 may be independent of the processor 1608, or the storage device 1602 may be included within the processor 1608 (e.g., within storage device 1612). In at least one embodiment, software program 1604 and / or software library (or instructions) 1606 may be stored in memory, a cache, or other storage devices and provided to the processor 1608 to cause one or more circuits of the processor 1608 to perform the operations described herein. In at least one embodiment, software program 1604 and / or software library (or instructions) 1606 may be integrated into one or more circuits of the processor 1608. Software program 1604 (which can be used to perform any of the operations described herein) may be stored on storage device 1602.

[0167] In at least one embodiment, software program 1604 may include one or more software modules. In at least one embodiment, the one or more software modules may include one or more GPU cores and one or more host functions. In at least one embodiment, unless the context explicitly specifies otherwise, in any implementation described herein, "module" refers to any combination of software logic, firmware logic, hardware logic, and / or circuitry configured to provide the functionality described herein. In at least one embodiment, "software" is embodied as a software package, code, and / or instruction set or instructions, and "hardware" as used in any implementation described herein includes (e.g., individually or in any combination) hardwired circuitry, programmable circuitry, state machine circuitry, fixed-function circuitry, execution unit circuitry, and / or firmware storing instructions executed by programmable circuitry. In at least one embodiment, a module may be embodied individually or collectively as circuitry forming part of a larger system (e.g., an integrated circuit (IC), a system-on-a-chip (SoC), etc.). In at least one embodiment, a module is combined with any suitable processing unit and / or combination of processing units (e.g., one or more CPUs, GPUs, GPGPUs, PPUs, and / or variants thereof, including those further described herein) to perform one or more processes.

[0168] In at least one embodiment, software program 1604 may include a collection of software code, commands, instructions, or other text sequences for instructing a computing device to perform one or more computational operations and / or invoke one or more other instruction sets (e.g., APIs or API functions or Instruction Set Architecture (ISA) level instructions) to be run or otherwise executed. In at least one embodiment, the API, when executed, enables a processor to access and modify video frames in compressed video. Instructions (e.g., hardware instructions) or microcode may contain ISA-level instructions, which may include native ISA instructions or non-native ISA instructions. Software program 1604 and / or software library (or instruction) 1606 (e.g., one or more modules) may be distributed among multiple processors that can communicate via buses, networks, writes to shared memory, and / or any suitable communication method (e.g., the communication methods described herein).

[0169] In at least one embodiment, system 1600 may include one or more software libraries 1606, which may provide one or more API and / or ISA instructions. In at least one embodiment, one or more API and / or ISA instructions may be used to cause information to be stored and to store instructions for one or more decompression algorithms for decompressing said information; to cause one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms to be stored; to indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; to perform interpolation on two or more pixels of said one or more video frames at least in part based on one or more frame identifiers of said one or more video frames; to indicate one or more properties of said one or more pixels at least in part based on one or more identifiers of said one or more video frames containing said one or more pixels; and to modify one or more properties of said one or more pixels at least in part based on one or more identifiers of said one or more video frames containing said one or more pixels. In at least one embodiment, one or more software libraries 1606 may be included in a driver and / or runtime. In at least one embodiment, software library 1606 (e.g., including one or more API and / or ISA instructions) may include software instruction sets that, if run or otherwise executed, cause processor 1608 to perform one or more computational operations, such as any of those described herein. In at least one embodiment, one or more API and / or ISA instructions may be distributed or otherwise provided as part of one or more packages of software library 1606, runtime, drivers, and / or any other grouping of software and / or executable code further described herein. In at least one embodiment, one or more API and / or ISA instructions may perform one or more computational operations in response to a call to software program 1604.

[0170] Processor 1608 may include any number of processors and any suitable processing units and / or combinations of processing units, such as, but not limited to, a central processing unit (“CPU”), a graphics processing unit (“GPU”), or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processors, parallel processors, GPGPUs, DPUs, and / or variations thereof, including processors further described herein), including any processors described herein, such as, but not limited to, those described herein. Figure 18-30BThe processor 1608 can retrieve or obtain instructions (e.g., one or more API and / or ISA instructions) from storage device 1602 using, for example, instruction fetch 1616 (e.g., for an instruction fetch phase). In at least one embodiment, the instructions may include instructions for causing information to be stored and instructions for storing indications of one or more decompression algorithms for decompressing the information. In at least one embodiment, the instructions may include instructions for causing one or more identifiers of one or more videos and one or more indications of one or more pixel interpolation algorithms to be stored. In at least one embodiment, the instructions may include instructions for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos. In at least one embodiment, the instructions may include instructions for performing interpolation on two or more pixels of one or more video frames, at least partially based on one or more frame identifiers of one or more video frames. In at least one embodiment, the instructions may include instructions for indicating one or more properties of one or more pixels, at least partially based on one or more identifiers of one or more video frames containing one or more pixels. In at least one embodiment, the instructions may include instructions for modifying one or more properties of the one or more pixels, at least in part, based on one or more identifiers of one or more video frames containing one or more pixels. In at least one embodiment, processor 1608 may include storage device 1612 and instruction queue 1610 for storing and queuing instructions fetched from storage device 1602. In at least one embodiment, the fetched instructions may be decoded by decoder 1618 to determine what operation processor 1608 should perform (e.g., during the instruction decoding phase). In at least one embodiment, processor 1608 may fetch additional operands (data) that can be used for the instructions, which may be stored, for example, in registers or storage device 1612. In at least one embodiment, micro-operations 1620 may perform operations on the data stored in one or more registers or storage device 1612. For example, each step of the instructions fetched by processor 1608 may be broken down during execution so that processor 1608 can execute the instructions step-by-step through a series of micro-operations 1620. In at least one embodiment, the program counter (PC) 1614 may store the address of the next instruction and may be updated to point to the next instruction to be executed by the processor 1608.

[0171] In at least one embodiment, processor 1608 may execute instructions (e.g., during execution phase). For example, processor 1608 may perform operations specified by the instructions, such as arithmetic operations, logical operations, or data transfers. In at least one embodiment, computing unit 1622 may execute instructions to perform any of the operations described herein. In at least one embodiment, computing unit may include ALU 1624 (Arithmetic Logic Unit), which can be used to perform arithmetic and logical operations. In at least one embodiment, computing unit may include FPU (Floating Point Unit) 1626, which can be used to perform floating-point calculations. In at least one embodiment, other circuitry 1628 may be used to perform other operations, such as vector and / or scalar operations. In at least one embodiment, accelerator 1630 may include one or more matrix multiplication accelerators, one or more parallel processing units (PPUs) (e.g., GPUs), or any other accelerators or processors further described herein. In at least one embodiment, software program 1604 may utilize one or more API and / or ISA instructions to perform various computational operations, such as matrix multiplication, arithmetic operations, or any other computational operations further described herein, using accelerator 1630. In at least one embodiment, one or more computational operations using accelerator 1630 may include at least one or more sets of computational operations that are accelerated at least in part by being performed by accelerator 1630, including: causing information to be stored and storing instructions for one or more decompression algorithms for decompressing the information; causing one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms to be stored; storing instructions for one or more decompression algorithms for decompressing the information; instructions for one or more decompression algorithms to be used to access one or more frames of one or more videos; performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels.

[0172] In at least one embodiment, system 1600 can be used to execute one or more instructions, including functions or operations, such as combinations. Figure 1-15The operations described herein. In at least one embodiment, system 1600, including one or more processors, causes one or more circuits to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; store instructions for one or more decompression algorithms for decompressing the information; instructions for one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and / or otherwise perform the operations described herein. In at least one embodiment, system 1600 is included Figure 1-15 The system shown and / or otherwise includes Figure 1-15 The system shown herein enables one or more circuits to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; store instructions for one or more decompression algorithms for decompressing the information; instructions for one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and / or otherwise perform the operations described herein. In at least one embodiment, system 1600 includes... Figure 16-35CThe hardware shown herein includes, for example, one or more hardware components for causing information to be stored and storing instructions for one or more decompression algorithms for decompressing said information, causing one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms to be stored, storing instructions for one or more decompression algorithms for decompressing said information, instructions that will be used to access one or more decompression algorithms for one or more frames of one or more videos, performing interpolation on two or more pixels of said one or more video frames at least in part based on one or more frame identifiers of said one or more video frames, indicating one or more properties of said one or more pixels at least in part based on one or more identifiers of said one or more video frames containing one or more pixels, modifying one or more properties of said one or more pixels at least in part based on one or more identifiers of said one or more video frames containing one or more pixels, and / or otherwise performing the operations described herein.

[0173] Data Center

[0174] Figure 17 An example data center 1700 according to at least one embodiment is illustrated. Data center 1700 may include one or more rooms having racks 1702 and auxiliary equipment for housing one or more racks 1702 and one or more substrates 1704. Racks 1702 may include one or more substrates 1704. Racks 1702 may include enclosures for housing and supporting individual substrates 1704. Operational aspects of racks 1702 may be adjustable at the rack level (corresponding to a group of substrates 1704) or at the substrate level (corresponding to an individual substrate 1704), among other options. Racks 1702 or substrates 1704 may have specific selected maximum operating parameters, such as, but not limited to, power consumption, operating frequency, etc. Data center 1700 may be supported by various cooling systems, such as, but not limited to, cooling towers, cooling loops, pumps, and other support systems. Cooling systems may include sensors and controllers for monitoring and managing the cooling characteristics of racks 1702. Substrates 1704 within racks 1702 may draw operating power from one or more power distribution units (PDUs; not shown). PDUs can be arranged within racks 1702, for example, between racks 1702 that include substrate 1704, or within racks 1702 that also house substrate 1704.

[0175] Rack 1702 and substrate 1704 may include subsystems, modules, add-in cards, and other semiconductor components. Substrate 1704 may include one or more computing units 1706, each computing unit 1706 may include one or more processors 1708, one or more memories 1710, and an interface controller 1712. Computing unit 1706 may include any number of processors, such as, but not limited to, a central processing unit (“CPU”), a graphics processing unit (“GPU”), or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processors, etc.), including any processors described herein, such as, but not limited to, those described herein. Figure 18-30B The processor in the computing unit 1706 may include one or more memory storage devices 1710 (e.g., dynamic read-only memory, solid-state storage devices, or disk drives), as well as network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power supply modules, and cooling modules, etc. One or more computing units 1706 may be a server having one or more of the aforementioned computing resources.

[0176] Computing unit 1706 may include individual computing unit groups housed in one or more racks (not shown), or in numerous racks within data centers in different geographical locations (also not shown). Individual computing unit groups may include grouped computing, networking, memory, or storage resources that can be configured or allocated to support one or more workloads. Several computing units (e.g., including CPUs and / or other processors) may be grouped within one or more racks to provide computing resources to support one or more workloads. Resource coordinator 1714 may configure or otherwise control one or more computing units 1706 or computing unit groups. Resource coordinator 1714 may include a Software Design Infrastructure (“SDI”) management entity for data center 1700. Resource coordinator 1714 may include hardware, software, or some combination thereof.

[0177] Data center 1700 may include any one or any combination of the framework layer 1720, software layer 1730, and application layer 1740. For example... Figure 17As shown, framework layer 1720 includes a job scheduler 1722, a configuration manager 1724, a resource manager 1726, and a distributed file system 1728. Framework layer 1720 may include a framework for supporting software 1732 of software layer 1730 and / or one or more applications 1742 of application layer 1740. Software 1732 or application 1742 may respectively include web-based service software or applications, such as, but not limited to, software or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure. Framework layer 1720 may be a type of free and open-source software web application framework, such as, but not limited to, Apache Spark. TM (Hereinafter referred to as "Spark"), which can utilize the distributed file system 1728 for large-scale data processing (e.g., "big data"). The job scheduler 1722 may include Spark drivers, which facilitate the scheduling of workloads supported by various layers of the data center 1700. The configuration manager 1724 may be able to configure different layers, such as, but not limited to, the software layer 1730 and the framework layer 1720 (which includes Spark and the distributed file system 1728 for supporting large-scale data processing). The resource manager 1726 may be able to manage clustered or grouped compute units 1706 mapped to or allocated to support the distributed file system 1728 and the job scheduler 1722. The resource manager 1726 may coordinate with the resource coordinator 1714 to manage these mapped or allocated compute resources.

[0178] Software 1732 may be included in software layer 1730, and may include software used by at least a portion of computing units 1706, one or more computing units 1706, groups of computing units 1706, and / or the distributed file system 1728 of framework layer 1720. One or more types of software may include, but are not limited to, internet web search software, email virus scanning software, database software, and streaming video content software.

[0179] Application 1742 may be included in application layer 1740 and may include one or more types of applications used by at least portions of computing unit 1706, one or more computing units 1706, groups of computing units 1706, and / or the distributed file system 1728 of framework layer 1720. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing applications, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used in conjunction with one or more embodiments.

[0180] Any of the Configuration Manager 1724, Resource Manager 1726, and Resource Coordinator 1714 can implement any number and type of self-modification actions based on any amount and type of data obtained in any technically feasible manner. Self-modification actions can alleviate the burden on data center operators of Data Center 1700 to make potentially erroneous configuration decisions and may avoid underutilized and / or poorly performing portions of the data center.

[0181] Data center 1700 may include tools, services, software, or other resources for training one or more machine learning models according to one or more embodiments described herein, or for using one or more machine learning models to predict or infer information. For example, a machine learning model can be trained by calculating weight parameters based on a neural network architecture using the software and computing resources described above regarding data center 1700. The trained machine learning model corresponding to one or more neural networks can be used with the resources described above regarding data center 1700 to infer or predict information using weight parameters calculated through one or more training techniques described herein.

[0182] The Data Center 1700 can use CPUs, application-specific integrated circuits (ASICs), GPUs, FPGAs, or other hardware (e.g., Figure 18-30B The embodiments described herein can be used to perform some or all of the processes and techniques described elsewhere, such as, but not limited to, training and / or inference using the resources described above. Furthermore, one or more of the software and / or hardware resources described above can be configured as a service to allow a user to train or perform information inference, such as, but not limited to, image recognition, speech recognition, or other artificial intelligence services.

[0183] In at least one embodiment, processor 1708 may include one or more processors and / or one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. In at least one embodiment, processor 1708 is configured by software 1732 to: store information and store indications of one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein. Data center 1700 may use logic, CPU, application-specific integrated circuit (ASIC), GPU, FPGA, or other hardware (e.g., Figure 18-30B (The embodiments in the document) can be used to perform any of the operations described above or elsewhere in this document.

[0184] processor

[0185] The following figure illustrates, in a non-limiting manner, example processors and processing systems that can be used to cause information to be stored and to store instructions for one or more decompression algorithms for decompressing said information, to cause one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms to be stored, to indicate one or more decompression algorithms to be used to access one or more frames of one or more videos, to perform interpolation on two or more pixels of said one or more video frames at least in part based on one or more frame identifiers of said one or more video frames, to indicate one or more properties of said one or more pixels at least in part based on one or more identifiers of said one or more video frames containing said one or more pixels, and to modify one or more properties of said one or more pixels at least in part based on one or more identifiers of said one or more video frames containing said one or more pixels, or otherwise perform some or all of the processes, operations and / or techniques described elsewhere herein. Example processors and processing systems can be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; the instructions will be used to access one or more decompression algorithms for one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein. Processors and processing systems may include logic, central processing units (CPUs), application-specific integrated circuits (ASICs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), XPUs (i.e., any computing architecture best suited to the needs of the application), or other hardware (e.g., Figure 18-30BThe processor and / or processing system described herein may include one or more circuits configured to cause information to be stored and to store indications of one or more decompression algorithms for decompressing said information, to cause one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, to indicate one or more decompression algorithms to be used to access one or more frames of one or more videos, to perform interpolation on two or more pixels of said one or more video frames at least in part based on one or more frame identifiers of said one or more video frames, to indicate one or more properties of said one or more pixels at least in part based on one or more identifiers of said one or more video frames containing said one or more pixels, and to modify one or more properties of said one or more pixels at least in part based on one or more identifiers of said one or more video frames containing said one or more pixels, or otherwise perform any of the operations described above or elsewhere herein. As used herein, one or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein. Figure 35A and Figure 35B Logic 3515, as described elsewhere in this document, is illustrated and can be used in one or more devices to perform, for example (but not limited to), the operations discussed herein according to at least one embodiment. Logic can refer to, for example, any combination of software logic, hardware logic, and / or firmware logic for providing the functionality and / or operations described herein, wherein the logic can be collectively or individually embodied as circuitry forming part of a larger system, such as an integrated circuit (IC), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a system-on-a-chip (SoC), or one or more processors (e.g., a CPU, a GPU).

[0186] Figure 18 A processor according to at least one embodiment is illustrated, which is a system-on-a-chip (SOC) 1800 (which may be referred to as a system-on-a-chip, superchip, or other names). SOC 1800 may include processor complex 1810 and processor complex 1840. SOC 1800 may include any number of processor complexes 1810 and / or processor complexes 1840, which may include any number of processors described herein in any combination, such as, but not limited to, processors described herein in any combination. Figure 18-30B The processor in the system. For example, processor 1810 may include a central processing unit (CPU), and processor 1840 may include a graphics processor. Alternatively, processor 1810 may include a graphics processor, and processor 1840 may include a graphics processor. The SOC 1800 may include any number of display controllers 1892, any number of multimedia engines 1894, any number of I / O interfaces 1870, any number of memory controllers 1880, and any number of fabrics 1860 in any combination. For ease of explanation, this document uses reference numbers identifying objects and bracket numbers identifying instances (if required) to denote multiple instances of similar objects. The SOC 1800 may include a processor from Broadcom Corporation, Palo Alto, California.

[0187] Processor complex 1810 may include a CPU, processor complex 1840 may include a GPU, and SOC 1800 may include a processing unit integrating processor complex 1810 and processor complex 1840 onto a single chip. Certain tasks may be assigned to processor complex 1810, while other tasks may be assigned to processor complex 1840. Processor complex 1810 may be configured to execute main control software associated with SOC 1800, such as, but not limited to, an operating system. Processor complex 1810 may be the main processor of SOC 1800, controlling and coordinating the operation of other processors. Processor complex 1810 may issue commands that control the operation of processor complex 1840 to perform some or all of the operations described herein. Processor complex 1810 may be configured to execute host-executable code derived from CUDA or other source code (e.g., HIP source code), while processor complex 1840 may be configured to execute device-executable code derived from CUDA or other source code to perform any of the operations described herein.

[0188] Processor complex 1810 may include cores 1820(1)-1820(4) and cache (e.g., L3 cache) 1830 for storing information for performing the operations described herein. Processor complex 1810 may include any number of cores 1820 in any combination and any number and type of cache. Cores 1820 may be configured to execute instructions of a specific instruction set architecture (“ISA”) to perform some or all of the operations described herein. Each core 1820 may include a CPU core. Cores 1820(1)-1820(4) may be referred to as compute units or arithmetic units. SOC 1800 may include any number of processor complexes 1810, architecture 1860, I / O interface 1870, and memory controller 1880.

[0189] Each core 1820 may include a fetch / decode unit 1822, an integer execution engine 1824, a floating-point execution engine 1826, and an L2 cache 1828. The fetch / decode unit 1822 may fetch instructions to perform some or all of the operations described herein (e.g., but not limited to APIs compiled into instructions) and decode those instructions, generate micro-operations, and dispatch individual micro-instructions to the integer execution engine 1824 and / or the floating-point execution engine 1826. The fetch / decode unit 1822 may concurrently dispatch one micro-instruction to the integer execution engine 1824 and another micro-instruction to the floating-point execution engine 1826. The integer execution engine 1824 may perform integer and memory operations. The floating-point engine 1826 may perform floating-point and vector operations. The fetch / decode unit 1822 may dispatch micro-instructions to one or more execution engines, which may replace both the integer execution engine 1824 and the floating-point execution engine 1826.

[0190] Each core 1820(i) (where i is an integer representing a specific instance of core 1820) can access the L2 cache 1828(i) included in core 1820(i). Each core 1820 included in core complex 1810(j) (where j is an integer representing a specific instance of core complex 1810) can be connected to other cores 1820 included in core complex 1810(j) via the L3 cache 1830(j) included in core complex 1810(j). The cores 1820 included in core complex 1810(j) (where j is an integer representing a specific instance of core complex 1810) can access all L3 caches 1830(j) included in core complex 1810(j). The L3 cache 1830 can include any number of slices.

[0191] Processor complex 1840 may be a graphics complex that can be configured to perform computational operations (e.g., the computational operations described herein) in a highly parallel manner. Processor complex 1840 may be configured to perform graphics pipeline operations, such as, but not limited to, drawing commands, pixel operations, geometric calculations, and other operations associated with rendering an image to a display. Processor complex 1840 may be configured to perform graphics-independent operations, such as, but not limited to, neural network training and / or simulation. Processor complex 1840 may be configured to perform both graphics-related and graphics-independent operations.

[0192] The processor complex 1840 may include any number of compute units 1850(1)-1850(N) (where N is any integer greater than 1) and an L2 cache 1842. The compute units 1850 may share the L2 cache 1842, which may store information that will be used to perform some or all of the operations described herein. The L2 cache 1842 may be partitioned. The processor complex 1840 may include any number of compute units 1850 and any number (including zero) and type of cache. The processor complex 1840 may include any number of dedicated graphics hardware.

[0193] Each compute unit 1850 may include any number of SIMD units 1852(1)-1852(N) (where N is any integer greater than 1) and shared memory 1854. Each SIMD unit 1852 may implement a SIMD architecture and may be configured to perform some or all of the operations described herein in parallel. Each compute unit 1850 may execute any number of thread blocks, but each thread block may execute on a single compute unit 1850, although in some embodiments, the thread block may execute on multiple compute units. A thread block may include any number of execution threads. A workgroup may be a thread block. Each SIMD unit 1852 may execute a set of threads. A set of threads (e.g., 16 threads), also referred to as a warp, subgroup, or wavefront (e.g., used by AMD and Intel), may belong to a single thread block and be configured to process different datasets based on a single instruction set. Prediction may be used to disable one or more threads in a warp, subgroup, or wavefront. A lane may be a thread. A work item can be a thread, such as (but not limited to) an OpenCL thread. Different thread bundles, subgroups, or wavefronts within a thread block can be synchronized together and communicate via shared memory 1854. Each compute unit 1850 can include one or more thread block clusters, where thread block clusters can implement programmable control over locality at a larger granularity than a single thread block of a single streaming multiprocessor (SM). Thread block clusters (also referred to as “clusters”) can support multiple thread blocks running concurrently across streaming multiprocessors, thereby synchronously and cooperatively acquiring, exchanging, or otherwise using data. In at least one embodiment, a streaming multiprocessor (“SM”) can refer to a streaming microprocessor, a streaming processor (“SP”), a streaming processing unit (“SPU”), a compute unit (“CU”), an execution unit (“EU”), and / or a slice, where a slice in this context can refer to a portion of the processing resources within a processing unit (e.g., 16 cores, a ray tracing unit, a thread bootstrap, or a scheduler).

[0194] Structure 1860 can be a system interconnect that facilitates data and control transfers across processor complex 1810, processor complex 1840, I / O interface 1870, memory controller 1880, display controller 1892, and multimedia engine 1894, for example, to perform some or all of the operations described herein. SOC 1800 can include any number and type of system interconnects other than or replacing structure 1860, facilitating data and control transfers across any number and type of directly or indirectly linked components, either internally or externally to SOC 1800. I / O interface 1870 can represent any number and type of I / O interfaces (e.g., PCI, PCI extensions (“PCI-X”), PCIe, Gigabit Ethernet (“GBE”), USB, etc.). Various types of peripheral devices can be coupled to I / O interface 1870. Peripherals that can be coupled to I / O interface 1870 can include keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, network interface cards, etc.

[0195] Display controller 1892 can display images on one or more display devices, such as, but not limited to, liquid crystal displays (“LCD”) devices. Multimedia engine 1894 can include any number and type of circuitry related to multimedia, such as, but not limited to, video decoders, video encoders, image signal processors, etc. Memory controller 1880 can facilitate data transfer between SOC 1800 and unified system memory 1890. Processor complex 1810 and processor complex 1840 can share unified system memory 1890. Unified system memory 1890 can include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as, but not limited to, synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. Unified system memory 1890 can include 3D stacked memory, including but not limited to high bandwidth memory (HBM), HBM2e, or HDM3.

[0196] The SOC 1800 can implement a memory subsystem comprising any number and type of memory controllers 1880 and memory devices (e.g., shared memory 1854), which may be dedicated to a single component or shared among multiple components to perform any of the operations described herein. The SOC 1800 can implement a cache subsystem comprising one or more cache memories (e.g., L2 cache 1828, L3 cache 1830, and L2 cache 1842), each cache memory may be dedicated to any number of components (e.g., core 1820, core complex 1810, SIMD unit 1852, compute unit 1850, and processor complex 1840) or shared among any number of components (e.g., core 1820, core complex 1810, SIMD unit 1852, compute unit 1850, and processor complex 1840).

[0197] In at least one embodiment, the SOC 1800 may include one or more circuitry for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0198] Figure 19A A parallel processor 1900 according to at least one embodiment is illustrated. The parallel processor 1900 may be implemented using one or more circuits and may be referred to as a programmable processor (e.g., CPU and / or GPU), logic, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other hardware (e.g., Figure 18-30B (The embodiments in the document) are used to perform any of the operations described above or elsewhere herein.

[0199] Parallel processor 1900 may include parallel processing unit 1902 for performing any of the operations described above or elsewhere herein. Parallel processing unit 1902 may include I / O unit 1904 that enables communication with other devices, including other instances of parallel processing unit 1902. I / O unit 1904 may be directly connected to other devices. I / O unit 1904 may be connected to other devices via the use of a hub or switch interface, such as, but not limited to, memory hub 1905. The connection between memory hub 1905 and I / O unit 1904 may form a communication link 1913. I / O unit 1904 may be connected to host interface 1906 and memory crossbar switch 1916, wherein host interface 1906 receives commands directed to perform processing operations, and memory crossbar switch 1916 receives commands directed to perform memory operations.

[0200] When host interface 1906 receives a command buffer via I / O unit 1904, host interface 1906 can route the work operations that execute these commands to front-end 1908. Front-end 1908 can be coupled to scheduler 1910 (which may be referred to as sequencer), which is configured to distribute commands or other work items to processing cluster array 1912. Scheduler 1910 can ensure that processing cluster array 1912 is correctly configured and in an active state before tasks are distributed to the cluster of processing cluster array 1912. Scheduler 1910 can be implemented via firmware logic executed on a microcontroller. The microcontroller-implemented scheduler 1910 can be configured to perform complex scheduling and work distribution operations at both coarse and fine granular levels, thereby enabling fast preemption and context switching of threads executing on processing array 1912. Host software can validate workloads scheduled on processing cluster array 1912 via one of multiple graphics processing paths. The workload can then be automatically distributed to the processing array cluster 1912 by the scheduler 1910 logic within the microcontroller, which includes the scheduler 1910.

[0201] Processing cluster array 1912 can perform any of the operations described above or elsewhere herein, and may include up to “N” processing clusters (e.g., clusters 1914A, 1914B through 1914N), where “N” represents a positive integer (which may be a different integer “N” than used in other diagrams). Each cluster 1914A-1914N in processing cluster array 1912 can execute a large number of concurrent threads. Scheduler 1910 may use various scheduling and / or work distribution algorithms to distribute work to clusters 1914A-1914N in processing cluster array 1912, which may vary depending on the workload generated by each type of program or computation. Scheduling may be dynamically handled by scheduler 1910 or may be assisted by compiler logic during the compilation of program logic configured to be executed by processing cluster array 1912. Different clusters 1914A-1914N of processing cluster array 1912 may be assigned to process different types of programs or perform different types of computations.

[0202] The processing cluster array 1912 can be configured to perform various types of parallel processing operations, such as, but not limited to, any of the operations described above or elsewhere herein. The processing cluster array 1912 can be configured to perform general-purpose parallel computing operations. For example, the processing cluster array 1912 may include logic for performing processing tasks, including filtering video and / or audio data, performing modeling operations (including physical operations), and performing data transformations.

[0203] Processing cluster array 1912 can be configured to perform parallel graphics processing operations. Processing cluster array 1912 may include additional logic for supporting the execution of such graphics processing operations, including but not limited to texture sampling logic for performing texture operations, as well as tessellation logic and other vertex processing logic. Processing cluster array 1912 can be configured to execute shader programs related to graphics processing, such as, but not limited to, vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. Parallel processing unit 1902 can transfer data from system memory via I / O unit 1904 for processing. During processing, the transferred data may be stored in on-chip memory (e.g., parallel processor memory 1922) during processing and then written back to system memory.

[0204] When the parallel processing unit 1902 is used to perform graphics processing, the scheduler 1910 can be configured to divide the processing workload into tasks of approximately equal size to better distribute graphics processing operations to multiple clusters 1914A-1914N of the processing cluster array 1912. Each part of the processing cluster array 1912 can be configured to perform different types of processing. For example, a first part can be configured to perform vertex shading and topology generation, a second part can be configured to perform tessellation and geometry shading, and a third part can be configured to perform pixel shading or other screen-space operations to produce a rendered image for display. Intermediate data generated by one or more clusters 1914A-1914N can be stored in a buffer to allow intermediate data to be transferred between clusters 1914A-1914N for further processing.

[0205] Processing cluster array 1912 can receive processing tasks to be executed via scheduler 1910, which receives commands defining the processing tasks from front end 1908. Processing tasks may include indexes of data to be processed, such as surface (patch) data, primitive data, vertex data, and / or pixel data, as well as state parameters and commands defining how to process the data (e.g., which program to execute). Scheduler 1910 can be configured to retrieve the index corresponding to the task, or can receive the index from front end 1908. Front end 1908 can be configured to ensure that processing cluster array 1912 is configured to be active before the workload specified by the incoming command buffer (e.g., batch buffer, push buffer, etc.) is initiated.

[0206] Each instance of one or more instances of parallel processing unit 1902 may be coupled to parallel processor memory 1922 to perform any of the operations described above or elsewhere herein. Parallel processor memory 1922 may be accessed via memory crossbar switch 1916, which may receive memory requests from processing cluster array 1912 and I / O unit 1904. Memory crossbar switch 1916 may access parallel processor memory 1922 via memory interface 1918. Memory interface 1918 may include multiple partition units (e.g., partition units 1920A, 1920B through 1920N), each partition unit may be coupled to a portion (e.g., a memory cell) of parallel processor memory 1922. The number of partition units 1920A-1920N can be configured to be equal to the number of memory units, such that the first partition unit 1920A has a corresponding first memory unit 1924A, the second partition unit 1920B has a corresponding memory unit 1924B, and the Nth partition unit 1920N has a corresponding Nth memory unit 1924N. The number of partition units 1920A-1920N may not be equal to the number of memory units.

[0207] Memory cells 1924A-1924N may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as, but not limited to, synchronous graphics random access memory (SGRAM), which includes graphics double data rate (GDDR) memory. Memory cells 1924A-1924N may also include 3D stacked memory, including but not limited to high-bandwidth memory (HBM), HBM2e, or HDM3. Render targets (e.g., but not limited to framebuffers or texture maps) may be stored in memory cells 1924A-1924N, allowing partitioning cells 1920A-1920N to write portions of each render target in parallel to efficiently utilize the available bandwidth of the parallel processor memory 1922. A local instance of the parallel processor memory 1922 may not be included to support a unified memory design that combines system memory with local cache memory.

[0208] Any cluster 1914A-1914N in the processing cluster array 1912 can process data to be written to any memory cell 1924A-1924N within the parallel processor memory 1922. The memory crossbar switch 1916 can be configured to transfer the output of each cluster 1914A-1914N to any partition cell 1920A-1920N, or to another cluster 1914A-1914N on which additional processing operations can be performed. Each cluster 1914A-1914N can communicate with the memory interface 1918 via the memory crossbar switch 1916 to read from or write to various external memory devices. The memory crossbar switch 1916 can be connected to the memory interface 1918 to communicate with the I / O unit 1904, or to a local instance of the parallel processor memory 1922, enabling processing units within different processing clusters 1914A-1914N to communicate with system memory or other memory local to the non-parallel processing unit 1902. The memory crossbar switch 1916 can use virtual channels to separate traffic flows between clusters 1914A-1914N and partition units 1920A-1920N.

[0209] Multiple instances of the parallel processing unit 1902 can be mounted on a single add-in card, or multiple add-in cards can be interconnected. Even if different instances of the parallel processing unit 1902 have different numbers of processing cores, different amounts of local parallel processor memory, and / or other configuration differences, these different instances can be configured to interoperate. For example, some instances of the parallel processing unit 1902 may include higher-precision floating-point units relative to other instances. Systems including one or more instances of the parallel processing unit 1902 or the parallel processor 1900 can be implemented in a variety of configurations and form factors, including but not limited to desktop, laptop, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems.

[0210] Figure 19A It also includes a block diagram of a partitioning unit 1920 according to at least one embodiment. The partitioning unit 1920 is... Figure 19AAn example of one of the partition units 1920A-1920N in the parallel processor memory. Partition unit 1920 may include L2 cache 1921, frame buffer interface 1925, and ROP 1926 (raster operation unit). L2 cache 1921 may be a read / write cache configured to perform load and store operations received from memory crossbar switch 1916 and ROP 1926. Read misses and urgent write-back requests may be output from L2 cache 1921 to frame buffer interface 1925 for processing. Updates may also be sent to the frame buffer via frame buffer interface 1925 for processing. Frame buffer interface 1925 may interface with one of the memory cells in the parallel processor memory, such as, but not limited to, Figure 19A The memory cells 1924A-1924N are located in the parallel processor memory 1922 (for example).

[0211] ROP 1926 can be a processing unit that performs raster operations, such as, but not limited to, stenciling, z-testing, blending, etc. ROP 1926 can then output processed graphics data stored in graphics memory. ROP 1926 may include compression logic for compressing depth or color data written to memory and decompressing depth or color data read from memory. The compression logic can be lossless compression logic that utilizes one or more compression algorithms. The type of compression performed by ROP 1926 can vary based on the statistical characteristics of the data to be compressed. For example, incremental color compression is performed on depth and color data on a per-tile basis.

[0212] ROP 1926 can be included in each processing cluster (e.g., Figure 19A Within clusters 1914A-1914N, not within partition unit 1920. Read and write requests for pixel data (not pixel fragment data) can be transferred via memory crossbar switch 1916. Processed graphics data can be displayed on the monitor and routed for further processing by the processor, or routed to... Figure 19A One of the processing entities within the parallel processor 1900 in the system is further processed.

[0213] In at least one embodiment, the parallel processor 1900 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0214] Figure 19B A block diagram including a processing cluster 1914 within a parallel processing unit according to at least one embodiment. The processing cluster may be... Figure 19AAn instance of one of the processing clusters 1914A-1914N is provided, which can be used to perform any of the operations described above or elsewhere herein. Processing cluster 1914 can be configured to execute many threads in parallel, where a “thread” refers to an instance of a specific program executed on a specific input dataset. Single Instruction Multiple Data (SIMD) instruction issuing techniques can be used to support the parallel execution of a large number of threads without providing multiple independent instruction units. Single Instruction Multiple Thread (SIMT) techniques can be used to support the parallel execution of a large number of typically synchronous threads using a common instruction unit configured to issue instructions to a set of processing engines within each processing cluster.

[0215] The operation of cluster 1914 can be controlled via pipeline manager 1932, which distributes processing tasks to the SIMT parallel processors. Pipeline manager 1932 can... Figure 19A The scheduler 1910 receives instructions and manages the execution of these instructions via the graphics multiprocessor 1934 and / or texture unit 1936. The graphics multiprocessor 1934 may be an example instance of a SIMT parallel processor. However, the processing cluster 1914 may include various types of SIMT parallel processors with different architectures. The processing cluster 1914 may include one or more instances of the graphics multiprocessor 1934. The graphics multiprocessor 1934 can process data and can distribute the processed data to one of several possible destinations, including other shader units, using the data cross switch 1940. The pipeline manager 1932 can facilitate the distribution of processed data by specifying the destination of the processed data to be distributed via the data cross switch 1940.

[0216] Each graphics multiprocessor 1934 within the processing cluster 1914 may include a set of identical functional execution logic (e.g., arithmetic logic units, load-memory units, etc.) for performing computations for any operations described above or elsewhere herein. The functional execution logic can be configured in a pipelined manner, where new instructions can be issued before previous instructions complete. The functional execution logic can support a wide range of operations, including integer and floating-point arithmetic, comparison operations, Boolean operations, bit shifting, and computation of various algebraic functions. Different operations can be performed using the same functional unit hardware, and arbitrary combinations of functional units are possible.

[0217] Instructions transmitted to the processing cluster 1914 can form threads, also known as thread bundles, subgroups, waves, or wavefronts. A group of threads executing across a set of parallel processing engines can be called a thread group. Thread groups can execute a common program on different input data. Each thread within a thread group can be assigned to a different processing engine within the graphics multiprocessor 1934. A thread group may contain fewer threads than the number of processing engines within the graphics multiprocessor 1934. When a thread group contains fewer threads than the number of processing engines, one or more processing engines may be idle during the processing cycle of that thread group. A thread group may also contain more threads than the number of processing engines within the graphics multiprocessor 1934. When a thread group contains more threads than the number of processing engines within the graphics multiprocessor 1934, processing can be performed in consecutive clock cycles. Multiple thread groups can execute concurrently on the graphics multiprocessor 1934.

[0218] The graphics multiprocessor 1934 includes an internal cache memory for performing load and store operations, such as, but not limited to, any of the operations described above or elsewhere herein. The graphics multiprocessor 1934 may forgo the internal cache and instead use a cache memory within the processing cluster 1914 (e.g., L1 cache 1948). Each graphics multiprocessor 1934 may also access partition units that can be shared across all processing clusters 1914 (e.g., ...). Figure 19A The graphics multiprocessor 1934 has an L2 cache within partition units 1920A-1920N and can be used to transfer data between threads. The graphics multiprocessor 1934 can also access off-chip global memory, which may include one or more of the local parallel processor memory and / or system memory. Any memory outside of the parallel processing unit 1902 can be used as global memory. The processing cluster 1914 may include multiple instances of the graphics multiprocessor 1934 and can share common instructions and data, which can be stored in the L1 cache 1948.

[0219] Each processing cluster 1914 may include an MMU 1945 (Memory Management Unit), which can be configured to map virtual addresses to physical addresses. One or more instances of the MMU 1945 may reside in... Figure 19AThe memory interface 1918 is located within the MMU 1945. The MMU 1945 may include: a set of page table entries (PTEs) for mapping virtual addresses to physical addresses on tiles; and optional cache line indexes. The MMU 1945 may include address translation backstop (TLB) buffers or caches, which may reside within the graphics multiprocessor 1934 or L1 1948 cache or processing cluster 1914. Physical addresses can be processed to distribute surface data access locally, allowing for efficient request interleaving between partition units. Cache line indexes can be used to determine whether a request for a cache line is a hit or a miss.

[0220] Processing cluster 1914 can be configured such that each graphics multiprocessor 1934 is coupled to a texture unit 1936 for performing texture mapping operations, such as determining texture sample locations, reading texture data, and filtering texture data. Texture data can be read from an internal texture L1 cache (not shown) or an L1 cache within the graphics multiprocessor 1934, and can be retrieved as needed from an L2 cache, local parallel processor memory, or system memory. Each graphics multiprocessor 1934 can output a processed task to a data crossbar switch 1940 to provide the processed task to another processing cluster 1914 for further processing, or store the processed task in an L2 cache, local parallel processor memory, or system memory via a memory crossbar switch 1916. Pre-ROP 1942 (pre-raster operation unit) can be configured to receive data from the graphics multiprocessor 1934 and direct the data to ROP units, which can be associated with partitioning units (e.g., ...) described herein. Figure 19A The PreROP 1942 unit is located together with partition units 1920A-1920N. The PreROP 1942 unit can perform color blending optimization, organize pixel color data, and perform address translation.

[0221] In at least one embodiment, the processing cluster 1914 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0222] Figure 19CA graphics multiprocessor 1934 according to at least one embodiment is illustrated, for example, to perform any of the operations described above or elsewhere herein. The graphics multiprocessor 1934 may be coupled to a pipeline manager 1932 of a processing cluster 1914. The graphics multiprocessor 1934 may include an execution pipeline including, but not limited to, an instruction cache 1952 (e.g., which may store instructions, such as, but not limited to, compiled API instructions), an instruction unit 1954, an address mapping unit 1956, a register file 1958, one or more general-purpose graphics processing unit (GPGPU) cores 1962, and one or more load / store units 1966, one or more of which may perform load / store operations to load / store instructions corresponding to the execution operations. The GPGPU cores 1962 and load / store units 1966 may be coupled to a cache memory 1972 and a shared memory 1970 via a memory and cache interconnect 1968. The GPGPU cores 1962 may be part of a SoC, such as, but not limited to, [other components]. Figure 18 It is part of the integrated circuit 1800.

[0223] Instruction cache 1952 can receive a stream of instructions to be executed (e.g., perform any operation described above or elsewhere herein) from pipeline manager 1932. Instructions can be cached in instruction cache 1952 and dispatched for execution by instruction unit 1954. Instruction unit 1954 can dispatch instructions into thread groups (e.g., thread bundles, subgroups, wavefronts, or waves), with each thread in the thread group assigned to a different execution unit within GPGPU core 1962. Instructions can access any of the local, shared, or global address spaces by specifying an address within a unified address space. Address mapping unit 1956 can be used to translate addresses in the unified address space into different memory addresses accessible by load / store unit 1966.

[0224] Register file 1958 provides a set of registers for the functional units of graphics multiprocessor 1934. Register file 1958 provides temporary storage for operands on data paths connected to functional units of graphics multiprocessor 1934 (e.g., GPGPU core 1962, load / store unit 1966). Register file 1958 can be partitioned among functional units such that each functional unit is allocated a dedicated portion of register file 1958. Register file 1958 can be partitioned among different thread bundles (which may be referred to as wavefronts, subgroups, and / or waves or threads) executed by graphics multiprocessor 1934.

[0225] Each GPGPU core 1962 may include a floating-point unit (FPU) and / or an integer arithmetic logic unit (ALU) for executing instructions of the graphics multiprocessor 1934. The architecture of the GPGPU core 1962 may be similar or different. A first part of the GPGPU core 1962 may include a single-precision FPU and an integer ALU, while a second part of the GPGPU core may include a double-precision FPU. The FPU may implement IEEE 754-2008 standard floating-point arithmetic or enable variable-precision floating-point arithmetic. The graphics multiprocessor 1934 may also include one or more fixed-function or special-function units for performing specific functions, such as, but not limited to, copying rectangles or pixel blending operations. One or more of the GPGPU cores 1962 may also include fixed-function or special-function logic.

[0226] The GPGPU Core 1962 may include SIMD logic capable of executing a single instruction on multiple sets of data. The GPGPU Core 1962 can physically execute SIMD4, SIMD8, and SIMD16 instructions, and logically execute SIMD1, SIMD2, and SIMD32 instructions. The SIMD instructions for the GPGPU Core can be generated by the shader compiler at compile time, or automatically generated when executing programs written and compiled for Single Program Multiple Data (SPMD) or SIMT architectures. Multiple threads of a program can be configured for a SIMT execution model that can be executed via a single SIMD instruction. For example, eight SIMT threads performing the same or similar operations can be executed in parallel via a single SIMD8 logic unit.

[0227] The memory and cache interconnect 1968 may include an interconnect network that connects each functional unit of the graphics multiprocessor 1934 to the register file 1958 and shared memory 1970. The memory and cache interconnect 1968 may be a cross-switch interconnect that allows the load / store unit 1966 to perform load and store operations between shared memory 1970 and the register file 1958. The register file 1958 may operate at the same frequency as the GPGPU core 1962, thus data transfer between the GPGPU core 1962 and the register file 1958 can have very low latency. The shared memory 1970 can be used to implement communication between threads executing on functional units within the graphics multiprocessor 1934. The cache memory 1972 can be used as a data cache, for example, for caching texture data transferred between functional units and texture units 1936. The shared memory 1970 can also be used as a program-managed cache. In addition to automatically caching data stored in cache memory 1972, threads executing on GPGPU core 1962 can also programmatically store data in shared memory.

[0228] The parallel processor or GPGPU described herein can be communicatively coupled to a host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general-purpose GPU (GPGPU) functions. The GPU can be communicatively coupled to the host processor / core via a bus or other interconnect (e.g., high-speed interconnects, such as, but not limited to, PCIe or NVLink). A System-on-a-Chip (SoC) may include the parallel processor or GPGPU described herein, which executes on the SoC. The GPU may be integrated as a core on a package or chip and communicatively coupled to the core via an internal processor bus / interconnect within the package or chip. Regardless of the GPU's connection method, the processor core can assign work to the GPU in the form of a sequence of commands / instructions contained in a job descriptor. The GPU can then use dedicated circuitry / logic to efficiently process these commands / instructions to perform any of the operations described above or elsewhere herein.

[0229] In at least one embodiment, the graphics multiprocessor 1934 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0230] Figure 20A processor 2000 according to at least one embodiment is illustrated. The processor 2000 may include a hybrid architecture processor (e.g., Lunar Lake or Meteor Lake) from Intel Corporation, Santa Clara, California, or other processors sharing at least some of the components described herein. The processor 2000 may include one or more central processing units (CPU 2002), one or more graphics processing units (GPU 2006), and / or one or more neural processing units (NPU 2008), which may be, for example, dedicated AI accelerators for offloading artificial intelligence (AI) workloads from the CPU 2002 and GPU 2006. The processor 2000 may use instructions that, if executed, cause the processor 2000 and / or any of its components to perform some or all of the processes and techniques described elsewhere herein. The processor 2000 may include any number of memory and cache units 2010 for facilitating processing between different components of the processor 2000. The memory and cache 2010 on processor 2000 may include one or more levels of cache (e.g., L1, L2, L3, and / or last-level cache) and high-bandwidth memory (e.g., HBM2e or HBM3) in any combination. Regarding processor 2000 and any components described above or elsewhere herein, one or more APIs described herein may, for example, be compiled into instructions that may be fetched by instruction fetching logic or equivalents, decoded by processor decoder or equivalents, scheduled (e.g., sequentially or out of order) for execution by scheduler or equivalents, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. APIs (and / or compiled instructions including APIs) may be stored in any storage device (e.g., cache and / or memory) internal or external to processor 2000. The results of APIs may be stored in storage devices internal or external to processor 2000, including registers, DRAM, flash memory, SRAM, cache, or other memory. One or more APIs described herein may include calls.

[0231] Processor 2000 may include the computing engine of CPU 2002 and may include any number of cores, such as, but not limited to, up to 16 cores / 22 threads. The cores in CPU 2002 may include P-cores (performance), E-cores (high efficiency), and LP-E cores (low-power, high-efficiency). Performance cores can be used for low-latency, single-threaded, computationally intensive workloads, while high-efficiency cores can be used for multi-threaded, less computationally intensive workloads. Low-power, high-efficiency cores can be used for scalable multi-threaded execution and offloading background tasks. P-cores can be used for single-threaded and limited-threaded execution, while E-cores and LP-E cores are used for multi-threaded throughput and power efficiency.

[0232] GPU 2006 can include any number of graphics engines, such as, but not limited to, those with 8 Xe cores (up to 128 execution units or EUs). Arc TM Graphics engine (Xe LPG). For example... Figure 20 As shown, GPU 2006 may include a vector engine 2010 and a matrix engine 2012, which, for example, can run FP, INT, and matrix operation tasks simultaneously, individually, or in batches. GPU 2006 may include a load / store unit 2014, as well as other memories, such as, but not limited to, an instruction cache (I$) 2016 and an L1 cache / subsystem local memory (SLM) 2018, which may, for example, store instructions for performing any of the operations described above or elsewhere herein.

[0233] NPU 2004 may include one or more AI Boost integrates a Neural Processing Unit (NPU). The NPU 2004 can be enumerated as an integrated PCIe device to the host processor. The NPU 2004 can include one or more (e.g., two) Neural Computation Engine (NCE) tiles 2030. Each tile can be configured with any combination of, but not limited to, the following: (e.g., 2000) Multiply-Accumulate (MAC) engines 2034, a post-processing engine (not shown), an AEP processor (not shown), and memory per tile (2MB dedicated SRAM), such as... Figure 20 As shown. For general computing needs, the neural computing engine 2030 may include an inference pipeline 2032, an activation function (AF) 2036, a data transformation 2038, a load / store 2040, and a streaming hybrid architecture vector engine (SHAVE) 2028 for high-performance parallel computing, which may include a DMA (Direct Memory Access) engine 2024 for transporting data between system memory DRAM (Dynamic Random Access Memory) 2026 and a software-managed cache. The built-in device MMU (Memory Management Unit) 2022, plus the IOMMU (Input-Output Memory Management Unit) (not shown), can support multiple concurrent hardware contexts and provide secure isolation between execution contexts according to the MCDM (Microsoft Compute Driver Model) architecture. The processor 2000 may also include a media unit (not shown), which may be included on or separate from the XCD or other components of the processor 2000 to enable video playback and video processing of compressed or uncompressed data, such as using HEVC, AV1, VP9, ​​and AVC hardware-accelerated decoding support and HEVC, VP9, ​​and AVC hardware-accelerated encoding support.

[0234] Thread bootstrap ( The Thread Director (which includes firmware built into the processor 2000) can prioritize and manage the distribution of workloads, thereby sending tasks to optimized cores. For example, the Thread Director can tie P cores, E cores, and / or LP-E cores (as described above) together with task scheduling capabilities and the ability to send less demanding tasks to E cores or LP-E cores. Deep learning acceleration ( DLBoost (not shown) can provide built-in AI acceleration for training and inference workloads and may include support for VNNI (for CPU) and DP4a (for GPU) instruction sets. This instruction set can be used with OpenVINO. TM The toolkit and oneAPI are optimized to accelerate INT8 inference. For example, the software stack described elsewhere in this document can be used to leverage OpenVINO. TM The toolkit enables AI inference. The processor 2000 can be configured to execute applications, such as, but not limited to, CUDA programs.

[0235] In at least one embodiment, the processor 2000 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0236] Processor 2000 may alternatively include a processor based on Qualcomm's AIEngine Direct architecture from Santa Clara, California, or other processors sharing at least some of the components described herein. It may include any number of NPUs, GPUs, CPUs, and other associated components, such as, but not limited to, an NPU 2004 as a Hexagon NPU, a GPU 2006 as an Adreno GPU, a CPU 2002 as a Kryo or Qualcomm Oryon CPU, and a Qualcomm Sensing Hub (not shown) and a memory subsystem 2010 in any combination. Hexagon NPU 2004 may include power rails, micro-tile inference units, hardware acceleration units, tensor units, scalar units, and vector units (all not shown), which may have dedicated or shared memory (e.g., cache or memory, such as HBM3) for storing, for example, instructions for performing any of the operations described above or elsewhere herein. The Adreno GPU 2006 can provide graphics and parallel processing for AI, in formats including but not limited to 32-bit floating-point (FP32), 16-bit floating-point (FP16), and 8-bit integer (INT8). The Kryo or Qualcomm Oryon CPU 2002 can execute AI workloads and handle the contextualization of ubiquitous generative AI applications. The CPU 2002 may also include an instruction fetch unit, a renaming and deprecation unit, a memory management unit, a vector execution unit, an integer execution unit, and a load and store unit for processing and instruction management. Regarding the processor 2000 and any of its components described above or elsewhere herein, one or more APIs described herein may, for example, be compiled into instructions that may be fetched by the instruction fetch unit, decoded by the processor decoder or equivalent, scheduled (e.g., sequentially or out of order) for execution by the scheduler or equivalent, executed by the execution logic or equivalent, reordered, and then deprecated by the renaming and deprecation unit. The API (and / or compiled instructions including the API) can be stored in any storage device (e.g., cache and / or memory) inside or outside the processor 2000. An arbitrary number of CPU cores 2002 can be included in an arbitrary number of CPU clusters, which can be coupled to memory and / or cache, such as, but not limited to, a shared L2 cache. Memory can be separate or shared; for example, the CPU clusters of CPU cores 2002 can be coupled to a memory subsystem 2010, which can include structures capable of reading and writing to memory (e.g., DRAM), system-level caches, and an arbitrary number of memory management units.The Qualcomm sensing hub (not shown) includes a miniature NPU, power rails, and conventional sensors (such as gyroscopes, accelerometers, or even barometers) that support voice and data streaming. The memory subsystem 2010 may include memory and cache on the processor 2000, which may include L1 or more levels of cache (e.g., L1, L2, L3, and / or last-level cache) and high-bandwidth memory (e.g., HBM2e or HBM3) in any combination, for example, for storing information and / or instructions for performing any of the operations described above or elsewhere herein. All or part of the memory and / or cache in the memory subsystem 2010 may be shared or used individually by any component or combination of components on the processor 2000 (e.g., GPU 2006, NPU 2004, and CPU 2002).

[0237] The Qualcomm AI Engine 2000 can be programmed and controlled using a software stack to perform some or all of the operations described herein, including, for example... The Neural Processing SDK is available for inference on Android, Linux, and Windows. Developer libraries and services support programming languages, virtual platforms, and compilers. At lower levels of the software stack, system software includes a basic real-time operating system (RTOS), system interfaces, and drivers. The software stack supports various operating systems, including Android, Windows, Linux, and QNX, as well as deployment and monitoring infrastructures such as Prometheus, Kubernetes, and Docker. OpenCL and DirectML are supported for direct cross-platform access to GPU 2006. For CPU 2002, LLVM compiler infrastructure optimizations enable accelerated and efficient AI inference. Regarding the Qualcomm AI Engine 2000 and any of its components described above or elsewhere herein, one or more APIs described herein can, for example, be compiled into instructions that can be fetched by instruction fetching logic or equivalents, decoded by processor decoders or equivalents, scheduled (e.g., sequentially or out of order) for execution by a scheduler or equivalent, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. APIs (and / or compiled instructions including APIs) can be stored in any storage device (e.g., cache and / or memory) inside or outside the Qualcomm AI Engine 2000. The results of the APIs can be stored in storage devices inside or outside the Qualcomm AI Engine 2000, including registers, DRAM, flash memory, SRAM, cache, or other memory.

[0238] In at least one embodiment, the processor 2000 or the Qualcomm AI engine 2000 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0239] Figure 21AA processor 2100 according to at least one embodiment is illustrated. The processor 2100 may include a Scalable family of processors from Intel Corporation, Santa Clara, California, or other processors that share at least some of the components described herein. The processor 2100 may include one or more cores 2112(1)-2112(N) capable of performing the operations described elsewhere herein, where N is any integer greater than 1. Cores 2112(1)-2112(N) may be interconnected using ring and / or mesh interconnects. Utilizing a mesh interconnect architecture, arrays of vertical and horizontal communication paths may allow traversal from one core to another 2112(1)-2112(N) via the shortest path (jumping to the correct row along the vertical path and to the correct column along the horizontal path). For the mesh interconnect, a die may accommodate cores 2112(1)-2112(N) and may include a Converged Mesh Stop Point (CMS) grid that may be associated with cores 2112(1)-2112(N) (e.g., 1:1). Each core can be associated with a low-level cache (LLC) slice 2114(1)-2114(N), or cores 2112(1)-2112(N) can share a cache, such as a low-level cache. LLC 2114(1)-2114(N) can be inclusive or non-inclusive (having blocks not present in the high-level cache) by merging blocks from a higher-level cache (e.g., L2 cache). Each core and LLC slice can include a caching and home agent (CHA) (not shown), which can maintain cache consistency by providing resource scalability via mesh interconnect. Super Path Interconnect ( UPI 2116 provides cache coherency capabilities. UPI 2116 enables coherent interconnects for scalable systems and allows multiple processors to share a single shared address space via links, such as, but not limited to, two or three UPI links per processor.

[0240] Processor 2100 may also include system agent 2110, which may house and / or perform various functions, such as, but not limited to, memory management, display functions, and / or input / output (I / O) functions. For example, processor 2100 may include one or more integrated memory controllers (IMCs) 2108. IMCs 2108 may control and manage memory, such as, but not limited to, different memory types, such as DDR RAM, such as DDR4, or other memory described elsewhere herein. System agent 2110 may include a display controller (not shown) for supporting one or more displays. System agent 2110 may also integrate PCIe 2104 (e.g., up to 20 PCIe lanes), which may, for example, be connected to an external dedicated graphics connector via a DMI bus (e.g., Intel's DMI 3.0 bus) 2106. System agent 2110 may include an image processing unit (IPU) (not shown) that integrates an on-die image signal processor (ISP). Structure 2102 provides scalability for connecting to other nodes (e.g., processors, such as processor 2100) and can, for example, be connected to Cornelis Networks (…). It can be used together with elements of a scalable system framework that provides performance for high-performance computing (HPC) workloads and the ability to scale to tens of thousands of nodes.

[0241] Figure 21B Components within a core 2112 according to at least one embodiment are illustrated. The core 2112 may include a front-end 2118, a back-end or execution engine 2132, and a memory subsystem 2142. The front-end 2118 may provide operations (e.g., operations described elsewhere herein) to the execution engine 2132 by decoding instructions stored in memory. For example, the front-end 2118 may include micro-operation (μOps) cache paths and / or traditional paths, and a branch prediction unit 2120 that can determine path instructions. A traditional path for instructions may include fetching variable-length (e.g., x86) instructions from the L1 instruction cache 2120 and instruction fetch and pre-decode 2122, queuing these instructions into an instruction queue 2124, and decoding the instructions into μOps that can be provided to the allocation queue 2128 using a decoder 2126. Alternatively, the μOps cache path may include a cache that includes decoded μOps (μOps 2130) that can be sent to the allocation queue 2128. The allocation queue 2128 can act as an interface between the front end 2118 and the execution engine 2132, and can provide instructions to the execution engine 2132. For example, one or more APIs described herein can be compiled into instructions that can be stored, processed, and executed by the front end 2118 and the execution engine 2132, and stored in the memory subsystem 2142.

[0242] Execution engine 2132 can receive micro-operations into reordering buffer 2134, which can register, rename, and deregister μOPs. μOPs can be sent from the reordering buffer to scheduler 2136, which can be connected to one or more different execution units 2138, which can be connected to address generation units (AGUs) 2140. Execution units 2138 can perform operations such as basic arithmetic logic unit (ALU) operations, multiplication, division, and / or more complex operations, such as, but not limited to, various vector operations. Scheduler 2136 can manage the queuing of μOPs for one or more execution units 2138 based on, for example, the operations that need to be performed.

[0243] The memory subsystem 2142 can handle load and store requests as well as sorting operations. For example, μOPs may be associated with memory accesses (e.g., load and store), and these μOPs can be sent through dedicated scheduler ports that can perform these memory operations. For example, store and load operations can be sent to load and store buffers 2144. The memory subsystem 2142 may also include shared or separate L1 data and instruction caches 2146, and an L2 cache 2148 that can be used and shared by the L1 data and instruction caches 2146. (As described above for...) Figure 21A Each core 2112 can be connected to a slice of a third-level cache (e.g., LLC 2114), which can be shared by all cores 2112.

[0244] In at least one embodiment, processor 2100 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0245] Figure 22An AI accelerator 2200 according to at least one embodiment is illustrated. Processor 2200 may include a processor with an AI accelerator architecture manufactured by Intel Corporation, Santa Clara, California, or other processors sharing at least some of the components described herein. AI accelerator 2200 may use instructions that, if executed by AI accelerator 2200, cause AI accelerator 2200 to perform some or all of the processes and techniques described elsewhere herein. For example, with respect to AI accelerator 2200 and any components described above or elsewhere herein, one or more APIs described herein may, for example, be compiled into instructions that may be fetched by instruction fetching logic or equivalents, decoded by processor decoders or equivalents, scheduled (e.g., sequentially or out of order) by a scheduler or equivalent for execution, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. APIs (and / or compiled instructions including APIs) may be stored in any storage device internal or external to AI accelerator 2200 (e.g., in cache and / or memory). The results of the API can be stored in internal or external storage devices of the AI ​​accelerator 2200, including registers, DRAM, flash memory, SRAM, cache, or other memory. The AI ​​accelerator 2200 may include one or more compute dies, which may include homogeneous or heterogeneous processors. The compute dies may include one or more central processing units (CPUs), one or more graphics processing units (GPUs), or a combination of both.

[0246] In at least one embodiment, the computational die may include a computational engine for performing AI computations. In at least one embodiment, the computational die of the AI ​​accelerator 2200 may be split into any number (e.g., four) clusters, which may be referred to as DCORE (Deep Learning Core) 2206, and include any number of matrix multiplication engines (MME) 2208, tensor processor cores (TPC) 2210, memory management units 2212, and L2 caches 2214 in any combination. The MME 2208 may perform operations using matrix multiplication, such as fully connected layers, convolutions, and batch general matrix multiplication (GEMM). The MME 2208 may be equipped with a multiplication-accumulation unit (MAC) (not shown), which may perform general matrix multiplication (GEMM) operations, such as, but not limited to, AxB multiplication, which involves generating a tensor C [NxM] from two input tensors A [NxK] and B [KxN]. The MME 2208 may be programmed with array dimensions, positions, data types, and various operands. The MME 2208 can retrieve tensors A and B from memory and pull them into its streaming buffer for parallel matrix multiplication by the MAC. After completion, the MME 2208 can push tensor C back to memory. The TPC 2210 may include any number of scalar units for performing scalar operations, any number of vector units for performing vector operations, any number of register files or local memory units (e.g., vector local memory), and load and store components for instructions, which may be coupled to memory or caches (e.g., HBM, L3 cache, and / or L2 cache) (all not shown). The TPC can support different types of parallel processing, such as Very Long Instruction Word (VLIW) Single Instruction Multiple Data (SIMD) data types such as, but not limited to, FP32, BF16, FP16, and FP8 (both E4M3 and E5M2), UINT32, INT32, UINT16, INT16, UINT8, and INT8 data types. Any number of computational dies can be interconnected. The interconnection of connectable computing dies can be via an intermediary bridge, which is transparent to the software, for example.

[0247] The memory on the AI ​​accelerator 2200 may include one or more levels of cache (e.g., L1, L2, L3, and / or last-level cache) and high-bandwidth memory (e.g., HBM2e or HBM3) in any combination. The memory and / or cache system may be unified or separate. The compute die of the AI ​​accelerator 2200 may include on-chip memory comprising one or more levels (e.g., two levels) of cache. On-chip SRAM or other memory described elsewhere herein may be used as a unified last-level cache (L3) or split into multiple slices of L2 cache accessible to the MME 2208 and TPC 2210 groups. The use of on-chip memory as an L2 or L3 cache is entirely software-configurable, and the software can dynamically determine its optimal cache allocation based on I / O tensors. AI accelerator 2200 may include one or more memory management units (MMUs) 2222 for managing memory, such as allowing the AI ​​accelerator 2200 memory subsystem to run in virtual space when accessing VRAM.

[0248] AI accelerator 2200 may include a communication port (e.g., a PCIe Gen5 x16 port) 2202 for communicating with the host and scheduling and synchronization unit 2204. AI accelerator 2200 may include a media unit 2216, which may include any number or combination of media decoder engines (DECs) 2220 and rotation engines (ROTs) 2218. AI accelerator 2200 may include a network unit 2224, which may include any number or combination of network ports 2226 and an accompanying RDMA engine 2228, L2 cache, and memory (e.g., HBM2e or HBM3) stack. AI accelerator 2200 may include a programmable control path entity (not shown) for managing the parallel and efficient execution of the various engines. The control path may include a submission queue (SQ) that can be issued by the runtime system, a completion queue (CQ) that can be used for job completion reporting, a programmable scheduling mechanism that can be used for task scheduling, a programmable hardware synchronization mechanism or "synchronization manager (SM)" that can be used for hardware synchronization, and a programmable interrupt service mechanism or "interrupt manager (INTR)" that can pass asynchronous events to drivers.

[0249] AI Accelerator 2200 may include media decoding units supporting video formats such as, but not limited to, HEVC, Progressive H.264, SVC base layer, MVC, VP9, ​​JPEG, and Progressive JPEG. AI Accelerator 2200 may support post-processing of the decoded media stream, such as, but not limited to, image downsizing (image resizing), vertical and horizontal scaling at different scaling ratios, image enlargement, image cropping, bilinear scaling, and Lancos scaling. AI Accelerator 2200 may implement two post-processing channels per decoder unit, one for scalar (up and down) and the other solely for outputting the original image. AI Accelerator 2200 may include a hardware rotation engine that performs the following transformations on the input image: 2D rotation, 3D rotation, projection, image warping and de-warping, resampling of the input data at user-defined coordinates, and rescaling.

[0250] The RDMA 2228 based on converged Ethernet on the AI ​​accelerator 2200 enables scaling from a single node (i.e., from a single AI accelerator 2200 to hundreds or thousands of nodes or AI accelerators 2200). The network subsystem 2224 may include... The accelerator accelerator includes an In-Gigabit Ethernet Communication Library (IGCL), a master controller coordinating data movement, and a programmable scheduling mechanism that enables smooth engine activation while maintaining task dependencies. The accelerator network subsystem may include a Gigabit Ethernet NIC port 2226, a Layer 2 MAC (not shown), and an RDMA engine 2228. The AI ​​accelerator 2200 may include an aggregation engine for performing summation activities. All engines in the processor 2200 can run in parallel; for example, the MME 2208, TPC 2210, and NIC 2226 can all operate simultaneously. Dependencies may exist between operations running on different engines; for example, the output of one engine may be used as the input of another engine, and / or the MME, TPC, and NIC may be scheduled to run in parallel. When one engine completes its execution, another engine can be scheduled to begin working on the next operation (executed immediately after its input is ready).

[0251] The AI ​​accelerator 2200 can be operated and controlled using a software layer 2228, which may include low-level components such as, but not limited to, a graph compiler, an automatic kernel fusionist and pre-compiled kernel libraries, and integrations with the AI ​​ecosystem such as, but not limited to, PyTorch, DeepSpeed, Hugging Face, vLLM, Ray, etc., or as described elsewhere in this document regarding software and programming platforms. The software layer 2228 may include implementations of algorithms such as, but not limited to, paged attention, flash attention, etc. The software layer 2228 can generate optimized binary code that implements a given model topology, such as, but not limited to, performing operator fusion, data layout management, parallelization, pipeline and memory management, and graph-level optimization.

[0252] In at least one embodiment, the AI ​​accelerator 2200 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0253] This paper describes a neuromorphic computing system employing a multi-core architecture, where each core houses computing elements including neurons, synapses with on-chip learning capabilities, and local memory for storing synaptic weights and routing tables. Figure 23This is a simplified block diagram 2300 illustrating at least a portion of an example of such a neuromorphic computing device 2305 according to at least one embodiment. The neuromorphic computing device 2305 may include a neuromorphic processor from Intel Corporation, Santa Clara, California, or other processors that include at least a portion of the components described herein. As shown in this example, the device 2305 may be equipped with a network 2310 consisting of multiple neural network cores interconnected by a network on the device, thereby potentially defining multiple distinct connections between the cores. For example, the device 2305 may provide a network 2310 of spiking neural network cores, each core capable of communicating via short packet spiking messages sent from one core to another through network channels. Each core (e.g., 2315) may have processing and memory resources, as well as logic, for implementing a number of primitive nonlinear time computation elements, such as, but not limited to, multiple (e.g., more than 1000) distinct artificial neurons (referred to herein as “neurons”). For example, each core may be able to implement multiple neurons concurrently, allowing the neuromorphic core to implement many, many neurons using the device 2305. With respect to the neuromorphic computing device 2305 and any components described above or elsewhere herein, one or more APIs or equivalents described herein may, for example, be compiled into instructions or equivalents that may be fetched by instruction fetching logic or equivalents, decoded by processor decoder or equivalents, scheduled (e.g., sequentially or out of order) for execution by scheduler or equivalents, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. The APIs (and / or compiled instructions including the APIs) may be stored in any storage device (e.g., cache and / or memory) internal or external to the neuromorphic computing device 2305. The results of the APIs may be stored in storage devices internal or external to the neuromorphic computing device 2305, including registers, DRAM, flash memory, SRAM, cache, or other memory equivalents.

[0254] continue Figure 23For example, neuromorphic computing device 2305 may also include processor 2320 and system memory 2325 for implementing one or more components to manage and provide the functionality of neuromorphic computing device 2305. For instance, a system manager 2330 may be provided to manage the global attributes and operations of neuromorphic computing device 2305 (e.g., attributes affecting core network 2310, multiple cores in network 2310, interconnection of neuromorphic computing device 2305 with other devices, managing access to global system memory 2325, and other potential examples). In one example, system manager 2330 may manage the definition and configuration of specific routing tables for individual routers in network 2310, orchestration of network definitions and attributes to be applied to network 2310 (e.g., weights, attenuation rates, etc.), core synchronization and time multiplexing management, routing input to appropriate cores, and other potential functions.

[0255] As another example, the neuromorphic computing device 2305 may also include a programming interface 2335 through which a user or system can specify the neural network definition to be applied (e.g., via routing tables and individual neuron attributes), implemented by the neuromorphic core grid 2310. A software-based programming tool may be provided with or separately from the neuromorphic computing device 2305, through which a user can define a specific neural network to be implemented using the neuromorphic core network 2310. The programming interface 2335 can receive input from a programmer, then generate the corresponding routing table and populate the specified parameters into the local memory of each neuromorphic core (e.g., 2315) to implement the corresponding custom artificial neural network implemented by the neuromorphic core 2315.

[0256] In certain circumstances, the neuromorphic computing device 2305 can advantageously engage and interoperate with other devices, including general-purpose computing devices, to enable specific applications and use cases. Therefore, in some cases, external interface logic 2340 may be provided to communicate with one or more other devices (e.g., via one or more defined communication protocols). External interface 2340 may be used to accept input data from another device or an external memory controller used as an input data source. External interface 2340 may additionally or alternatively be used to allow the results or outputs of computations performed using the neural network implemented using the neuromorphic computing device 2305 to be provided to another device (e.g., another general-purpose processor implementing machine learning algorithms) to enable additional applications and enhancements, among other examples.

[0257] like Figure 23The diagram illustrates a network 2310 interconnected by a network of multiple neural network cores on a device, showing a portion of a network structure interconnecting multiple neuromorphic cores (e.g., 2315a-d). For example, several neuromorphic cores (e.g., 2315a-d) can be provided in a mesh, each core interconnected via a network including multiple routers (e.g., 2350). In one implementation, each neuromorphic core (e.g., 2315a-d) can be connected to a single router (e.g., 2350) in the router array, and the router can be connected to at least one other router (e.g., [other router]). Figure 23 (As shown at 2310 in [reference]). As an example, in one particular implementation, four neuromorphic cores (e.g., 2315a-d) can be connected to a single router (e.g., 2350), and each router 2350 can be connected to two or more other routers to form a multi-core mesh, thereby allowing each neuromorphic core to interconnect with every other neuromorphic core in the neuromorphic computing device 2305. Furthermore, since each neuromorphic core can be configured to implement multiple different neurons, the router network of the neuromorphic computing device 2305 can similarly implement connections or artificial synapses (or simply "synapses") defined between any two of the potential many (e.g., 30,000+) neurons defined using the neuromorphic core network 2310 provided in the neuromorphic computing device 2305.

[0258] Figure 23A block diagram of the internal components of an example implementation of the neuromorphic core 2315 is shown. In one example, a single neuromorphic core may implement a number of neurons (e.g., 1024), which share the architectural resources of the neuromorphic core 2315 in a time-division multiplexing manner. In one example, each neuromorphic core 2315 may include a processor block 2355 capable of executing arithmetic functions and routing related to the implementation of the digitally implemented artificial neurons, such as, but not limited to, those explained herein. Each neuromorphic core 2315 may also provide local memory in which routing tables of the neural network can be stored and accessed, accumulated potentials of each cell body of each neuron implemented using core 2315 can be tracked, parameters of each neuron implemented by core 2315 can be recorded, and other data and usage can be recorded. Components or architectural resources of the neuromorphic core 2315 may also include: an input interface 2365 for receiving input spike messages generated by other neurons on other neuromorphic cores; and an output interface 2370 for sending spike messages to other neuromorphic cores via a mesh network 2310. In some instances, the routing logic of the neuromorphic core 2315 can be implemented at least partially using the output interface 2370. Furthermore, in some cases, the core (e.g., 2315) can implement multiple neurons within an example SNN, and some of these neurons can be interconnected. In this case, spiking messages sent between neurons hosted on the core 2315 can forgo communication via the routing structure of the neuromorphic computing device 2305 and can be managed locally within the specific neuromorphic core 2315.

[0259] Each neuromorphic core may also include logic for implementing artificial dendrites 2380 and artificial cell bodies 2385 (hereinafter referred to as “dendrites” and “cell bodies”, respectively) for each neuron 2375. Dendrites 2380 may be hardware-implemented processes for receiving impulses from network 2310. Cell bodies 2385 may be hardware-implemented processes for receiving the current time-accumulated neurotransmitter mass of each dendrite and evolving the potential states of each dendrite and cell body to generate outgoing impulse messages at appropriate times. Dendrites 2380 may be defined for each connection receiving input from another source (e.g., another neuron). In one implementation, the dendritic process 2380 may receive and process the impulse message as it arrives serially from network 2310 in a time-division multiplexed manner. With the reception of impulses, neuronal activation (tracked using cell bodies 2385 (and local memory 2360)) may increase. When the activation of a neuron 2375 exceeds a threshold set for neuron 2375, neuron 2375 generates a spike message, which is propagated via output interface 2370 to a fixed set of fan-out neurons. The network distributes the spike message to all destination neurons, which in turn can update their activation in a transient, time-dependent manner in response. This can lead to some of the destination neurons also exceeding their respective thresholds and triggering further spike messages, just as in real biological neural networks.

[0260] As described above, the neuromorphic computing device 2305 can reliably implement spiking-based neural computing models. Such models are also referred to as spiking neural networks (SNNs). In addition to neuronal and synaptic states, SNNs incorporate temporal concepts. For example, in SNNs, communication occurs via event-driven action potentials or spiking, which convey no explicit information other than the spiking time and the implicit source and destination neuron pairs corresponding to the spiking transmission. The computation of the result of a dynamic nonlinear integral as a weighted spiking input occurs in each neuron. In some implementations, recurrent and dynamic feedback can be incorporated into the SNN computation model. Furthermore, various network connectivity models can be employed to model a wide range of real-world networks or relationships, including fully connected (all-to-all) networks, feedforward trees, completely random projections, "small-world" networks, and other examples. Isomorphic two-dimensional networks at the neuromorphic core (e.g., but not limited to...) Figure 23The network shown in the example can advantageously support all these network models. Since some or all of the cores of the neuromorphic computing device 2305 can be connected, some or all of the neurons defined in a core can also be fully connected via a certain number of router hops. The neuromorphic computing device 2305 may also include a fully configurable routing table for defining various neural networks by allowing neurons in each core to distribute their spurs to any number of cores in the grid 2310 to achieve a completely arbitrary connection graph.

[0261] In improved implementations of systems capable of supporting SNNs, for example, but not limited to... Figure 23 The example illustrates a very large-scale integrated circuit (VLSI) hardware device that can provide high-speed, reliable circuitry to implement SNNs (Spiritual Neural Networks) to model the information processing algorithms used by the brain, but in a more programmable way. For instance, while a biological brain can only perform a specific set of defined behaviors (a consequence of years of development), a neuromorphic processor device can provide the ability to rapidly reprogram all neural parameters. Therefore, a single neuromorphic processor can be used to implement a wider range of behaviors than a single slice of biological brain tissue. This distinction can be achieved by employing neuromorphic processors with neuromorphic designs that are radically different from those found in natural neural circuitry.

[0262] As an example, a neuromorphic processor can implement a spontaneous neural network (SNN) using time-multiplexed computation in both a spiking communication network and the neuronal mechanism of the neuromorphic computing device 2305. Therefore, the physical circuitry of the neuromorphic computing device 2305 can be shared by many neurons to achieve a higher neuron density. Through time multiplexing, the network can connect N cores with a total wiring length of O(N), while the length of discrete point-to-point wiring will be extended to O(N). 2 This significantly reduces wiring resources to accommodate planar and non-plastic VLSI routing techniques, among other examples. In the neuromorphic core, time multiplexing can be implemented through dense memory allocation, for example, using static random access memory (SRAM) with a shared bus, address decoding logic, and other multiplexed logic elements. The state of each neuron can be stored in the processor's memory, where data describing the state of each neuron includes the state of the collective synapse of each neuron, all currents and voltages on its membrane, and other example information (e.g., but not limited to configuration and other information).

[0263] Neuromorphic processors can be implemented in a “digital” manner, unlike other processors that employ more “analog” or “isomorphic” neuromorphic approaches. For example, a digital implementation can use digital adder and multiplier circuitry to integrate synaptic currents, in contrast to an analog isomorphic neuromorphic approach that accumulates charge on capacitors in a manner similar to how neurons accumulate synaptic charge on their lipid membranes. For instance, the accumulated synaptic charge for each neuron can be stored in the local memory of the corresponding core. Furthermore, at the architectural level of an example digital neuromorphic processor, reliable and deterministic operation can be achieved through time synchronization across the core network, ensuring that any two executions of the design, given the same initial conditions and configuration, will produce the same results. Asynchronicity can be reserved at the circuit level to allow individual cores to operate as quickly and freely as possible while maintaining determinism at the system level. Therefore, in neural computing, the concept of time as a time variable can be abstracted away from the “wall clock” time used by the hardware to perform computations. Thus, in some implementations, a time synchronization mechanism can be provided that globally synchronizes the neuromorphic cores at discrete time intervals. Synchronization mechanisms allow neural computation to be completed at the fastest speed allowed by the circuitry, and there is a difference between the runtime and the biological time for modeling neuromorphic systems.

[0264] In operation, the neuromorphic computing device 2305 can start in an idle state when all neuromorphic cores are inactive. As each core asynchronously loops through its neurons, it generates impulse messages, which are routed by the mesh interconnect to the appropriate destination core containing all destination neurons. The implementation of multiple neurons on a single neuromorphic core can be time-multiplexed, and time steps can be defined, where all impulses involving multiple neurons can be processed and considered using the shared resources of the respective cores. When each core completes its service to its neurons within the corresponding time step, in some implementations, the core can communicate with neighboring cores using synchronization messages (e.g., using a handshake) to refresh the mesh of all transmitted impulse messages, allowing the core to safely determine that all impulses have been serviced within a certain time step. At this point, all cores can be considered synchronized, allowing them to advance their time steps and return to the initial state to begin the next time step.

[0265] Given this context, as described above, an apparatus (e.g., 2305) can be provided to realize an interconnected neuromorphic core grid 2310, wherein core 2315 can realize multiple artificial neurons capable of interconnecting to realize an SNN. Each neuromorphic core (e.g., 2315) can provide two loosely coupled asynchronous processes: an input dendrite process (e.g., 2380) that receives impulses from network 2310 and applies them to the appropriate destination dendritic compartment at an appropriate future time; and an output cell body process (e.g., 2385) that receives the current-time accumulated neurotransmitter mass of each dendritic compartment and evolves the membrane potential state of each dendrite and cell body to generate an outgoing impulse message at an appropriate time (e.g., when the threshold potential of the cell body is reached). It should be noted that, from a biological perspective, the names of dendrites and cell bodies used herein are only approximate to the function of these features and should not be interpreted too literally.

[0266] In at least one embodiment, the neuromorphic computing device 2305 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0267] Figure 24 This is a block diagram of an embodiment of a multi-node network capable of enabling remote memory computing according to any embodiment. System 2400 may represent the node network described herein, for example, the node network may be used to perform some or all of the operations described herein. System 2400 may represent a data center. System 2400 may represent a server farm. System 2400 may represent a data cloud or processing cloud. System 2400 may represent a supercomputer. System 2400 may include tens, hundreds or thousands of nodes. The nodes of system 2400 may include processors, such as, but not limited to, a central processing unit (CPU), a graphics processing unit (GPU) or any combination of processors described herein, such as, but not limited to, processors described herein. Figure 18-30BOther processors in the system. For any processor in the system 2400 and any components described above or elsewhere herein, one or more APIs or equivalents described herein may, for example, be compiled into instructions or equivalents that may be fetched by instruction fetching logic or equivalents, decoded by processor decoder or equivalents, scheduled (e.g., sequentially or out of order) for execution by scheduler or equivalents, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. APIs (and / or compiled instructions including APIs) may be stored in any storage device (e.g., cache and / or memory) inside or outside the processor or node. The results of APIs may be stored in storage devices inside or outside the processor or node, including registers, DRAM, flash memory, SRAM, cache, or other memory equivalents. System 2400 may include more than nine thousand nodes, each node comprising two Intel Xeon Max processors, six Intel Max series GPUs, and a unified memory architecture, such as, but not limited to, the architecture used in Intel's Aurora supercomputer in Santa Clara, California, or other supercomputers that share at least some of the components described herein.

[0268] One or more clients 2402 send requests to system 2400 via network 2404. Network 2404 represents one or more local area networks, wide area networks, or a combination of both. Clients 2402 can be human or machine clients that generate requests for operations to be performed by system 2400. System 2400 executes the application or data computation task requested by client 2402.

[0269] System 2400 may include one or more racks, which represent structural and interconnect resources for housing and interconnecting multiple computing nodes. Rack 2410 may include multiple nodes 2430. Rack 2410 may carry multiple blade assemblies 2420(0)-2420(N-1), where N is an integer greater than or equal to 2. Carrying may refer to providing power, structural or mechanical support, and interconnection. Blades 2420(0)-2420(N-1) may refer to computing resources on a printed circuit board (PCB), where the PCB houses the hardware components of one or more nodes 2430. Blades 2420(0)-2420(N-1) may or may not include a chassis, enclosure, or other “box” besides those provided by rack 2410. Blades 2420(0)-2420(N-1) may include an enclosure with exposed connectors for connection to rack 2410. System 2400 may or may not include rack 2410, and each blade 2420 (e.g., 2420(0)) may include a chassis or housing that may be stacked or otherwise closely proximate with other blades and allow nodes 2430 to interconnect. System 2400 may include 10,624 compute blades, comprising 63,744 Intel Max series GPUs and 21,248 Intel Xeon Max CPUs on 166 racks.

[0270] System 2400 may include structure 2470, which represents one or more interconnectors of nodes 2430. Structure 2470 may include multiple switches 2472 or routers or other hardware for routing signals between nodes 2430. Furthermore, structure 2470 may couple system 2400 to network 2404 for access by client 2402. In addition to routing devices, structure 2470 may also be considered to include cables or ports or other hardware devices for coupling nodes 2430 together. Structure 2470 may have one or more associated protocols for managing signal routing through system 2400. One or more protocols are at least partially dependent on the hardware devices used in system 2400.

[0271] As shown in the figure, rack 2410 may include N blades (e.g., 2420(0)-2420(N-1)). In addition to rack 2410, system 2400 may also include rack 2450. As shown in the figure, rack 2450 may include M blades (e.g., 2460(0)-2460(M-1)). M is not necessarily the same as N; therefore, it is understood that various different hardware device components may be used and coupled together into system 2400 via structure 2470. Blades 2460(0)-2460(M-1) may be the same as or similar to blades 2420(0)-2420(N-1). Node 2430 may be any type of node described herein and is not necessarily of the same type. System 2400 is not limited to homogeneous or non-homogeneous.

[0272] The nodes in blade 2420(0) are shown in detail. However, other nodes in system 2400 may be the same or similar. At least some nodes 2430 may be compute nodes, having processor 2432 and memory 2440. A compute node is a node having processing resources (e.g., one or more processors) that executes an operating system and can receive and process one or more tasks. At least some nodes 2430 may include storage server nodes, which have servers as processing resources 2432 and memory 2440. A storage server is a node having more storage resources than a compute node, and instead of having processors for performing tasks, a storage server includes processing resources for managing access to storage nodes within the storage server.

[0273] Node 2430 may include interface controller 2434, which may represent logic for controlling node 2430's access to structure 2470. The logic may include hardware resources for interconnecting to physical interconnect hardware. The logic may include software or firmware logic for managing the interconnect. Interface controller 2434 may include a host structure interface, which may include a structure interface according to any embodiment described herein.

[0274] Node 2430 may include a memory subsystem 2440. Memory 2440 may include a memory computation resource (comp) 2442, which represents the ability of memory 2440 to perform one or more memory computations. System 2400 supports remote memory operations, such as, but not limited to, those described elsewhere herein. Therefore, node 2430 may request a remote node to perform a memory computation, wherein the data used for the computation remains local to the executing node and is not sent via structure 2470 or from memory to the structure interface. In response to the execution of the memory computation, the executing node may provide the result to the requesting node.

[0275] Processor 2432 may include one or more individual processors. Each individual processor may include a single processing unit, a multi-core processing unit, or a combination thereof. A processing unit may include a main processor, such as, but not limited to, a CPU (Central Processing Unit), a peripheral processor (such as, but not limited to, a GPU (Graphics Processing Unit)), or a combination thereof. Memory 2440 may be or include a memory device and a memory controller.

[0276] The term "memory device" can refer to different types of memory. Memory devices generally refer to volatile memory technology. Volatile memory is memory whose state (and the data stored within it) is uncertain if power is interrupted. Non-volatile memory is memory whose state is deterministic even if power is interrupted. Dynamically volatile memory can refresh the data stored in the device to maintain its state. An example of dynamically volatile memory includes DRAM (Dynamic Random Access Memory) or variations thereof, such as, but not limited to, Synchronous DRAM (SDRAM). The memory subsystem described in this article is compatible with a variety of memory technologies, such as, but not limited to, DDR3 (Double Data Rate version 3, originally released by JEDEC (Joint Electron Device Engineering Committee) on June 27, 2007, currently version 21), DDR4 (DDR version 4, initial specification released by JEDEC in September 2012), DDR4E (DDR version 4, extended version, currently under discussion by JEDEC), LPDDR3 (Low Power DDR version 3, JESD209-3B, released by JEDEC in August 2013), and LPDDR4 (Low Power Double Data Rate (LPDDR) version 4). JESD209-4 (originally released by JEDEC in August 2014), WIO2 (Wide I / O2), JESD229-2 (originally released by JEDEC in August 2014), HBM (High Bandwidth DRAM), JESD235 (originally released by JEDEC in October 2013), DDR5 (DDR version 5, currently under discussion by JEDEC), LPDDR5 (currently under discussion by JEDEC), HBM2 (HBM version 2, currently under discussion by JEDEC) or combinations of other memory technologies, as well as technologies derived from or extended based on such specifications.

[0277] In addition to or as an alternative to volatile memory, in one embodiment, a reference to a memory device may refer to a non-volatile memory device whose state is deterministic even when power is interrupted. In one embodiment, a non-volatile memory device is a block-addressable memory device, such as, but not limited to, NAND or NOR technology. Therefore, the memory device may also include future-generation non-volatile devices, such as, but not limited to, three-dimensional cross-point (3DXP) memory devices, other byte-addressable non-volatile memory devices, or memory devices using chalcogenide phase change materials (e.g., chalcogenide glasses). In one embodiment, the memory device may be or include multi-threshold NAND flash memory, NOR flash memory, single-level or multi-level phase change memory (PCM) or switched phase change memory (PCMS), resistive memory, nanowire memory, ferroelectric transistor random access memory (FeTRAM), magnetoresistive random access memory (MRAM) incorporating memristor technology, or spin-transfer torque (STT)-MRAM, or any combination of the foregoing, or other memories.

[0278] In at least one embodiment, system 2400 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0279] Figure 25An accelerated processing unit 2500 according to at least one embodiment is illustrated. The accelerated processing unit 2500 may include a processor based on the CDNA architecture of AMD Inc., Santa Clara, California, or other processors sharing at least some of the components described herein. The accelerated processing unit 2500 may include one or more accelerator complex dies (XCDs) 2504 for performing the operations described elsewhere in this document, such as, but not limited to, graphics processing and / or parallel processing and instruction-level parallel computing, including support for multiple precisions (INT8, FP8, BF16, FP16, TF32, FP32, and FP64) and sparse matrix data (i.e., sparsity). In some cases, the XCD may be referred to as a graphics computing die (GCD). The accelerated processing unit 2500 may include one or more complex computing dies (CCDs) 2506 for performing the operations described elsewhere in this document, such as, but not limited to, operations performed by a host processor. In some cases, the CCD may be referred to as a core complex or CCX, such as, but not limited to, the CCX used in AMD Ryzen processors. XCDs and CCDs can share any type of cache or memory (e.g., one or more memory cells 2502), or a cache or memory can be allocated to each XCD or CCD or group of XCDs. For example, AMD Infinity Fabric within the package connects XCDs and CCDs to a shared AMD Infinity Cache 2508, and in some embodiments, to high-bandwidth memory (e.g., HMB3). Accelerated processing unit 2500 may include an AMD MI300a processor comprising three CPU die (or CCD) and six accelerator die (XCD) on top of four input-output dies (IODs) that may be layered on a single silicon die (e.g., via AMD Infinity Fabric) and linked together to eight high-bandwidth DRAM stacks in a ring to form a superchip. For systems using only accelerators, the AMD MI300x processor replaces the CCD with two or more XCDs.

[0280] Accelerated processing unit 2500 may include one or more input / output (I / O) interfaces. For example, XCD 2504 and CCD 2506 may coexist on one or more input-output dies (IODs) 2510, which may include one or more I / O interfaces. IOD 2510 may include any number and type of I / O interfaces (e.g., PCI, PCI expansion (“PCI-X”), PCIe, Gigabit Ethernet (“GBE”), USB, etc.). Various types of peripheral devices may be coupled to I / O interface 2570. The I / O interfaces of IOD 2510 may also be used to connect one or more accelerated processing units 2500, for example, in a server architecture.

[0281] Accelerated processing unit 2500 may include one or more memory units 2502 for storing instructions and other information for performing the operations described in other parts of this document. Memory units 2502 may include any volatile memory, such as, but not limited to, the memory types described in other parts of this document, and may include, for example, high-bandwidth memory (e.g., HMB3) or high-bandwidth DRAM. The memory associated with accelerated processing unit 2500 (e.g., memory unit 2502) may include system memory, which can be used for, for example, commands, instructions, and constants, as well as input and output. Memory unit 2502 may also include device memory, which can be used for storage and, for example, for commands, instructions, and constants, as well as input and output, as a return buffer, and for private data. Memory unit 2502 may be linked to one or more IODs 2510. In at least one embodiment, L1 cache 2520 initiates a memory hierarchy including a shared L2 cache 2528 (e.g., within an XCD). AMD Infinity Cache TM It is the last-level cache (LLC) located on the active I / O die (IOD). The CCD 2506 and XCD2504 can have independent or shared memory. AMD Infinity Architecture and AMD Infinity Fabric TM The technology enables consistent, high-throughput unification of GPU and CPU chip technologies (such as XCD, CCD, and / or CCX) with memory (such as stacked HBM3 memory) in a single device and across multiple device platforms.

[0282] like Figure 25As shown, the XCD 2504 may include a set of shared global resources 2530, which may include a hardware scheduler 2532 and an asynchronous compute engine (ACE) 2524. The ACE 2524 sends tasks (e.g., compute shader workgroups) to compute units (CUs or cores) 2534. Each of the ACEs 2524 (e.g., four) may be associated with a CU 2534 (e.g., 40 CUs), and some CUs 2534 may be disabled for yield management. CUs 2534 may have dedicated caches or shared caches (e.g., L2 caches) 2528 for consolidating all memory traffic on a single die. CU2534 may include threaded and parallel processor cores, including instruction fetching and scheduling using a scheduler (S) 2512, a matrix core unit (MCU) 2516, and a shader core (SC) 2518 (e.g., execution units for scalar, vector, and matrix data types), and a load / store pipeline with an L1 cache 2520 and a local data share (LDS) 2514. The local data share may include, for example, sticky-note RAM with built-in arithmetic capabilities, allowing data sharing between threads in a workgroup. Instruction cache 2540 (e.g., for storing and providing instructions for performing operations described elsewhere in this document) and constant cache 2538 may be connected to one or more CUs and may be shared between two CUs. Matrix core 2516 can handle various data types, such as, but not limited to, INT8, FP8, FP16, BF16, and TF32 data types. Accelerated processing unit 2500 may include computation units 2534, which may be arranged in an array format, such as as a data parallel processor (DPP) array. The hyper-threaded dispatch processor 2542 can communicate with the compute unit 2534, and the command processor 2544 can read commands written by the host to memory-mapped registers in the system memory address space (not shown). When a command is completed, the command processor 2544 can send a hardware-generated interrupt to the host processor (e.g., a CCD). The memory controller 2536 can also directly access all device memories and system memory regions specified by the host. To satisfy read and write requests, the memory controller 2536 can perform the functions of a direct memory access (DMA) controller, including calculating the memory address offset based on the format of the requested data in memory. For example, one or more APIs described herein can be compiled into instructions that can be stored in the instruction cache 2540, then fetched by the instruction fetch logic in the processor 2540, decoded by the processor decoder or equivalent, scheduled (e.g., sequentially or out of order) by the scheduler or equivalent for execution, executed by the execution logic or equivalent, reordered, and then retired by the retirement logic or equivalent.The API (and / or compiled instructions including the API) can be stored in any storage device, either inside or outside the processor 2500 (e.g., in cache and / or memory). The results of the API can be stored in storage devices, either inside or outside the processor 2500, including registers, DRAM, flash memory, SRAM, cache, or other memory equivalents.

[0283] Applications may include programs running on the main processor (e.g., a CCD) and programs running on one or more XCDs (referred to as kernels). Programs can be controlled by host commands that set internal base addresses and other configuration registers, specify data fields on which the accelerator processing unit 2500 can run, invalidate and flush caches on the accelerator processing unit 2500, and cause the accelerator processing unit 2500 to begin executing a program. A kernel can be referred to as a program executed by the accelerator processing unit 2500. Kernels can execute independently on each work item or as a group of work items, referred to as a wavefront, which can execute kernels on all (e.g., 64) work items in a single pass. The computation unit 2534 may include: a scalar arithmetic logic unit (ALU) that can operate on a single value for each wavefront (shared by all work items); a vector ALU that can operate on a unique value for each work item; a local data share 2514 that allows work items within a workgroup to communicate and share data; a scalar memory (not shown) that can transfer data between the scalar general-purpose registers (SGPRs) and memory via cache; and a vector memory that can transfer data between the vector general-purpose registers (VGPRs) and memory, including sampling texture maps. Kernel control flow can be manipulated using scalar ALU instructions, which may include if / else statements, branches, and loops. Scalar ALU (SALU) and memory instructions can operate on the entire wavefront and operate on one or more SGPRs. Vector memory and ALU instructions can operate on all work items in the wavefront simultaneously.

[0284] In at least one embodiment, the acceleration processing unit 2500 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0285] Figure 26Processor 2600 is shown, including, but not limited to, a Zen architecture-based processor (e.g., Zen1, 2, 3, 4, 5, or other architectures) from AMD Inc. of Santa Clara, California, or other processors that share at least some of the components described herein. Processor 2600 includes one or more CPU dies 2602(1)-2602(N), where N is any integer greater than 1. CPU die 2602 may include any number of processor cores 2616 (e.g., for performing any operations described elsewhere herein) and any number of cache memories (e.g., for storing instructions and other information to perform any operations described elsewhere herein). For example, L2 cache unit 2618 may be coupled to processor core 2616, and processor core 2616 may share and / or be individually coupled to L2 cache unit 2618. Processor core 2616 may be coupled to L3 cache 2622 and / or a shared L3 cache, which may be the lowest level cache (LLC) 2622 used to access data and other information used by processor core 2616. One or more processor cores 2616 and one or more L2 cache units 2618 may be included in a core complex (CCX) 2620, which may include (e.g., 32MB) a shared cache (e.g., L3 cache 2622). Core complex 2620 may be manufactured onto a die (CCD or CPU die) 2602. For example, up to 12 core complexes 2620 may be configured in a processor along with 8 CPU dies 2602, thereby providing up to 96 processor cores 2616 for processor 2600. For example, a "Zen 4c" core complex 2620 may include up to 8 cores 2616 and a shared 16MB L3 cache 2622. Two core complexes of these core complexes 2620 can be combined onto a single CPU die 2602, resulting in 16 cores per die and a total of 32MB of L3 cache per die 2622. Up to eight CPU dies 2602 can be combined with I / O units 2604 to provide up to 128 processor cores 2616 for the CPU. Up to four "Zen 4c" dies mentioned above can be combined to provide up to 64 processor cores 2616 for the CPU.

[0286] Processor 2600 may include various configurations for input / output operations, which will be further described herein. I / O unit 2604 may include one or more memory controllers 2606 capable of managing the memory usage of processor 2600 (e.g., DDR5 memory). I / O unit 2604 may include one or more SATA disk controllers for managing storage device 2612, and one or more Compute Express Links (CXL) for providing CPU-to-device and CPU-to-memory connectivity, which can be flexibly assigned to specific functions during server design. TM 1.1+ Memory controller 2614. I / O unit 2604 may include PCIe controller 2608 for connecting peripherals and other components connected to processor 2600. I / O unit 2604 may also include USB port 2610 for connecting to other components separate from processor 2600. CPU die 2602 may support any number of connections to I / O unit 2604, for example, one or two connections. As shown, I / O unit 2604 may include components further described herein, and I / O unit 2604 may be an I / O die accommodating several different components. Memory controller 2606, PCIe controller 2608, USB port 2610, SATA controller 2612 and / or CXL controller 2614 may be individually integrated into any location within processor 2600, or integrated in any group or combination.

[0287] Processor 2600 may include an Infinity Fabric 2624 interconnect (which may be similar to or based on a PCIe architecture) that provides connectivity between the CPU (e.g., CPU dies 2602(1)-2602(N)), graphics processor 2626, inference engine 2632, and other components in a multi-chip architecture (e.g., security processor 2628 and I / O unit 2604). One or more AMD Infinity Fabrics TM Interconnect 2610 can be connected to CPU dies 2602(1)-2602(N) and used as a connection between CPUs. One or more Infinity Fabric connections 2610 can connect each CPU die 2602 to the I / O unit 2610.

[0288] In at least one embodiment, processor 2600 may include a central processing unit (CPU) and other related hardware and software described above and further herein. Processor 2600 may also include a graphics processor 2626. Graphics processor 2626 may be used for image generation and processing, as well as other computations and operations described further herein. Graphics processor 2626 may be based on AMD's RDNA 3 or 3.5 architecture, located in Santa Clara, California. Graphics processor 2626 may include a graphics computing die (GCD) and a memory cache die (MCD). The GCD may include any number of computing units (CUs) for graphics or other processing, such as operations performed by an arithmetic logic unit (ALU) described further herein. Graphics processor 2626 may include an L2 cache available for use by the computing units. The MCD (not shown) may include any number of memory cells and may include a cache (e.g., an L3 cache) and a memory interface for coupling to memory (e.g., memory 2642(1)-(N), where N is an integer). Components within the graphics processor 2626 can be connected using various methods, such as using Infinity Fabric 2624 interconnects, either internally or externally to the graphics processor 2626.

[0289] Inference engine 2632 can provide neural processing capabilities to processor 2600 for computational processes used in neural networks, deep learning, and other artificial intelligence-related operations, which will be further described herein. Processor 2600 may include: a security processor 2628 for managing the security of processor 2600; a display controller 2630 for controlling the display; a system management unit 2634 for managing and operating some or all components on processor 2600; a multimedia engine 2636 for audio and video operations; a fusion controller hub 2638 for managing USB, SATA, and PCIe connections to processor 2600; and a sensor fusion hub 2640 for managing sensors (e.g., accelerometers). Processor 2600 may also include memory 2642(1)-(N), where N is any integer. Memory may include different memory types, such as LPDDR5 and / or DDR5, or other memory described elsewhere herein.

[0290] To perform the operations further described herein, processor 2600 may include an execution pipeline including a front-end that may include a cache for storing instructions (e.g., an L1 cache) (not shown). A branch predictor may modify the instruction stream. Instructions may be decoded by a decoder, dispatched to a back-end for execution, and renamed. For example, the instruction fetch and decode pipeline may be dispatched to integer or floating-point execution operations, which may be scheduled by a scheduler and passed to vectors and / or general-purpose registers. Floating-point multipliers and / or addition operations may be processed, and an arithmetic logic unit (ALU) may also be used to perform computations, such as arithmetic and logical operations. The output of the computation unit may be coupled to a load / store queue that may be connected to a cache, such as an L1 cache and / or an L2 cache.

[0291] With respect to processor 2600 and any components described above or elsewhere herein, one or more APIs or equivalents described herein may, for example, be compiled into instructions or equivalents (e.g., AVX-512 instructions based on a SIMD model). These instructions or equivalents may be fetched by instruction fetching logic or equivalents, decoded by processor decoder or equivalents, scheduled (e.g., sequentially or out of order) for execution by scheduler or equivalents, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. APIs (and / or compiled instructions including APIs) may be stored in any storage device (e.g., cache and / or memory) internal or external to processor 2600. The results of the APIs may then be stored in storage devices internal or external to processor 2600, including registers, DRAM, flash memory, SRAM, cache, or other memory equivalents.

[0292] In at least one embodiment, the processor 2600 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0293] Figure 27 An example of a processing core 2700 is shown, which may implement an Arm architecture (e.g., v9.0-A) or other processors that share at least some of the components described herein. TM The V2 core 2700 can be implemented within a DynamIQ shared unit (DSU) cluster via a DSU-110 interconnect 2754 for connecting one or more cores, for example, for parallel processing. Neoverse TMThe V2 core can be implemented as a single core in a DSU cluster configured for direct interconnection, with or without L3 cache, listener filter, or listener control unit (SCU) logic (not shown). Neoverse TMThe V2 core may include a CPU bridge 2752 for connecting core 2700 to the DSU-110 interconnect. This bridge may also connect core 2700 to an external memory system and the remainder of the on-chip system. The L1 instruction memory system 2702 may fetch instructions from instruction cache 2704 and deliver instructions (e.g., one or more compileable APIs described herein) to instruction decoding unit 2710, for example, to perform some or all of the operations described above or elsewhere herein. The L1 instruction memory system 2702 may include L1 instruction cache 2704 (e.g., with 64-byte cache lines), L1 instruction translation back buffer (TLB) 2706 (e.g., natively supporting 4KB, 16KB, 64KB, and 2MB page sizes), and macro operation cache (MOP) 2708 (e.g., a 1536-entry, 4-way skewed associated L0 MOP cache), which may include decoded and optimized instructions for higher performance. Instruction decoding unit 2710 may decode AArch64 instructions into its internal format. Register renaming unit 2712 can perform register renaming to facilitate out-of-order execution and dispatch decoded instructions to various issue queues. Instruction issue unit 2714 can control when decoded instructions are dispatched to the execution pipeline, and it can include issue queues for storing instructions to be dispatched to the execution pipeline. Integer execution pipeline 2716 can be included in the execution pipeline and includes integer execution unit 2718 capable of performing arithmetic and logical data processing operations. Vector execution unit 2720 can be included in the execution pipeline and can execute advanced SIMD and floating-point arithmetic (FPU) 2722, execute Scalable Vector Extension (SVE) and Scalable Vector Extension 2 (SVE2) instructions 2724, and can selectively execute cryptographic instructions 2726. Advanced SIMD may include a media and signal processing architecture that primarily adds instructions for audio, video, 3D graphics, image, and speech processing. The floating-point architecture provides support for single-precision and double-precision floating-point operations. The L1 data memory system 2730 executes load and store instructions, as well as service memory coherence requests. The L1 data memory system 2730 may include an L1 data cache 2732 and a fully associative L1 data TLB 2734, natively supporting 4KB, 16KB, and 64KB page sizes and 2MB and 512MB block sizes. The memory management unit (MMU) 2728 provides fine-grained memory system control through a set of virtual-to-physical address mappings and memory attributes, which can be stored in a translation table and saved to the TLB 2734 after address translation. The L2 memory system 2736 may include an L2 cache 2738 and can be connected to the DSU-110 2754 via an asynchronous CPU bridge 2752. TMThe V2 core 2700 supports a range of debugging, testing, and tracing options, including the tracing unit 2742, the tracing buffer 2740, and the embedded logic analyzer (ELA) 2748. Neoverse TM The V2 core 2700 implements the Statistical Analysis Extension (SPE) 2744, which provides a statistical view of the performance characteristics of executed instructions. Software developers can leverage these views to optimize code for better performance. The Performance Monitoring Unit (PMU) 2746 provides a performance monitor that can be configured to collect statistics on the operation of each core and memory system. This information can be used for debugging and code analysis. The General Purpose Interrupt Controller (GIC) CPU interface 2750, when integrated with external allocator components, serves as a resource for supporting and managing interrupts in a cluster system. In a cluster, each Neoverse... TM A CPU bridge 2752 can exist between the V2 core 2700 and the DSU-110 2754. The CPU bridge 2752 controls buffering and synchronization between the core 2700 and the DSU-110 2754. The CPU bridge 2752 can be asynchronous to allow each core 2700 to use a different frequency, power, and area implementation point. The CPU bridge 2752 can operate synchronously without affecting other interfaces, such as, but not limited to, asynchronous debug and tracing interfaces.

[0294] In at least one embodiment, core 2700 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0295] Figure 28 One or more chips including one or more tensor processing units (TPUs) 2800 are shown according to at least one embodiment. Figure 28The TPU 2800 may include an application-specific integrated circuit (ASIC), for example, for performing some or all of the operations described above or elsewhere herein, such as, but not limited to, machine learning workloads that accelerate matrix operations. The TPU 2800 may be an ASIC from Alphabet Corporation, Mountain View, California. The cloud TPU includes a cloud service that enables the TPU to be used as a scalable resource for processing tasks, such as, but not limited to, machine learning workloads that can run on frameworks such as, but not limited to, TensorFlow, PyTorch, and JAX.

[0296] Chip 2800 may include any number of TPUs, which may include tensor cores 2806. Tensor cores 2806 may include one or more core sequencers 2808, vector processing units (VPUs) 2810, matrix multiplication units (MXUs) 2812(A)-2814(N) (where N is any integer greater than 1), and transpose permutation units 2816. Core sequencer 2808 may fetch instructions (e.g., VLIW (Very Long Instruction Word)) from the instruction memory (Imem) of core 2806, perform scalar operations using scalar data memory (Smem) and scalar registers (Sregs) (not shown), and forward vector instructions to vector processing units (VPUs) 2810. For example, an instruction may initiate eight operations: two scalar operations, two vector ALU operations, vector loading and storing, and queuing data into matrix multiplication and transpose units and a pair of slots for queuing data from them. The VPU2810 can perform vector operations using a large on-chip vector memory (Vmem) and vector registers (Vregs). The VPU 2810 can stream data to or from the MXU via a decoupled FIFO. The VPU 2810 can collect and distribute data to the Vmem using both data-level parallelism (2D matrix and vector function units) and instruction-level parallelism (8 operations per instruction). Large two-dimensional matrix multiplication units (MXUs) 2812(A)–2812(N) can, for example, use a systolic array to reduce area and power consumption, and use a large, software-controlled on-chip memory instead of a cache. The transpose-reduction-permute unit 2816 can perform (e.g., 128x128) matrix transpose, reduction, and permute on the VPU 2810 channels. A high-bandwidth memory 2804 can be used for on-chip applications and can be coupled to the host queue 2802, for example, via PCIe. One or more chips 2800 can be connected together for computing. For example, one or more chips 2800 can be connected to form a torus, such as a 2D torus. Chips 2800 can also include any number (e.g., four) of inter-core interconnect (ICI) links 2818, which can enable direct connections between chips to form a supercomputer.

[0297] For any processor in chip 2800 and any components described above or elsewhere herein, one or more APIs or equivalents described herein may, for example, be compiled into instructions or equivalents that may be fetched by instruction fetching logic or equivalents, decoded by processor decoder or equivalents, scheduled (e.g., sequentially or out of order) for execution by scheduler or equivalents, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. APIs (and / or compiled instructions including APIs) may be stored in any storage device (e.g., cache and / or memory) external to or internal to any processor in chip 2800. The results of the APIs may then be stored in any storage device internal to or external to any processor in chip 2800, including registers, DRAM, flash memory, SRAM, cache, or other memory equivalents.

[0298] In at least one embodiment, chip 2800 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; the instructions will be used to access one or more decompression algorithms for one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0299] Figure 29A vector processor according to at least one embodiment is illustrated. The vector processor 2900 may support the RISC-V standard. The vector processor 2900 may include one or more cores 2910 (e.g., scalar units) and one or more vector processing units (VPUs) 2942 (e.g., vector units), which may, for example, perform some or all of the operations described above or elsewhere herein. The core 2910 may include an Andes Custom Extension (ACE) 2916, which can be used to deliver custom instructions to the processor 2900, for example, via an ACP 2938. The core 2910 may include a 1-cycle multiplier and a 1-cycle instruction / data local memory (ILM / DLM) for improving parallelism by allowing simultaneous instruction fetching and data access. A memory management unit (MMU) 2924 manages system memory and cache, and provides branch execution, instruction pair issuance, L1 instruction / data cache, and local memory storage. The core 2910 may include a physical memory protection and programmable physical memory attribute unit (PMP / PPMA) 2922. Core 2910 may include a digital signal processor (DSP) 2928 and a floating-point unit (FPU) 2926, as well as a load-memory unit (LSU) 2932 for interaction with memory hierarchies (D$2934 and I$2930). Core 2910 may include a branch prediction unit 2918 and a multiplier unit 2920.

[0300] The vector processing unit (VPU) 2942 may include one or more vector function units (FU) 2946(A)-2946(N) (these vector function units may be linked together for parallel processing), a separate memory path for loading / storing RISC-V vectors (RVVs) via the ACE-RVV 2948 and the AndesStreaming port (ASP) 2944, and a vector load / store unit (VLSU) 2950.

[0301] The vector processor 2900 may include bus interfaces such as, but not limited to, a cache-accessible L2 cache port 2956, a non-cache-accessible MMIO port 2954, an input-output coherence port (IOCP) 2958 for a cacheless bus master, a local memory access port for accessing the ILM / DLM 2912 (which may be coupled to SRAM 2906) and the high-bandwidth vector memory (HVM) 2936, and a shared peripheral port (SPP) 2952 for external peripherals. Other memory ports include the LM slave port AXI 2902, the HVM subordinate port AXI 2904, MEM (AXI) 2962, and AXI 2960. The trace I / F 2914 can capture, encode, and transfer off-chip data, such as records of executed processor instructions, via the Inst.Trace I / F 2908. Software tools can use these records to reconstruct the exact execution sequence of a program.

[0302] For any processor in processor 2900 and any components described above or elsewhere herein, one or more APIs or equivalents described herein may, for example, be compiled into instructions or equivalents that may be fetched by instruction fetching logic or equivalents, decoded by processor decoder or equivalents, scheduled (e.g., sequentially or out of order) for execution by scheduler or equivalents, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. APIs (and / or compiled instructions including APIs) may be stored in any storage device external to or internal to processor 2900 (e.g., in cache and / or memory). The results of the APIs may then be stored in storage devices internal to or external to processor 2900, including registers, DRAM, flash memory, SRAM, cache, or other memory equivalents.

[0303] In at least one embodiment, the vector processor 2900 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0304] Figure 30A A schematic diagram of an example multi-core tiled processor microarchitecture is shown. Figure 30A Multi-core tiled processors in a system can include language processing processors. For example... Figure 30AAs shown, each “tile” in the processor architecture is a processing element bundled together using an on-chip network (NoC), which can be used to perform some or all of the operations described above or elsewhere in this document. For example, each tile may have an instruction dispatch 3004 and integer (INT) units 3006 and floating-point (FP) units 3008, a load-memory unit (LSU) 3012 for engaging with a memory hierarchy (data cache (D$) 3010 and instruction cache (I$) 3014), and a network (NET) interface 3016 for communicating with other tiles. Some tiles in the processor 3000 may include a memory controller 3002 for managing and controlling memory, as further described herein. The processor 3000 may have a functionally sliced ​​architecture. The processor 3000 may reside on an application-specific integrated circuit (ASIC). Figure 30A The layout of an ASIC can be represented. Processor 3000 may include a coprocessor designed to execute instructions for a predictive model. A predictive model refers to any model configured to make predictions based on input data. The predictive model can use a classifier for classification predictions. The predictive model can be a machine learning model, such as, but not limited to, a tensor flow model, and processor 3000 is a tensor flow processor.

[0305] The processor 3000 can employ different microarchitectures, and these microarchitectures will Figure 30B Each tile in the process represents a separate functional unit. Conversely, the functional tiles of the processor 3000 can be aggregated into multiple functional processing units (hereinafter referred to as "slices") 3004, each corresponding to a specific functional type (e.g., FP / INT 3018, NET 3020, MEM 3022). For example, as... Figure 30B As shown, each slice can correspond to a row of functional tiles extending in a north-south direction. Furthermore, the processor 3000 may also include communication channels for carrying data between tiles in different slices, each communication channel extending horizontally in an east-west direction. Each communication channel can be connected to each slice 3004 of the processor 3000.

[0306] The slices 3004 of the processor 3000 may each correspond to different functions and may include arithmetic logic slices (e.g., FP / INT 3018), channel switching slices (e.g., NET 3020), and memory slices (e.g., MEM 3022). Arithmetic logic units can perform one or more arithmetic and / or logical operations on data received via a communication channel to generate output data. Examples of arithmetic logic units may be matrix multiplication units and vector multiplication units. Memory slices include memory units that store data. Memory slices can provide data to other slices via communication channels. Memory slices can also receive data from other slices via communication channels. Channel switching slices can configurably route data from one communication channel to any other communication channel. For example, data from a first channel can be provided to a second channel via a channel switching slice. In some embodiments, a channel switching slice can be implemented as a crossbar switch. Each slice 3004 also includes its own instruction queue (not shown) for storing instructions and an instruction control unit (ICU) for controlling instruction execution. Instructions in a given instruction queue can only be executed by a tile in its associated functional slice and not by other slices of the processor 3000.

[0307] By arranging the tiles of processor 3000 into different functional slices 3004, the on-chip instruction and control flow of processor 3000 can be separated from the data flow. For example, according to some embodiments, Figure 30B One of the arrows illustrates the instruction flow within the processor architecture. According to at least one embodiment, Figure 30B Another arrow in the diagram illustrates the data flow within the processor architecture. As shown, instruction and control flow can move across the tiles of processor 3000 in a first direction (e.g., north-south along the length of a functional slice, as indicated by the first arrow), while data flow can move across the tiles of processor 3000 in a second direction (e.g., east-west across a functional slice, as indicated by the second arrow), which is perpendicular to the first direction.

[0308] Different functional slices of the processor 3000 can correspond to MEM 3022 (memory), VXM (vector execution module), MXM (matrix execution module), NIM (numerical interpretation module), and SXM (swapping and permutation module). Each slice can include N tiles, all of which can be controlled by the same instruction control unit (ICU) (not shown). Each slice can operate completely independently and can only be coordinated using barrier-like synchronization primitives or by the compiler using tractable determinism. Each tile of the processor 3000 can correspond to an execution unit organized as ×MSIMD tiles. For example, each tile of the on-chip memory of the processor 3000 can be organized to atomically store L-element vectors. Therefore, MEM slices with N tiles can work together to store or process large vectors (e.g., with a total of N×M elements).

[0309] Tiles within a slice can execute instructions in an "interleaved" manner, where instructions can be issued tile-by-tile within the slice over N cycle periods. Functional slices can be physically arranged on the chip to allow for efficient data flow for pipelined execution over hundreds of cycles for common patterns. The data flow can perform a single "u-turn" (direction change) corresponding to a single matrix operation before being written back to memory; in some embodiments, a particular data flow can change direction multiple times before writing the resulting data back to memory (due to multiple matrix and vector operations).

[0310] When using a processor 3000 with a function slice architecture (e.g., TSP), the TSP compiler (not shown) generates an explicit plan of how the processor 3000 executes programs (e.g., microprograms). The compiler can specify when each operation will be executed, which function slices will perform the work, and which STREAM registers will hold operands. The compiler can maintain a high-fidelity (cycle-accurate) model of the processor 3000 (e.g., TSP) hardware state so that microprograms can coordinate data flow.

[0311] Processor 3000 (e.g., TSP) can use a web-hosted compiler that takes a model (e.g., an ML model, such as but not limited to a TensorFlow model) as input and issues a proprietary stream of instructions for processor 3000 (e.g., TSP). The compiler is responsible for coordinating the control and data flow of the program and specifying any instruction-level parallelism by explicitly bundling instructions that can and should be executed concurrently so that they can be dispatched together. The main hardware architecture includes the architecture-visible streaming register file (STREAM), which will be described in more detail below, and acts as a conduit for operands to flow from MEM slices (e.g., SRAM) to functional slices (and vice versa).

[0312] The MEM 3022 of the processor 3000 can be used as: (1) a storage for model parameters, microprograms, and data on which they operate; and (2) an on-chip network (NoC) for transferring data operands from the MEM to functional slices and returning computation results to the MEM. In some embodiments, the on-chip memory may consume approximately 75% of the chip area of ​​the processor 3000. In some embodiments, the on-chip memory of the MEM tile may include SRAM instead of DRAM due to the bandwidth requirements of the processor 3000. The on-chip memory capacity of the processor 3000 can be determined by: (i) the number of ML models that can reside on the chip simultaneously, (ii) the size of any given model, and (iii) partitioning of large models for adaptation to a multi-chip system. In some embodiments, the MEM system of the processor 3000 may provide multiple memory slices organized into two distinct hemispheres (referred to as “MEM WEST” and “MEM EAST”, respectively).

[0313] The memory slices in each hemisphere can be mirrored, such that the slices are physically numbered {0,...L} in the Eastern Hemisphere and {L,...0} in the Western Hemisphere, such that memory slice 0 in each hemisphere corresponds to the slice of the VXM slice closest to the hemisphere, where each hemisphere comprises L slices. Data transfer towards the chip center can be referred to as inward, while data transfer towards the outer edge of the chip (easternmost or westernmost) can be referred to as outward. Although the memory hemispheres of the processor 3000 can be referred to as East and West, it is understood that other names may be used to refer to different memory hemispheres in other embodiments.

[0314] In some embodiments, streaming register files (referred to as STREAMS) transfer operands and results between the SRAM of the MEM slices of the processor 3000 and the functional slices. In some embodiments, multiple MEM slices (e.g., 2 to 10 adjacent MEM slices) can be physically organized into sets. Each slice set can be located between a pair of STREAMS register files, allowing each slice to read or write to the STREAMS registers in either direction. By placing the STREAMS register files between sets of MEM slices, the number of cycles required to transfer data operands across hemispheres can be reduced (e.g., reduced by a factor corresponding to the number of slices per set). The number of slices per set can be configured based on the distance of data transfer within a single clock cycle.

[0315] for Figures 30A-30BAny processor and any component described above or elsewhere herein, one or more APIs or equivalents described herein may, for example, be compiled into instructions or equivalents that may be fetched by instruction fetching logic or equivalents, decoded by processor decoder or equivalents, scheduled (e.g., sequentially or out of order) for execution by scheduler or equivalents, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. The API (and / or compiled instructions including the API) may be stored in any storage device internal or external to the processor 3000 (e.g., in cache and / or memory). The results of the API may then be stored in storage devices internal or external to the processor 3000, including registers, DRAM, flash memory, SRAM, cache, or other memory equivalents.

[0316] In at least one embodiment, the processor 3000 may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be software-configurable to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; indicate one or more decompression algorithms to be used to access one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0317] Software Structure

[0318] The following figures illustrate, in a non-limiting manner, examples of software structures for implementing at least one embodiment.

[0319] Figure 31 A software stack of a programming platform according to at least one embodiment is illustrated. The programming platform may include a platform for accelerating computational tasks by utilizing hardware on a computing system. In at least one embodiment, software developers can access the programming platform through libraries, compiler instructions, and / or extensions to programming languages. The programming platform may be CUDA, Radeon Open Computing Platform (“ROCm”), OpenCL (OpenCL... TMDeveloped by the Khronos Group), SYCL, or Intel OneAPI.

[0320] The software stack 3100 of the programming platform can provide an execution environment for the application 3101. The application 3101 may include any computer software that can be launched on the software stack 3100. The application 3101 may include artificial intelligence (“AI”) / machine learning (“ML”) applications, high-performance computing (“HPC”) applications, virtual desktop infrastructure (“VDI”), or data center workloads.

[0321] Application 3101 and software stack 3100 run on hardware 3108. Hardware 3108 may include one or more GPUs, CPUs, FPGAs, AI engines, and / or other types of computing devices supporting programming platforms. Software stack 3100 may be vendor-specific and compatible only with vendor-specific devices, such as CUDA, ROCm, OneAPI, OpenCL, or other implementations. Hardware 3108 may include a host connected to one or more devices that can be accessed via application programming interface (“API”) calls to perform computational tasks. In at least one embodiment, the devices within hardware 3108 may include GPUs, FPGAs, AI engines, or other computing devices (but may also include CPUs) and their memory, while the host within hardware 3108 may include CPUs (but may also include computing devices) and their memory. For any hardware 3108 described above or elsewhere herein, the one or more APIs described herein may, for example, be compiled into instructions that may be fetched by instruction fetching logic, decoded by a processor decoder, scheduled (e.g., sequentially or out of order) for execution by a scheduler, executed by execution logic, reordered, and then retired by retirement logic. The API (and / or compiled instructions including the API) may be stored in any storage device (e.g., cache and / or memory) internal or external to hardware 3108. The results of the API may be stored in storage devices internal or external to hardware 3108, including registers, DRAM, flash memory, SRAM, cache, or other memory. One or more APIs described herein may receive calls. One or more APIs described herein may communicate with a library or a portion of a library to perform the function described by the call. One or more APIs described herein may receive calls and communicate with a library or a portion of a library to perform the function described by the call.

[0322] The software stack 3100 of the programming platform may include multiple libraries 3103, a runtime 3105, optional drivers / interfaces 3107, and device kernel drivers 3108. Each library 3103 may include data and programming code that can be used by computer programs and utilized during software development. Libraries 3103 may include pre-written code and subroutines, classes, values, type specifications, configuration data, documentation, help data, and / or message templates. Libraries 3103 may include functionality optimized for execution on one or more types of devices. Libraries 3103 may include functionality for performing mathematical, deep learning, and / or other types of operations on the device. Libraries 3103 may be associated with corresponding APIs 3102, which may include one or more APIs for exposing the functionality implemented in libraries 3103. A processor (e.g., CPU, GPU) may execute, call, or otherwise use one or more APIs to determine kernel priority. For example, a first kernel (e.g., a parent kernel) may launch a second kernel (e.g., a child kernel), and the processor may use the second kernel to launch an additional kernel (e.g., a grandchild kernel) independent of the first kernel. The processor can execute APIs or call APIs from memory to support dynamic stream priorities (e.g., updating priorities when performing operations using streams). For example, when the processor executes the API, it allows the programmer to copy stream priorities from one stream to one or more other streams.

[0323] Software stack 3100 may include APIs that support dynamic stream prioritization (e.g., updating priority while performing operations on the stream), allowing programmers to set the stream's priority at any time after the stream is created. Software stack 3100 may include APIs that support dynamic stream prioritization (e.g., updating priority while performing operations on the stream), allowing programmers to obtain the stream's current priority, where the priority is one of several attributes of the stream. Software stack 3100 may include APIs that support dynamic stream prioritization (e.g., updating priority while performing operations on the stream), allowing programmers to obtain the stream's current priority as a single attribute. Software stack 3100 may include APIs that support dynamic stream prioritization (e.g., updating priority while performing operations on the stream), allowing programmers to start the kernel to perform operations on the stream at a set priority, which may be different from the stream priority. Software stack 3100 may include an API for indicating whether an object (e.g., a thread synchronization object, such as, but not limited to, a barrier) tracks whether all data movement operations of a set of threads running on the GPU have a specified state after a specified time period, wherein the specified state may be a state indicating that data has been moved and is ready for use, and is specified using expected parity values ​​as input to the API.

[0324] Software stack 3100 may include one or more APIs for updating the kernel. The processor may execute APIs or call APIs from memory to update existing APIs, thereby supporting a context-independent kernel. This allows programmers to add kernel nodes to a graph without a graph context, so that the graph context can be dynamically associated with the kernel at runtime. Software stack 3100 may include one or more APIs that allow programmers to obtain kernel identifiers and graph contexts as separate parameters from kernel nodes, thereby enabling parameter retrieval from both the kernel and the context-independent kernel. Software stack 3100 may include one or more APIs that use parallel processors (e.g., but not limited to one or more graphics processing units) to initiate task graphs (e.g., task graphs) and execute one or more task graphs (e.g., including one or more programs).

[0325] Software stack 3100 may include one or more APIs for associating one or more instructions with one or more memory sorting operations (e.g., but not limited to fence or memory barrier operations). Instructions may be associated with one or more domains, causing memory sorting operations to execute in association with one or more specific domains without interfering with instructions in other domains. APIs may indicate that a thread has reached (e.g., reached a thread synchronization barrier) or completed a certain phase of work associated with an asynchronous data movement operation on the GPU. Software stack 3100 may include one or more APIs that allow programmers to manually indicate an expected transaction count when a thread completes a certain phase of work; this transaction count can be used to update an object used to track whether all data movement operations for a set of threads have been completed.

[0326] Application 3101 can be written as source code and then compiled into executable code, as described below. Figure 32 and Figure 33 As discussed in more detail, the executable code of application 3101 can run, at least partially, within the execution environment provided by software stack 3100. During the execution of application 3101, code that needs to run on the device (rather than the host) may be encountered. In this case, runtime 3105 can be invoked to load and launch the required code on the device. Runtime 3105 can include any technically feasible runtime system capable of supporting the execution of application 3101.

[0327] Runtime 3105 can be implemented as one or more runtime libraries associated with a corresponding API (shown as API 3104). One or more such runtime libraries may include functions for memory management, execution control, device management, error handling, and / or synchronization, etc. Memory management functions may include functions for allocating, dealing with, and copying device memory, as well as functions for transferring data between host memory and device memory. Execution control functions may include functions for launching functions on the device (sometimes referred to as the "kernel" when the function is a global function that can be called from the host) and setting attribute values ​​in buffers maintained by the runtime libraries so that a given function can be executed on the device.

[0328] The runtime library and the corresponding API 3104 can be implemented in any technically feasible manner. One (or any number) APIs can expose a set of low-level functions for fine-grained control of the device, while another (or any number) APIs can expose a set of high-level functions for such functions. High-level runtime APIs can be built on top of low-level APIs. One or more runtime APIs can be language-specific APIs, which can be layered on top of language-independent runtime APIs.

[0329] Optional drivers or interfaces 3107 can be implemented, for example, for CUDA and ROCm implementations, which will be described further below. Optional drivers / interfaces 3107 can be associated with optional driver or interface APIs, such as, but not limited to, the CUDA and / or ROCm APIs.

[0330] One or more processors disclosed in the “processing system” may execute, access, or otherwise use the software stack 3100. For example, the system-on-a-chip 1800, parallel processor 1900, graphics multiprocessor 1934, processor 2000, processor 2100, accelerator 2200, neuromorphic processor 2305, supercomputer 2400, acceleration processing unit 2500, processor 2600, processor 2700, tensor processing unit 2800, processor 2900, and language processing unit 3000 may execute, use, call, or otherwise implement (e.g., by accessing memory) one or more APIs included in the software stack 3100.

[0331] Device kernel driver 3108 can be configured to facilitate communication with the underlying device. Device kernel driver 3108 can provide low-level functionality for APIs (such as, but not limited to, API 3104) and / or other software. Device kernel driver 3108 can be configured to compile intermediate representation (“IR”) code into binary code at runtime. For CUDA or other implementations (such as, but not limited to, ROCm, OneAPI, or OpenCL), device kernel driver 3108 can compile non-hardware-specific parallel thread execution (“PTX”) IR code into binary code for a specific target device at runtime (and cache the compiled binary code), which is sometimes referred to as “finalized” code. Doing so allows the finalized code to run on a target device that may not have existed when the source code was initially compiled into PTX code. Alternatively, the device source code can be compiled into binary code offline without device kernel driver 3108 compiling the IR code at runtime.

[0332] Processors described elsewhere in this document (e.g., but not limited to) Figure 18-30BThe processor may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be configured by software (e.g., software stack 3100) to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; the instructions will be used to access one or more decompression algorithms for one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0333] According to at least one embodiment, Figure 31 The software stack 3100 can execute in a CUDA implementation. The CUDA software stack 3100, on which applications 3101 can be launched, may include CUDA libraries 3103, a CUDA runtime 3105, a CUDA driver 3107, and a device kernel driver 3108. The CUDA software stack 3100 can execute on hardware (e.g., a graphics multiprocessor 1934, which may include a CUDA-enabled GPU developed by NVIDIA Corporation in Santa Clara, California).

[0334] Application 3101, CUDA runtime 3105, and device kernel driver 3108 can perform the functions described above and elsewhere in this document. CUDA driver 3107 may include a library (libcuda.so) that implements CUDA driver API 3106. Similar to CUDA runtime API 3104 implemented by the CUDA runtime library (cudart), CUDA driver API 3106 exposes functions for memory management, execution control, device management, error handling, synchronization, and / or graphics interoperability. CUDA driver API 3106 differs from CUDA runtime API 3104 in that CUDA runtime API 3104 simplifies device code management by providing implicit initialization, context (similar to processes), and module (similar to dynamically loaded libraries) management. Compared to the high-level CUDA runtime API 3104, CUDA driver API 3106 can serve as a low-level API, providing finer-grained device control, especially in terms of context and module loading. CUDA Driver API 3106 can expose context management functions not exposed in CUDA Runtime API 3104. CUDA Driver API 3106 can also be language-agnostic, supporting technologies such as OpenCL in addition to CUDA Runtime API 3104. Furthermore, development libraries, including CUDA Runtime 3105, can be considered separate from driver components, including user-mode CUDA driver 3107 and kernel-mode device driver 3108 (sometimes referred to as the "display" driver).

[0335] CUDA library 3103 may include mathematical libraries, deep learning libraries, parallel algorithm libraries, and / or signal / image / video processing libraries, which parallel computing applications (such as, but not limited to, application 3101) may utilize. CUDA library 3103 may include mathematical libraries, such as, but not limited to, the cuBLAS library (an implementation of the basic linear algebra subroutine (“BLAS”) for performing linear algebra operations), the cuFFT library (for computing the Fast Fourier Transform (“FFT”)), and the cuRAND library (for generating random numbers), etc. CUDA library 3103 may include deep learning libraries, such as, but not limited to, the cuDNN primitive library for deep neural networks and the TensorRT platform for high-performance deep learning inference, etc.

[0336] In at least one embodiment, the processor described elsewhere herein (e.g., but not limited to Figure) Figure 18-30BThe processor may include one or more circuits for causing information to be stored and storing indications of one or more decompression algorithms for decompressing the information, for causing one or more identifiers of one or more videos and one or more indications of pixel interpolation algorithms to be stored, for indicating one or more decompression algorithms to be used to access one or more frames of one or more videos, for performing interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames, for indicating one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, and for modifying one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise performing any of the operations described above or elsewhere herein. One or more circuits may be configured by software (e.g., software stack 3100) to: store information and store instructions for one or more decompression algorithms for decompressing the information; store one or more identifiers of one or more videos and one or more instructions for pixel interpolation algorithms; the instructions will be used to access one or more decompression algorithms for one or more frames of one or more videos; perform interpolation on two or more pixels of one or more video frames at least in part based on one or more frame identifiers of one or more video frames; indicate one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels; and modify one or more properties of one or more pixels at least in part based on one or more identifiers of one or more video frames containing one or more pixels, or otherwise perform any of the operations described above or elsewhere herein.

[0337] According to at least one embodiment, Figure 31 The software stack 3100 can execute in the ROCm implementation. An application 3101 can be launched on the ROCm software stack 3100, which includes a language runtime 3103, a system runtime 3105, a thunk 3107, and a ROCm kernel driver 3108. The ROCm software stack 3100 executes on hardware 3109, which may include a ROCm-enabled GPU developed by AMD Inc. of Santa Clara, California.

[0338] Application 3101 is executable and can be combined with the above. Figure 31Similar functionality to that discussed. Furthermore, the language runtime 3103 and system runtime 3105 can execute functions related to the above. Figure 31 The runtime discussed is similar in functionality to 3105. The difference between the language runtime 3103 and the system runtime 3105 is that the system runtime 3105 is a language-independent runtime that implements the ROCr System Runtime API 3104 and uses the Heterogeneous System...

Claims

1. A processor comprising: one or more circuits to execute an application programming interface (API) to cause information to be stored and store an indication of one or more decompression algorithms used to decompress the information.

2. The processor of claim 1, wherein the one or more circuits are to further execute the API to allocate memory on a processor to store the information.

3. The processor of claim 1, wherein the information is compressed video data.

4. The processor of claim 1, wherein the indication of the one or more decompression algorithms is to be stored in a table to match an identifier of the information.

5. The processor of claim 1, wherein the API is to be called by a CPU and executed on a GPU.

6. The processor of claim 1, wherein the one or more decompression algorithms are to be used by one or more other APIs to decompress the information.

7. The processor of claim 1, wherein the API is to be called from a host program of video editing software.

8. A method comprising: executing an application programming interface (API) to cause information to be stored and store an indication of one or more decompression algorithms used to decompress the information.

9. The method of claim 8, further comprising: executing the API to allocate memory on a processor to store the information.

10. The method of claim 8, wherein the information is compressed video data.

11. The method of claim 8, further comprising: storing the indication of the one or more decompression algorithms in a table to match an identifier of the information.

12. The method of claim 8, further comprising: calling the API by a central processing unit (CPU) and executing the API by a graphics processing unit (GPU).

13. The method of claim 8, wherein the one or more decompression algorithms are to be used by one or more other APIs to decompress the information.

14. The method of claim 8, further comprising: calling the API from a host program of video editing software.

15. A system comprising: one or more processors to execute an application programming interface (API) to cause information to be stored and store an indication of one or more decompression algorithms used to decompress the information.

16. The system of claim 15, wherein the one or more processors are to further execute the API to allocate memory on a processor to store the information.

17. The system of claim 15, wherein the information is compressed video data.

18. The system of claim 15, wherein the indication of the one or more decompression algorithms is to be stored in a table to match an identifier of the information.

19. The system of claim 15, wherein the API is to be called on a central processing unit (CPU) and executed on a graphics processing unit (GPU).

20. The system of claim 15, wherein one or more other APIs are to use the one or more decompression algorithms to decompress the information.

Images