Filtering unit

Abstract

A processing unit including a filtering unit is provided. The filtering unit is configured to apply filtering to a plurality of sequences of input values to determine an output value. The filtering unit comprises a control block, and the control block comprises a plurality of sequencers and an arbiter. The control block is configured to assign each of the sequences of input values to one of the sequencers, and each of the sequencers is configured to define a sequence of operations of a filtering process to be performed for the sequence of input values assigned to the sequencers. The filtering unit also includes a data path block including hardware logic configured to perform the operation by processing the value of the operation to generate a result of the operation as part of the filtering process. The arbiter is configured to control the sequencer's access to the hardware logic component of the data path block according to a prioritization rule, wherein each operation to be performed by the hardware logic component has a priority according to the prioritization rule.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority from UK patent application 2407296.9, filed 22 May 2024, which is incorporated herein in its entirety by reference. TECHNICAL FIELD

[0003] The present disclosure is directed to processing units comprising a filtering unit configured to apply filtering to input values to determine output values. In particular, the present disclosure relates to graphics processing units (GPUs) comprising a texture filtering unit configured to apply texture filtering to a plurality of sequences of input values to determine output values. The filtering can be part of a rendering process for rendering an image of a scene. BACKGROUND

[0004] Graphics processing units (GPUs) are used to render images of 3D scenes. A primitive is typically used to represent a surface (or “geometry”) of an object in a scene to be rendered. Some graphics processing units (GPUs), such as those configured to implement tile-based rendering techniques, implement a geometry processing stage and a fragment processing stage to render an image. Figure 1 A graphics processing system 100 is shown, which includes a GPU 102 that includes geometry processing logic 104 and fragment processing logic 106. The graphics processing system 100 also includes a memory 108. The geometry processing stage is implemented using the geometry processing logic 104, and the fragment processing stage is implemented using the fragment processing logic 106. Some logic of the GPU 102, such as execution units, can be used to implement parts of both the geometry processing stage and the fragment processing stage.

[0005] The GPU 102 receives geometry represented by a sequence of primitives from, for example, an application that submits a draw call to the GPU. The primitives can represent objects in a scene to be rendered. For example, the primitives can be points, lines, or polygons, such as triangles. A primitive can be represented by a set of vertices (e.g., a triangle primitive is represented by a set of three vertices), with data associated with each vertex. For example, vertex data for a vertex can describe the position of the vertex, as well as information related to the way in which the primitive including the vertex should be rendered (which can be referred to as “attributes” or “variables”). During the geometry processing stage, the geometry processing logic 104 performs one or more geometry processing functions on the primitives, such as transforming the positions of the vertices into render space, and clipping / pruning primitives that are outside of the render space. During the geometry processing stage, the geometry processing logic 104 can also perform binning on the processed primitives in order to determine which primitives are present within each tile of the render space. The geometry processing logic 104 outputs data, which can be stored in a buffer in the memory 108.

[0006] During the fragment processing stage, the fragment processing logic 106 reads in data from the memory 108 that was stored in the memory 108 by the geometry processing logic 104 during the geometry processing stage. The fragment processing logic 106 can independently process tiles of the render space. The fragment processing logic 106 reads primitive data for a primitive from the memory 108 and then performs fragment processing on the primitive. The fragment processing can involve, for example, (i) performing rasterization on the primitive to determine primitive fragments representing the primitive at discrete sample locations within the render space, (ii) performing hidden surface removal on the primitive fragments to remove fragments that are occluded, e.g., by other fragments in the scene, and (iii) performing fragment shading and / or texturing on the remaining fragments to determine an appearance at each sample location. Each sample location can correspond to a pixel of an image being rendered. In some instances, each pixel can correspond to multiple sample locations, and an averaging process can be performed on values determined at the sample locations in order to determine a rendered pixel value. The rendered pixel values can be stored, e.g., in a frame buffer. At the end of the fragment processing stage, the rendered pixel values can be output and used to represent an image. The image can be used in any suitable manner, e.g., the image can be stored, displayed on a display, and / or transmitted to another device, e.g., over a network such as the Internet.

[0007] Applying textures to primitive fragments is a common way of determining the appearance of primitives within a scene. Textures are typically represented as an array of one or more texels (i.e., a "bitmap"), where a "texel" in a texture is analogous to a "pixel" in an image. A texture can be sampled to determine a texture value to be applied to a primitive fragment at a particular location on the primitive. Directly reading from a texture often does not provide satisfactory image quality, as the projection of 3D geometry often requires some form of resampling, and as a result, as part of rendering a scene, a graphics processing unit (GPU) performs texture filtering. This can be because, for example, a primitive fragment location does not map exactly to an integer texel location in a texture, and in different cases, a pixel footprint can be larger or smaller than a texel footprint. Texture filtering can be applied to different rendering techniques, including rasterization and ray tracing.

[0008] There are many different texture filtering methods, including bilinear interpolation, volumetric filtering, anisotropic filtering, and trilinear filtering, and these methods can be applied in various combinations in various instances. Filtering can be a computationally expensive operation, and the hardware required to implement filtering can be large. Figure 1The fragment processing logic 106 shown in the middle includes a texture processing unit (TPU) 110, which includes a bilinear interpolation unit ("Bilerp") 112 and an accumulation filter ("Accfilt") 114. The bilinear interpolation unit 112 includes hardware configured to efficiently perform bilinear interpolation. The accumulation filter 114 includes hardware configured to efficiently perform one or more of volume filtering, anisotropic filtering, and trilinear filtering. UK patent GB2567507B describes how the accumulation filter 114 can be implemented. The present disclosure is primarily concerned with improvements in the way in which the accumulation filter 114 is implemented.

[0009] It is generally desirable to: (i) reduce latency of a processing unit such as a GPU, (ii) reduce power consumption of the processing unit, and (iii) reduce size (e.g., silicon area) of the processing unit. Tradeoffs can need to be made among these three factors. For example, size of the processing unit can be reduced at the expense of increasing latency and / or power consumption of the processing unit (e.g., by implementing more functionality in software rather than in hardware). As another example, latency of the processing unit can be reduced at the expense of increasing size and / or power consumption of the processing unit (e.g., by increasing the amount of hardware implemented in the processing unit). SUMMARY

[0010] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the DETAILED DESCRIPTION. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0011] A processing unit is provided, the processing unit comprising a filter unit configured to apply a filter to a plurality of sequences of input values to determine an output value, the filter unit comprising:

[0012] a control block comprising a plurality of sequencers and an arbiter, wherein the control block is configured to assign each of the sequences of input values to one of the sequencers, and wherein each of the sequencers is configured to define a sequence of operations of a filter process to be performed for the sequence of input values assigned to the sequencer; and

[0013] a data path block comprising hardware logic configured to perform the operations by processing values of the operations to generate results of the operations as part of the filter process;

[0014] wherein the arbiter is configured to control access of the sequencers to the hardware logic of the data path block in accordance with a prioritization rule, wherein each operation to be performed by the hardware logic has a priority in accordance with the prioritization rule, wherein:

[0015] Operations in the first group of operations have a high priority, wherein the first group of operations includes input operations involving input values;

[0016] Operations in the second group of operations have a medium priority, wherein the second group of operations includes intermediate operations that do not involve input values and determine intermediate result values rather than determining output values; and

[0017] Operations in the third group of operations have a low priority, wherein the third group of operations includes output operations that determine output values.

[0018] The filtering unit can be a texture filtering unit. The filtering process can be a texture filtering process.

[0019] The hardware logic can be configured to:

[0020] in each of a plurality of clock cycles, receive a plurality of values for the operation; and

[0021] pipeline process the received values to generate a result of the operation as part of the filtering process.

[0022] The hardware logic can be configured to pipeline process the received values of the operation for the plurality of clock cycles can be a number of clock cycles that is less than a number of sequencers in the control block.

[0023] The first group of operations can include all input operations.

[0024] The third group of operations can include all output operations that are not input operations.

[0025] The second group of operations can include all intermediate operations that do not involve input values and determine intermediate result values rather than determining output values.

[0026] The third group of operations can include all operations that do not involve input values and accumulate values derived from all previously received input values of the sequence of operations.

[0027] The second group of operations can include all operations that do not involve input values and do not accumulate values derived from all previously received input values of the sequence of operations.

[0028] Each of the sequencers can be configured to define an operation sequence in a filtering mode that includes anisotropic filtering and trilinear filtering, the operation sequence comprising: (i) a first group of anisotropic filtering operations ending with a dot product operation, (ii) a second group of anisotropic filtering operations, and (iii) a trilinear interpolation operation that combines the result of the first group of anisotropic filtering operations with the result of the second group of anisotropic filtering operations to determine the output value of the operation sequence, wherein the dot product operation at the end of the first group of anisotropic filtering operations may be in the third group of operations.

[0029] The second set of operations may include all intermediate operations of the dot product operation that are not at the end of the first set of anisotropic filtering operations in the filtering mode that includes anisotropic filtering and trilinear filtering.

[0030] The arbitrator can be configured to control the sequencer's access to the hardware logic components of the data path blocks by:

[0031] Receive from the sequencer a request to identify the operation to be performed by the hardware logic component of the data path block; and

[0032] The priorities of the operations identified by the request are compared to determine which of the identified operations should be sent to the hardware logic component of the data path block next.

[0033] The arbitrator can be configured to control the sequencer's access to the hardware logic components of the data path blocks by:

[0034] Determine which of the identified operations has the highest priority among the identified operations;

[0035] If it is determined that one of the identified operations has the highest priority among the identified operations, then the one of the identified operations is subsequently sent to the hardware logic component of the data path block; and

[0036] If it is determined that a plurality of the identified operations have the highest priority among the identified operations, then a tie-breaking mechanism is used to determine which of the plurality of the identified operations will be sent next to the hardware logic component of the data path block.

[0037] The tie-breaking mechanism can sort sequencers starting from a base sequencer. An arbitrator can be configured to determine which of the plurality of identified operations, determined to have the highest priority among the identified operations, was requested by the sequencer, which, according to the sequencer ordering in the tie-breaking mechanism, is the first of the plurality of sequencers requesting to be determined to have the highest priority among the plurality of identified operations.

[0038] Before all sequencers have sent all input operations in their operation sequences for processing by the hardware logic unit, the base sequencer may be one of the sequencers that has not yet sent all input operations in its operation sequence for processing by the hardware logic unit. The arbitrator may be configured to update the indication of which of the sequencers is currently the base sequencer in response to determining that the final input operation of the operation sequence requested by the sequencer currently acting as the base sequencer has been sent to the hardware logic unit, such that, if possible, the base sequencer is the sequencer that has not yet sent all of the input operations in its operation sequence for processing by the hardware logic unit.

[0039] The data path block may also include a set of temporary registers for each sequencer. These temporary registers for the sequencer may be arranged to store intermediate result values ​​generated by the hardware logic components when operations are performed in the sequence of operations defined by the sequencer.

[0040] The sequencer can be configured to determine a destination location in one of the temporary registers for storing each intermediate result value generated by the hardware logic components.

[0041] The hardware logic components may include a first input and a second input. The set of temporary registers used for the sequencer may include:

[0042] The first subgroup of temporary registers is configured to provide values ​​to the first input of the hardware logic component instead of to the second input of the hardware logic component; and

[0043] The second subgroup of temporary registers is configured to provide values ​​to the second input of the hardware logic component instead of the first input of the hardware logic component.

[0044] The data path block may include multiplexing logic, which is configured to:

[0045] Receive a value from the temporary register;

[0046] Receive input values; and

[0047] The value is output to the first input and the second input of the hardware logic component.

[0048] The arbitrator can be configured to use the multiplexing logic to control which values ​​are provided to the first and second inputs of the hardware logic component in each of a plurality of clock cycles.

[0049] The sequencer can be configured such that if two intermediate result values ​​will be used together in subsequent operations of hardware logic components, the first of the two intermediate result values ​​is stored in a temporary register in a first subgroup, and the second of the two intermediate result values ​​is stored in a temporary register in a second subgroup.

[0050] The set of temporary registers used for each sequencer may include five temporary registers. The temporary registers for the first subgroup of the sequencer may include two temporary registers, and the temporary registers for the second subgroup of the sequencer may include three temporary registers.

[0051] Hardware logic components can be two-dimensional dot product units.

[0052] A data path block may not include a dedicated addition unit. A data path block can be configured to perform addition operations using a two-dimensional dot product unit by setting the coefficients of the two-dimensional dot product unit to have a value of 1.

[0053] The filtering unit can be configured to apply filtering in a filtering mode, which includes one or more of volumetric filtering, anisotropic filtering, and trilinear filtering. The filtering unit can also be configured to:

[0054] Receive an indication of which of the following filters—volume filtering, anisotropic filtering, and trilinear filtering—will be applied; and

[0055] Receive one or more filter coefficients for applying the indicated filter.

[0056] The filtering unit can be arranged to receive the plurality of input value sequences as a set of input values. The input values ​​within the set can be interleaved input values ​​from the plurality of input value sequences. The input values ​​within the set can be accessed simultaneously from memory.

[0057] The processing unit may also include a bilinear interpolation unit configured to apply bilinear interpolation to the value and provide the input value to the filtering unit.

[0058] The input value can be an input texture value. The output value can be an output texture value.

[0059] Each sequence can represent a color channel.

[0060] The filtering unit can be implemented in hardware logic.

[0061] The processing unit can be a graphics processing unit.

[0062] The filtering process can be part of the process used to render the image of the scene. The process used to render the image of the scene can be a rasterization process or a ray tracing process.

[0063] The processing unit can be contained in hardware on an integrated circuit.

[0064] A method is provided for applying filtering to a sequence of multiple input values ​​within a processing unit to determine an output value, wherein the processing unit includes a filtering unit, the filtering unit includes multiple sequencers and a data path block, the data path block includes hardware logic components, and the method includes:

[0065] Each of the input value sequences is assigned to one of the sequencers, wherein each sequencer defines a sequence of operations for a filtering process to be performed on the input value sequence assigned to the sequencer;

[0066] The sequencer's access to the hardware logic components of the data path block is controlled according to the prioritization rules.

[0067] The operation is performed at the hardware logic component by processing the value of the operation to generate the result of the operation as part of the filtering process;

[0068] Each operation to be performed by the hardware logic component has a priority according to the prioritization rule, wherein:

[0069] The operations in the first group of operations have high priority, and the first group of operations includes input operations involving input values;

[0070] The operations in the second group of operations have medium priority, and this second group of operations includes intermediate operations that do not involve input values ​​and determine intermediate result values ​​rather than output values; and

[0071] The operations in the third group of operations have low priority, and the third group of operations includes output operations that determine the output value.

[0072] The access of the control sequencer to the hardware logic components of the data path block may include:

[0073] Receive from the sequencer a request to identify the operation to be performed by the hardware logic component of the data path block;

[0074] Determine which of the identified operations has the highest priority among the identified operations;

[0075] If it is determined that one of the identified operations has the highest priority among the identified operations, then the one of the identified operations is subsequently sent to the hardware logic component of the data path block; and

[0076] If it is determined that a plurality of the identified operations have the highest priority among the identified operations, then a tie-breaking mechanism is used to determine which of the plurality of the identified operations will be sent next to the hardware logic component of the data path block.

[0077] The tie-breaking mechanism can sort the sequencers starting from the base sequencer. The hardware logic component that determines which of the plurality of identified operations will be sent next to the data path block according to the tie-breaking mechanism may include:

[0078] The sequencer determines which of the plurality of identified operations is requested by the sequencer that is determined to have the highest priority among the identified operations, the sequencer being the first of the sequencers that requested to be determined to have the highest priority among the plurality of identified operations according to the sequencer ordering in the tie-breaking mechanism.

[0079] Before all sequencers have sent all input operations in their operation sequences for processing by the hardware logic unit, the base sequencer may be one of the sequencers that has not yet sent all input operations in its operation sequence for processing by the hardware logic unit. The method may include updating an indication of which of the sequencers is currently the base sequencer in response to determining that the final input operation of the operation sequence requested by the sequencer currently acting as the base sequencer has been sent to the hardware logic unit, such that, if possible, the base sequencer is the sequencer that has not yet sent all of the input operations in its operation sequence for processing by the hardware logic unit.

[0080] Computer-readable code may be provided, which is configured to enable any of the methods described herein to be performed when the code is executed.

[0081] An integrated circuit definition dataset may be provided, which, when processed in an integrated circuit manufacturing system, configures the integrated circuit manufacturing system to manufacture processing units as described herein.

[0082] A processing unit may be provided, the processing unit including a filtering unit configured to apply filtering to a plurality of input value sequences to determine an output value, the filtering unit including:

[0083] A control block, comprising a plurality of sequencers, wherein the control block is configured to assign each of the input value sequence to one of the sequencers, and wherein each of the sequencers is configured to define an operation sequence for a filtering process to be performed on the input value sequence assigned to the sequencer; and a data path block, comprising:

[0084] A hardware logic component, comprising a first input and a second input, wherein the hardware logic component is configured to perform an operation by processing values ​​received at the first and second inputs for operation to generate a result of the operation as part of the filtering process; and

[0085] A set of temporary registers for each of the sequencers, wherein the set of temporary registers for the sequencers is arranged to store intermediate result values ​​generated by the hardware logic components when an operation in the sequence of operations defined by the sequencers is performed, wherein the set of temporary registers for the sequencers includes:

[0086] The first subgroup of temporary registers is configured to provide values ​​to the first input of the hardware logic component instead of to the second input of the hardware logic component; and

[0087] The second subgroup of temporary registers is configured to provide values ​​to the second input of the hardware logic component instead of the first input of the hardware logic component.

[0088] The data path block may also include multiplexing logic, which is configured to:

[0089] Receive a value from the temporary register;

[0090] Receive input values; and

[0091] The value is output to the first input and the second input of the hardware logic component.

[0092] The control block may also include an arbitrator configured to use the multiplexing logic to control which values ​​are provided to the first and second inputs of the hardware logic component in each of a plurality of clock cycles.

[0093] The sequencer can be configured to determine a destination location in one of the temporary registers for storing each intermediate result value generated by the hardware logic components.

[0094] The sequencer can be configured such that if two intermediate result values ​​will be used together in subsequent operations of hardware logic components, the first of the two intermediate result values ​​is stored in a temporary register in a first subgroup, and the second of the two intermediate result values ​​is stored in a temporary register in a second subgroup.

[0095] A method can be provided to apply filtering to a sequence of multiple input values ​​within a processing unit to determine an output value, wherein the processing unit includes a filtering unit, the filtering unit including a plurality of sequencers and a data path block, the data path block including hardware logic components and a set of temporary registers for each of the sequencers, the method comprising:

[0096] Each of the input value sequences is assigned to one of the sequencers, wherein each sequencer defines a sequence of operations for a filtering process to be performed on the input value sequence assigned to the sequencer;

[0097] The operation is performed at the hardware logic unit by processing values ​​received at a first and a second input for operation to generate a result of the operation, as part of the filtering process; and

[0098] For each of the sequencers, intermediate result values ​​generated by the hardware logic components when performing operations in the sequence of operations defined by the sequencer are stored in the set of temporary registers used by the sequencer.

[0099] The set of temporary registers used for the sequencer includes:

[0100] The first subgroup of temporary registers is configured to provide values ​​to the first input of the hardware logic component instead of to the second input of the hardware logic component; and

[0101] The second subgroup of temporary registers is configured to provide values ​​to the second input of the hardware logic component instead of the first input of the hardware logic component.

[0102] More generally, the filtering unit described herein is an example of a computational unit capable of processing multiple sequences of input values ​​to determine an output value. In particular, a processing unit including a computational unit configured to process multiple sequences of input values ​​to determine an output value may be provided, the computational unit comprising:

[0103] A control block comprising a plurality of sequencers and an arbitrator, wherein the control block is configured to assign each of the input value sequence to one of the sequencers, and wherein each of the sequencers is configured to define a sequence of operations to be performed on the input value sequence assigned to the sequencer; and

[0104] A data path block, the data path block including hardware logic components, the hardware logic components being configured to perform the operation by processing the value of the operation to generate the result of the operation as part of the process;

[0105] The arbitrator is configured to control the sequencer's access to the hardware logic components of the data path block according to a prioritization rule, wherein each operation performed by the hardware logic components has a priority according to the prioritization rule, wherein:

[0106] The operations in the first group of operations have high priority, and the first group of operations includes input operations involving input values;

[0107] The operations in the second group of operations have medium priority, and this second group of operations includes intermediate operations that do not involve input values ​​and determine intermediate result values ​​rather than output values; and

[0108] The operations in the third group of operations have low priority, and the third group of operations includes output operations that determine the output value.

[0109] The computation unit can be a filtering unit, such as a texture filtering unit. The computation unit can be configured to process the plurality of input value sequences by applying a filter, such as a texture filter, to the plurality of input value sequences to determine an output value. The operation sequence defined by the sequencer can be an operation sequence of a filtering process, such as an operation sequence of a texture filtering process.

[0110] A method can be provided for processing multiple input value sequences within a processing unit to determine an output value, wherein the processing unit includes a computation unit, the computation unit includes multiple sequencers and a data path block, the data path block includes hardware logic components, and the method includes:

[0111] Each of the input value sequences is assigned to one of the sequencers, wherein each sequencer defines a sequence of operations to be performed on the input value sequence assigned to the sequencer;

[0112] The sequencer's access to the hardware logic components of the data path block is controlled according to the prioritization rules.

[0113] The operation is performed at the hardware logic component by processing the value of the operation to generate the result of the operation, as part of the process;

[0114] Each operation to be performed by the hardware logic component has a priority according to the prioritization rule, wherein:

[0115] The operations in the first group of operations have high priority, and the first group of operations includes input operations involving input values;

[0116] The operations in the second group of operations have medium priority, and this second group of operations includes intermediate operations that do not involve input values ​​and determine intermediate result values ​​rather than output values; and

[0117] The operations in the third group of operations have low priority, and the third group of operations includes output operations that determine the output value.

[0118] The processing unit may be contained in hardware on an integrated circuit. A method for manufacturing the processing unit at an integrated circuit manufacturing system may be provided. An integrated circuit definition dataset may be provided, which, when processed in the integrated circuit manufacturing system, configures the system to manufacture the processing unit. A non-transitory computer-readable storage medium may be provided, on which a computer-readable description of the processing unit is stored, which, when processed in the integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit containing the processing unit.

[0119] An integrated circuit manufacturing system may be provided, comprising: a non-transitory computer-readable storage medium storing a computer-readable description of processing units thereon; a layout processing system configured to process the computer-readable description to generate a circuit layout description of an integrated circuit including the processing units; and an integrated circuit generation system configured to manufacture the processing units according to the circuit layout description.

[0120] Computer program code for performing any of the methods described herein may be provided. A non-transitory computer-readable storage medium may be provided having computer-readable instructions stored thereon, which, when executed in a computer system, cause the computer system to perform any of the methods described herein.

[0121] As will be apparent to those skilled in the art, the above features can be appropriately combined and can be combined with any aspect of the examples described herein. Attached Figure Description

[0122] The example will now be described in detail with reference to the accompanying drawings, in which:

[0123] Figure 1 The graphics processing system is shown;

[0124] Figure 2A An example of volume filtering is shown;

[0125] Figure 2B An example of anisotropic filtering is shown;

[0126] Figure 2C An example of trilinear filtering is shown;

[0127] Figure 3This is the first illustration of the instance texture filtering unit;

[0128] Figure 4A , Figure 4B and Figure 4C Graphical representations of three different sequences of texture filtering operations that can be implemented by the texture filtering unit as described herein are shown;

[0129] Figure 5 This is the second illustration of the instance texture filtering unit;

[0130] Figure 6 This is a flowchart illustrating a method, as described in this paper, for applying filtering to a sequence of multiple input values ​​within a processing unit to determine an output value;

[0131] Figure 7 Four distinct sets of instances of the input value sequence are shown;

[0132] Figures 8A-8G The example timing diagrams for different example filtering modes are shown, along with the registers used to store the results of the operations performed in the different filtering modes.

[0133] Figure 9 A computer system in which the processing system is implemented is shown; and

[0134] Figure 10 An integrated circuit manufacturing system for producing integrated circuits containing processing units is shown.

[0135] The accompanying drawings illustrate various examples. Those skilled in the art will understand that the element boundaries (e.g., boxes, groups of boxes, or other shapes) shown in the drawings represent one instance of a boundary. In some instances, it may be that one element can be designed as multiple elements, or multiple elements can be designed as one element. Where appropriate, common reference numerals are used throughout the drawings to indicate similar features. Detailed Implementation

[0136] The following description is presented with the aid of examples to enable those skilled in the art to make and use the invention. The invention is not limited to the embodiments described herein, and various modifications to the disclosed embodiments will be readily apparent to those skilled in the art.

[0137] The embodiments will now be described with the aid of examples only. As described above, the texture processing unit (TPU) 110 samples the texels of the texture and performs filtering on the sampled texels to determine the filtered texture value of the fragment being processed in the GPU 102. Specifically, the TPU 110 performs address calculation to locate the texture in memory 108, requests texture information from memory using the calculated address, decodes the requested texture data as needed, then performs filtering on the texels, and then outputs the determined color value, for example, back to the execution unit in the GPU 102, which can apply the determined color value to the primitive fragment as part of the fragment processing phase.

[0138] The way TPU 110 performs texture filtering depends on the filtering mode used in a particular application. TPU has dedicated hardware to accelerate some of the most common filtering techniques. For example, TPU 110 includes a bilinear interpolation unit 112 configured to apply bilinear interpolation to texture values ​​(e.g., texels). TPU 110 can also perform more complex types of filtering, such as volumetric filtering of 3D textures, anisotropic filtering, and / or trilinear filtering (involving texels from different mipmap levels). To perform these more complex types of filtering, TPU includes a texture filtering unit 114 implemented in hardware (in... Figure 1 This is referred to as an accfilt filter. If the TPU 110 does not support a particular filtering algorithm, it can perform some steps in dedicated hardware and then pass the resulting intermediate values ​​to the execution unit (i.e., the "shader core") in the GPU to perform the final calculation, or it can bypass the dedicated hardware entirely and send the texels directly to the execution unit.

[0139] Some filtering patterns use bilinear filtering as a “building block” or first step in more complex filtering techniques. For example, applications performing anisotropic filtering typically first perform a bilinear filtering step to compute an intermediate value, and then apply an anisotropic “weighted average” to the intermediate value. Therefore, bilinear interpolation unit 112 can be configured to apply bilinear interpolation to texture values ​​(e.g., texels) and then provide the values ​​as input to texture filtering unit 114. However, in some other instances, texture filtering unit 114 may receive texture values ​​(e.g., texels) as input values ​​that have not yet been processed by bilinear interpolation unit 112.

[0140] This disclosure relates to improvements to the texture filtering unit 114. As described above, the texture filtering unit 114 includes hardware configured to perform different types of filtering. In particular, the texture filtering unit 114 is configured to perform one or more of volumetric filtering, anisotropic filtering, and trilinear filtering, or any combination thereof. For example, the texture filtering unit 114 may receive two input samples per clock cycle and perform filtering operations on them, as defined by control inputs. The filtering operations may include any combination of volumetric filtering, anisotropic filtering, and trilinear filtering operations, and in the examples described in detail herein, are performed in the following order: volumetric filtering, anisotropic filtering, and trilinear filtering.

[0141] Figure 2A An example of volumetric filtering for filtering texels of a 3D texture is shown. Texture values ​​are sampled at location 202 within the 3D texture. The TPU 110 acquires eight texels 2041 to 2048 of the texture closest to sample location 202. (See diagram) Figure 2A As shown, eight acquired texels 204 form a cube around the sample positioning 202. Specifically, the eight texels form two planes on opposite sides of the sample positioning 202. Figure 2A As shown, texels 2041 to 2044 form a first plane below sample location 202, and texels 2045 to 2048 form a second plane above sample location 202. A bilinear interpolation unit 112 performs bilinear interpolation to determine interpolated values ​​at locations 2061 and 2062 within each plane. Specifically, the bilinear interpolation unit 112 determines the interpolated value at location 2061 in the first plane by performing bilinear interpolation on texels 2041 to 2044 forming the first plane; and the bilinear interpolation unit 112 determines the interpolated value at location 2062 in the second plane by performing bilinear interpolation on texels 2045 to 2048 forming the second plane. The interpolated values ​​at locations 2061 and 2062 are provided as input values ​​to a texture filtering unit 114. The texture filtering unit 114 performs interpolation (referred to herein as a volumetric filtering operation) to determine an output value representing the texture value at sample location 202.

[0142] Figure 2B An example of anisotropic filtering is shown. Anisotropic filtering can be used to improve the rendering of textures at tilt angles. To implement anisotropic filtering, the texture filtering unit 114 acquires more samples in one direction than in the vertical direction. Figure 2BIn the example shown, texture filtering unit 114 acquires six samples 2141 to 2146. Typically, samples are acquired along a straight line, but other patterns are possible. The anisotropy level is expressed as the "anisotropy ratio," which can be, for example, between 2 and 16. The number of samples acquired by texture filtering unit 114 can be determined by rounding the anisotropy ratio to the nearest even number. To determine the result of anisotropy filtering, texture filtering unit 114 determines a weighted average of the sample values. For example, for all samples except the last two, the weight in the weighted sum is determined to be 1 / (anisotropy ratio), and then the weights of the last two samples are determined to be equal to each other and set to a single value such that the sum of the weights of all samples is 1. To implement anisotropic filtering with an anisotropy ratio greater than 2, texture filtering unit 114 is configured to: (i) sum the samples that are "not the last samples," i.e., samples other than the last two samples; (ii) sum the last two samples separately; and (iii) perform a dot product of the two sums with appropriate weights. When the anisotropy ratio is 2, there are only two samples. To implement anisotropic filtering with an anisotropy ratio of 2, texture filtering unit 114 is configured to perform a dot product to multiply the two sample values ​​by the same weights and add the results.

[0143] Figure 2C An example of trilinear filtering is shown. Textures can be stored as "mipmaps," which are arrays of texels stored at different resolution levels. For example, the height and width of each mipmap level can be twice the size of the previous mipmap level, making each mipmap level one-quarter the size of the previous mipmap level. When sampling a texture, mipmaps are used to avoid scaling artifacts, which can be caused by sampling the texture at a resolution unsuitable for the primitive fragment to which the sampled texture values ​​are applied. The TPU 110 can determine the level of detail (LOD) to be sampled from the texture. The LOD can depend on the distance between the viewpoint in the scene and the fragment to which the texture is being applied. The determined LOD may not precisely correspond to the resolution of one of the texture's mipmap levels; that is, the determined LOD often falls between two mipmap levels. The TPU 110 selects the two mipmap levels closest to the determined LOD and determines the filter value from said two mipmap levels. For example, refer to... Figure 2CTPU 110 selects two mipmap levels 2161 and 2162 that are closest to the determined LOD. TPU 110 determines a first filter value 2181 from the first mipmap level 2161 and a second filter value 2182 from the second mipmap level 2162. Then, texture filtering unit 114 determines the result of trilinear filtering by performing interpolation between the first determined filter value 2181 and the second determined filter value 2182. Figure 2C The LOD score between the two mipmap levels is represented as 220, where the LOD score can be used as an interpolation coefficient in the interpolation.

[0144] As described above, the texture filtering unit 114 can perform one or more of volumetric filtering, anisotropic filtering, and trilinear filtering in any combination.

[0145] Figure 3 This is a first illustration of an instance texture filtering unit 300, which can be implemented as follows: Figure 1 The texture filtering unit 114 in the processing system 100 shown. For example... Figure 3 As shown, the texture filtering unit 300 includes several inputs, including multiple filter coefficients 302 and two input values ​​304 and 306, which may be texture values. These input values ​​304 and 306 are input data filtered in a filtering operation performed by the texture filtering unit 300, and the precise filtering operation performed is at least partially specified by the filter coefficients 302. In various instances, the texture filtering unit 300 may receive one input value per clock cycle (via input 304 or input 306); however, in other instances, the texture filtering unit may be configured to receive two or more input values ​​per clock cycle, and if more than two input values ​​are to be received per clock cycle, the texture filtering unit 300 may include additional inputs for this purpose. Figure 3 (Not shown in the text). The received input value (e.g., which may be the result of bilinear filtering performed by the bilinear interpolation unit 112) is typically a floating-point value (e.g., a full-precision binary floating-point format that may be referred to as F32); however, the hardware and methods described herein can be used with input values ​​of any format (e.g., half-precision binary floating-point format, F16, or integer format).

[0146] The texture filtering unit 300 also includes a control block 308 and a data path block 310. The control block 308 includes multiple sequencers 312, arbitrators 314, and multiplexers 316. In various instances, the data path block 310 may include, for example... Figure 3The four sequencers 312 shown are identical. In other instances, different numbers of sequencers may exist. The data path block 310 includes hardware logic unit 318 and a set of temporary registers 320 for each sequencer 312 (e.g., in...). Figure 3 In the example shown, there are four sets of temporary registers (one set for each sequencer). In various examples, each set of temporary registers 320 may include five or six registers. In other examples, different numbers of registers may exist in each set of temporary registers 320, and in some examples, the temporary registers 320 may be replaced by alternative memory structures configured to have similar functionality (e.g., FIFO or other memory).

[0147] Data path block 210 also includes multiplexing logic 322 and 323, which control where intermediate values ​​are stored. Specifically, the first multiplexing logic 322 (which may be referred to as a storage multiplexer) controls where intermediate values ​​are stored (i.e., in which set of temporary registers 320), and the second multiplexing logic 323 (which may be referred to as a source multiplexer) controls which values ​​(whether input or intermediate) are input to hardware logic unit 318. In other instances, multiplexers 322-323 may be replaced by alternative logic that performs similar switching and / or selection functions. The operation of multiplexing logic 322-323 is as follows: Figure 3 In the example shown, it is controlled by control block 308, for example, as Figure 3 The control lines from arbitrator 314 to multiplexers 322 and 323 and to hardware logic unit 318 are shown. Arbitrator 314 can select the sequencer operation to be performed and can determine the storage destination of the instruction, wherein the destination indication is sent in parallel with the operation through the hardware logic unit, and then controls the write enable of the multiplexing and temporary register 320 (of multiplexer 322). However, in other instances, the control of multiplexers 322-323 can be implemented as a separate pipeline, FIFO, or other state machine.

[0148] Texture filtering unit 300 receives multiple input values ​​(e.g., texture values) at inputs 304 and 306, and generates a single filtered output (output via output 324) in data path block 310 under the control of control block 308. Filtering is implemented by hardware logic unit 318, which may include, for example, one or more adder units, multipliers, two-dimensional dot product units (DP2), three-input adders, fused multiply-add (FMA) adders, or any other suitable computational logic unit. In the primary example described herein, hardware logic unit 318 is a two-dimensional dot product unit (DP2) that, given four inputs a, b, c, d, can output the value ac + bd, and also allows efficient computation of (c-1)*a + c*b. The operation sequence may require several passes through hardware logic unit 318 (described in more detail below), and intermediate results (i.e., results that are not the final filtered output value output by hardware logic unit 318) are stored in temporary register 320 of the corresponding sequencer (i.e., the sequencer that controls the pipeline when generating intermediate results). These intermediate results can then be input from temporary register 320 back to hardware logic unit 318 as input for subsequent passes through hardware logic unit 318. In various instances, the final filtered output value may also be stored in temporary register 320 (e.g., when it cannot yet be sent to the output but the texture filtering unit wishes to continue further computation without delay). Such stored final filtered output values ​​can then be output from the temporary register, either bypassing or passing through hardware logic unit 318, for example, as interpolations with zero coefficients. This ability to store the final filtered output value can significantly improve performance when alternating between simple and complex filtering functions (e.g., between anisotropic-2 filtering mode and anisotropic-16 filtering mode).

[0149] In various instances, the texture filtering unit 300 is arranged to perform any combination of volumetric filtering, anisotropic filtering, and trilinear filtering using hardware logic unit 318; however, in other instances, the texture filtering unit 300 is arranged to perform any combination of two or more different filtering methods and can perform any kind of linear filtering, and in various instances, bilinear filtering can also be performed. Therefore, the coefficients 302 input to the texture filtering unit 300 include at least one coefficient for each filtering method that the texture filtering unit 300 can implement, such as vfrac, afrac, and tfrac (collectively referred to as *frac), where vfrac is the coefficient for volumetric filtering, afrac is the coefficient for anisotropic filtering, and tfrac is the coefficient for trilinear filtering. In various instances, the values ​​of the coefficients can change every clock cycle, or can change less frequently, or can be constant (e.g., vfrac can change every clock cycle, afrac can change less frequently, and tfrac can be constant). When using only an appropriate subset of filtering methods, the coefficients of those unused methods can be set to default values ​​(e.g., the coefficient afrac can be set to one when anisotropic filtering is not used), or a separate enable signal 303 can be provided.

[0150] In instances where enable signals 303 are provided, these may have values ​​specifying whether each filtering method (or mode) is enabled, along with any necessary parameters for the filtering method (e.g., anisotropic filtering mode). For example, providing three enable signals may be detailed below, but certain combinations (e.g., {0, 0, 0}) may not be permitted or may have special meanings (e.g., causing bypassing the input to the output):

[0151]

[0152] Texture filtering unit 300 is arranged to perform filtering using one or more of a set of filtering methods, or any combination thereof, and as described above, this can involve computation using multiple input values ​​and multiple passes through hardware logic unit 318 (where each pass uses a different input value, which may be an input texture value or an intermediate value computed in a previous pass through the hardware logic unit). Examples of texture filtering operations are shown in... Figure 4A , Figure 4B and Figure 4C As shown in the figure. In these examples, hardware logic component 318 is the DP2 unit.

[0153] Figure 4AA graphical representation of the sequence of operations performed when texture filtering unit 300 combines volumetric filtering and anisotropic-2 filtering (e.g., where vol_en = 1 and ani_rt = 1) is shown. In this example, volumetric interpolation is first performed using DP2 unit 318 through two consecutive operations 402, followed by anisotropic multiplication and a final addition (box 404, i.e., the third pass through DP2 unit 318) using DP2 unit 318 again. If trilinear filtering (e.g., where vol_en = 1, tri_en = 1, and ani_rt = 1) is also used, as... Figure 4A As shown, trilinear interpolation can then be performed again using DP2 unit 318 (box 406, i.e., in the fourth pass through DP2 unit 318). The total number of input values ​​is given by the following formula:

[0154] Number of input values ​​= (vol_en+1)*(ani_rt+1)*(tri_en+1)

[0155] Therefore, if only volumetric filtering and anisotropic-2 filtering are used, there are only 4 inputs (2*2=4), while if trilinear filtering is also used, there are 8 inputs (2*2*2=8). When each input transaction has two inputs, the number of transactions is given by dividing the number of inputs by two, and... Figures 4A-4C The numbers at the top of the operation sequence diagram shown refer to these transactions, not the inputs.

[0156] Figure 4B A graphical representation of the sequence of operations performed when texture filtering involves anisotropy -4+ is shown, where all anisotropic modes starting from 4 are involved (e.g., where vol_en = 0 and ani_rt > 1), and Figure 4B The square brackets surrounding the input transactions indicate those transactions that will not occur for anisotropy -8. In this example, all 16 input values ​​(1*16*1=16) are first summed in pairs using DP2 unit 318 through 8 consecutive operations 412, i.e., through 8 iterations of DP2 unit 318. Figure 4B In the example shown, before DP2 unit 318 is used for anisotropic multiplication and final addition (box 416), multiple operations are performed using DP2 unit 318 (box 414, i.e., six more passes through DP2 unit 318) to accumulate the group (0-6) as: ((((0+1)+4)+6)+((2+3)+5)). If trilinear filtering is also used (e.g., where vol_en = 0, tri_en = 1, and ani_rt = 15), trilinear interpolation can again be performed using DP2 unit 318 (box 406), and the total number of input values ​​used is 32 (i.e., 1*16*2 = 32). It should be understood that Figure 4BThe input group shown is just one instance, and other instances may accumulate groups in different ways.

[0157] Figure 4C This illustrates a graphical representation of the sequence of operations performed when texture filtering involves volumetric filtering and anisotropy-4+ (e.g., where vol_en = 1 and ani_rt = 15, resulting in 2 * 16 * 1 = 32 inputs, but as...). Figure 4B In the middle, around Figure 4C The square brackets in the input transactions indicate those transactions that will not occur for anisotropy -4. In this example, volume interpolation is first performed using multiple consecutive operations of DP2 unit 318 (box 402, i.e., 16 passes through DP unit 318), and then, in this example, the anisotropic inputs (0-13) are accumulated as: ((((((0+1)+4)+6)+8)+10)+12)+((((((2+3)+5)+7)+9)+11)+13)) as described above. Figure 4C The accumulation order shown is merely an example and other orders can be used alternatively. The aiso_last inputs (14-15) are summed together (box 422, i.e., another pass through DP2 unit 318), and then DP2 unit 318 is used for anisotropic multiplication and the final addition (box 416). If trilinear filtering is also used (e.g., where vol_en = 1, tri_en = 1, and ani_rt = 15), trilinear interpolation is again performed using DP2 unit 318 (box 406), and the total number of input values ​​used is 64 (i.e., 2 * 16 * 2 = 64).

[0158] Input values ​​and / or intermediate values ​​are input to hardware logic unit 318 (e.g., to implement...). Figures 4A-4C The order of any of the texture filtering operations shown is controlled by control block 308. Each sequencer 312 operates in one of a set of predefined and hard-coded operation modes, where the selected mode defines a sequence of filtering operations applied to the input texture value stream or multiple interleaved input texture value streams. The sequencer's selection of the operation mode depends on enable signal 303 (which may be referred to as a control input or mode input). Each of the hard-coded operation modes can be defined in a microprogram, which may be non-programmable and synthesized into hardware, and may include a state machine (e.g., a finite state machine (FSM)) running on a stored function. The FSM defines a sequence of operations to be performed on a set of input data to generate the final output of the texture filtering operation.

[0159] like Figure 3As shown, control block 308 includes multiple sequencers 312 (e.g., four sequencers), and the sequencers 312 have overlapping activity, meaning that more than one sequencer 312 can simultaneously request access to hardware logic component 318. Therefore, the sequencers 312 indicate their status to arbitrator 314 (e.g., in the form of requests to access resources, which include hardware logic component 318 and may also include output 324). Arbitrator 314 grants access to resources (e.g., hardware logic component 318) according to a fixed priority mechanism (i.e., based on predefined rules). As described in more detail below, arbitrator 314 prioritizes operations involving new input values ​​(e.g., input 304 or input 306) and lowers the priority of requests to determine output values. Figure 3 As shown, the arbitrator 314 controls access to resources in the data path block by controlling the multiplexer 316 within the control block 308. The output from the multiplexer 316 includes a control signal that controls the operation of the source multiplexer 323 in the data path block 310, and thus controls the value input to the hardware logic unit 318 in any clock cycle.

[0160] Arbitrator 314 can be used in any sequence of operations (e.g., Figure 4A The boxes in the middle are 404 or 406. Figure 4B and 4C The order of output results is controlled by access to the hardware logic component 318 for the final DP2 operation, as controlled in block 416 or 406, and in various instances, operation can be prevented from starting if the output is not ready (via arbitrator 314). This means that no additional storage is required at output 324 (e.g., to hold more than one result). However, in other instances, a storage device can be provided before output 324, and the final result can be stored in said storage device and output in the order controlled by control block 308, or, as mentioned above, the output can be stored in temporary register 320 if it is not ready.

[0161] Figure 5 This is the second illustration of the instance texture filtering unit 300. Specifically, Figure 5 yes Figure 3 A more detailed illustration of the texturing filter unit 300 shown is provided. Figure 3 Compared to the example shown, Figure 5 Additional signal streams (e.g., signals that enable sequencers to indicate what inputs they want, which input is the last, when they are free, stop signals, etc.) and register levels are shown. Figure 5The control block 308 is also shown to include a main controller unit 502. In this example, the main controller 502 instructs any of the sequencers 312 when it can begin processing the filtering sequence (i.e., a new sequence to begin texture filtering operations). Additionally, the main controller unit 502 tracks the order in which the sequencers should produce their outputs, and thus ensures that the outputs are generated from the texture filtering unit 500 in the correct order.

[0162] exist Figure 3 and Figure 5 In the example shown, data path block 310 includes only a single DP2 unit 318. In other examples, data path block 310 may include different numbers and / or combinations of computational logic blocks. In many examples, resources within data path block 310 are limited, i.e., the total throughput of the hardware logic components of data path block 310 is lower than the total throughput that sequencer 312 can request.

[0163] As described above, the sequencer can operate on either a stream of input values ​​or an interleaved stream of input values. When using interleaving, it helps that individual filtered output input values ​​are not immediately received one after another. The use of the interleaved stream can be signaled to the texture filtering unit 300 using an additional enable (or control) signal 303:

[0164]

[0165] Interleaving input value streams (e.g., texture values) can be used in situations where multiple texture values ​​(e.g., R and G values) are accessed from memory simultaneously but require individual filtering (e.g., performing color filtering separately for each color channel). This can improve efficiency because texture values ​​from different streams (e.g., RGBA) can be stored contiguously, and by using interleaving, the need to deserialize the texture values ​​(which can be expensive for long sequences) is avoided.

[0166] In the above examples, the texture filtering unit 300 is arranged to implement one or more of the following: volumetric filtering, anisotropic filtering, and trilinear filtering. In other examples, the texture filtering unit can be arranged to implement any combination of volumetric filtering, anisotropic filtering, trilinear filtering, and bilinear filtering. To enable bilinear filtering without significantly reducing the final output rate, the number of inputs to the texture filtering unit configured to receive texture values ​​can be increased (e.g., inputs 304, 306), and the number of computational hardware logic units in the data path block can be increased.

[0167] Figure 6 This is a flowchart of a method for applying filtering to multiple input value sequences within the processing unit 300 to determine the output value.

[0168] In step S602, the texture filtering unit 300 receives a sequence of input values. This sequence can be received from the bilinear interpolation unit 112; that is, the input values ​​can be the result of bilinear interpolation already performed on the texture values ​​by the bilinear interpolation unit 112. For example, in volumetric filtering mode, the bilinear interpolation unit 112 can perform bilinear interpolation on texel values ​​2041 to 2044 to determine value 2061, and can perform bilinear interpolation on texel values ​​2045 to 2048 to determine value 2062, where values ​​2061 and 2062 are the input values ​​(304 and 306) provided to the texture filtering unit 300. As another example, in the trilinear filtering mode, the bilinear interpolation unit 112 can perform bilinear interpolation on the texel values ​​in the first mipmap level 2161 to determine value 2181, and can perform bilinear interpolation on the texel values ​​in the second mipmap level 2162 to determine value 2182, where values ​​2181 and 2182 are input values ​​(304 and 306) provided to the texture filtering unit 300. In other examples, the input values ​​304 and 306 of the texture filtering unit 300 may not have been processed by the bilinear interpolation unit 112; for example, input values ​​304 and 306 may be texel values ​​already retrieved from memory 108.

[0169] A sequence of input values ​​can be received as a set. Input values ​​within a set can be accessed simultaneously from memory. For example, Figure 7 Four distinct sets of instances of input value sequences are shown. In this instance, two input values ​​are received in the input loop, and... Figure 7 Each input value sequence shown comprises six input values, meaning it can be received in three input cycles. Each input cycle (i.e., each pair of input values) is represented by a box in the right column, where the number in the box indicates which input cycle corresponds to a particular input value sequence. Different types of shading are used to indicate different input value sequences. Figure 7 A first set 702 comprising one input value sequence, a second set 704 comprising two input value sequences, a third set 706 comprising three input value sequences, and a fourth set 708 comprising four input value sequences are shown. In each set containing more than one input value sequence (e.g., in sets 704, 706, and 708), the input loop alternates between different sequences, i.e., such that a first input loop (i.e., a first pair of input values) is received sequentially for each sequence, then a second input loop (i.e., a second pair of input values) is received sequentially for each sequence, and so on. In this way, the input values ​​within a set are interleaved input values ​​from the plurality of input value sequences. Each sequence may represent a corresponding color channel (e.g., red, green, blue, or alpha color channel) and may be processed by a corresponding one of the sequencers 312. The number of input value sequences within a set does not exceed the number of sequencers 312 in control block 308.

[0170] In step S604, control block 308 assigns each of the input value sequences to one of the sequencers 312. For example, the input value sequences may be assigned to the sequencers 312 in a round-robin manner. As described above, the sequencer 312 receives enable signals (vol_en, tri_en, and ani_rt) to specify the filtering to be performed. Each of the sequencers 312 defines a sequence of operations for the texture filtering process to be performed on the input value sequences assigned to the sequencer. Filters of the same type are applied to the input value sequences in the set. The sequencer also receives filter coefficients (vfrac, afrac, and tfrac) for the filtering operations described above.

[0171] In step S606, the arbitrator 314 controls the sequencer 312's access to the hardware logic component 318 of the data path block 310 according to the priority rules. Specifically, the arbitrator 314 selects which sequencer can access the hardware logic component 318.

[0172] In step S608, hardware logic unit 318 performs an operation by processing the values ​​used for the operation to generate a result of the operation, for example, as part of a texture filtering process. In each of a plurality of clock cycles, hardware logic unit 318 receives a plurality of values ​​used for the operation, and then processes the received values ​​through a pipeline within the plurality of clock cycles to generate the result of the operation as part of a texture filtering process. Different operations can be in different stages of the pipeline simultaneously. As described above, the result of the operation can be an intermediate result value, which can be stored in temporary register 320 for subsequent operations, or the result of the operation can be an output value, which can be stored in output register 504 and provided on output 324 of texture filtering unit 300. The "plural clock cycles" by which the hardware logic unit is configured to process the received values ​​used for the operation through a pipeline is a number of clock cycles (e.g., three) less than the number of sequencers (e.g., four) in the control block.

[0173] In prior art systems (e.g., as described in British Patent GB2567507B), there are two priority levels: high priority for input loops and low priority for non-input loops, and in the event of a tie, the sequencer with the lowest index is selected. However, the inventors of this application have recognized that having only two priority levels can sometimes lead to bubbles appearing in the work performed by the data path block; that is, this can result in loops where hardware logic unit 318 is not used by any sequencer 312, and can also lead to irregular performance results. As described in more detail below, according to this disclosure, arbitrator 314 uses three priority levels to determine which sequencers 312 can access hardware logic unit 318. The additional priority levels can avoid bubbles in the work performed by data path block 310 and also result in more regular performance results. It should be noted that this paper discusses performance and bubble occurrence based on the following three assumptions: (i) new inputs are always ready after a previous input is read (i.e., there is no input stoppage), and the next set of inputs is ready if all inputs in the set are received; (ii) outputs can be processed immediately (i.e., there is no output stoppage); and (iii) the filtering mode (i.e., the combination of volumetric filtering, anisotropic filtering, and trilinear filtering to be applied) and the number of sequences in the set remain constant over the inspection duration. Performance may be less than ideal if any of these assumptions are not met; however, these assumptions are expected to hold true over long periods when the component is active.

[0174] As described above, arbitrator 314 uses three priority levels (in addition to zero priority used when hardware logic component 318 is not needed). Specifically, each operation to be performed by hardware logic component 318 has a priority according to a prioriation rule, where:

[0175] The operations in the first group have high priority (hereinafter referred to as priority level '3'). The first group of operations includes input operations that involve input values. For example, all input operations (i.e., all operations that involve input values) can be included in the first group of operations and are therefore given high priority.

[0176] The operations in the second group have medium priority (hereinafter referred to as priority level '2'). The second group of operations includes intermediate operations that do not involve input values ​​and determine intermediate result values ​​rather than output values.

[0177] Operations in the third group have low priority (hereinafter referred to as priority level '1'). The third group of operations includes output operations that determine the output value. For example, all output operations that are not input operations (i.e., those that do not involve input values) can be included in the third group of operations and are therefore given low priority.

[0178] In some instances, the second set of operations may include all intermediate operations (i.e., all operations that do not involve the input value and determine the intermediate result value rather than the output value).

[0179] However, in the examples described in detail below, some intermediate operations are included in the third group of operations (and therefore have lower priority) instead of the second group. Specifically, in the examples described in detail below, the third group of operations includes all operations that do not involve input values ​​and accumulate values ​​derived from all previously received input values ​​in the operation sequence. If information from all input values ​​received up to that point in the sequence is needed to compute those operations, it can be considered that the operations accumulate values ​​derived from all previously received input values ​​used for the operation sequence. In this way, the third group of operations includes all output operations that are not input operations, and also includes the anisotropic dot product operation at the end of the first group of anisotropic filtering operations when a filtering mode involving trilinear filtering and anisotropic filtering with an anisotropy ratio above a threshold is implemented. For example, if volume filtering is being applied, the threshold could be two, and if no volume filtering is applied, the threshold could be four. This is referenced below. Figures 8A-8G The examples shown (especially) Figure 8E and Figure 8G The example shown (which is an example of applying trilinear filtering and anisotropic filtering with an anisotropy ratio above a threshold) will be explained in more detail.

[0180] Therefore, according to the prioritization rules applied by arbitrator 314, operations that produce output are given low priority (if they do not involve input values), most operations that do not produce output and do not require processing input values ​​are given medium priority, and operations that require input values ​​are given the highest priority. As explained in the following paragraphs, distinguishing between low-priority and medium-priority operations helps avoid bubbles in the work performed by data path block 310 and also results in more consistent performance outcomes.

[0181] Specifically, the final output DP2 operations from multiple operation sequences are saved until the end, allowing them to be executed together, thus filling each other's "wait loops." This is useful because wait loops exist precisely before the execution of the output operations of a sequence. This is because the operations that produce the input values ​​for the output operations need to be completed before the output operations can begin. For example, there can be at least two wait loops (and often more than two) before an output operation can be scheduled. While operations that produce the input values ​​for the output operations of a sequence are being executed, the sequence typically does not have any other operations to be performed. Therefore, to avoid bubbles, it is useful that other work to be performed exists in other sequences while operations that produce the input values ​​for the final output operations of the first sequence are being executed. A simple way to ensure this is to save the output loops from different sequences until the end, allowing them to be executed together, thus filling each other's bubbles. This also helps to keep the sequences synchronized as they are processed in the set, so that they complete together.

[0182] As described below Figures 8A-8G As seen in the example, all operation sequences accumulate one or more input values ​​in a loop into a single value, combining two input values ​​or intermediate values ​​at a time. These intermediate values ​​are stored in a finite number of "temporary" registers (e.g., five temporary registers for each operation sequence). The use of temporary registers allows for parallel processing of sequences with a large number of input loops, thus hiding the latency of hardware logic unit 318 between dependent operations within the operation sequence. For example, hardware logic unit 318 could be a DP2 unit with a latency of three clock cycles, meaning that DP unit 318 takes three clock cycles to perform one operation. Values ​​stored in registers are used only once by later operations and can then be overwritten with different values. The final operation in the sequence sends its result directly to the output of the hardware logic unit without going through the temporary registers. After the first four temporary registers have been filled (in the "start" phase), the values ​​from two registers are summed into one register to make room for the result of a new input loop (this occurs in the "MID" phase of the sequence). During these two phases, the work within the sequence can be parallelized enough to hide the delay of the three loops of the DP2 unit: by the time a later DP2 operation needs the DP2 result, it is already available in the register. After these phases, no more input loops are received.

[0183] In later stages of the operation sequence (referred to as the "End," "ANI," and "TRI" stages in the examples described below), the DP2 operations are sparser, and each value reduces the number of registers with unused values ​​until the last two values ​​are combined to produce the output. Each operation can only begin when the two operations that produced its input have completed, and there are not enough operations to execute in parallel to hide the delay of the DP2 unit. This means the sequence must accommodate multiple "wait" loops to ensure the data used is valid. Note that in filtering modes with large anisotropy ratios and trilinear filtering, the End and ANI stages occur twice, once when half of the input loop has been received and again at the end. There are multiple (e.g., four) sequencers 312 that can all run in parallel, so when one sequence in the sequence has a wait loop, the DP2 unit 318 is not unused: in fact, sequences running on different sequencers can fill the gaps in the sequence currently having a wait loop.

[0184] As long as there is no input or output stoppage, and the filtering mode and set size remain unchanged, the priority assigned to the operations as described above ensures that the DP2 unit is used in each loop. Specifically, if an input loop is ready to be processed, it always takes priority. However, each input loop belongs to only one sequence, so if the next sequence receiving input samples is not yet ready to receive them, an input stoppage may still be necessary. This means that only one sequencer can have this highest priority (priority '3') at a time. Most non-input loops will have a medium priority (priority '2'), meaning that it can be selected as long as no sequencer has priority 3. In the case of multiple sequencers requesting to process medium-priority operations, the first sequencer in the active set (the one receiving input) has the highest priority, followed by the next sequencer, then the next sequencer after that, and then the next sequencer after that (for sequencer 3, the next sequencer is sequencer 0). Low priority (priority '1') is assigned to sequencers that need to perform non-input operations that accumulate all previously received input values. These loops share a common feature: they occur at the end of the sparse portion of the sequence; in other words, the preceding loops contain many waiting loops that need to be filled with loops from other sequencers (in or out of the same set) to fully utilize the DP2 units for maximum performance. If there are three or four sequences in each set, these waiting loops can be filled with operations from other sequencers in the same set. If there are only one or two sequences in each set, the last few loops in that set (all with priority '2' or '1') are deferred when the next set begins processing its input and are finally executed when the sequence in the next set reaches the sparse section. If there is a high anisotropy ratio, this can be after receiving the first MIP MAP level input sample, or it can be at the end of the sequence. This takes advantage of the fact that there are no strict time delay requirements for operations, so the final operation can be deferred for a considerable period. The limitation of doing so is that at any given time, all sequencers are in use, and new inputs can only be processed by completing the old sequence and releasing its sequencer.

[0185] As described above, in a filtering pattern that includes trilinear filtering and anisotropic filtering with an anisotropy ratio above a threshold, each of the sequencers comprises a sequence of operations including: (i) a first set of anisotropic filtering operations at a first mipmap level ending with a dot product operation; (ii) a second set of anisotropic filtering operations at a second mipmap level; and (iii) a trilinear interpolation operation that combines the result of the first set of anisotropic filtering operations with the result of the second set of anisotropic filtering operations to determine the output value of the operation sequence. Examples of these filtering patterns are described below. Figure 8E and Figure 8GAs shown in the diagram. In these instances, the dot product operation at the end of the first group of anisotropic filtering operations is in the third group of operations (i.e., it is given low priority even though it is not an output operation, i.e., even though its output value is uncertain). In these filtering modes, all intermediate operations that are not dot product operations at the end of the first group of anisotropic filtering operations are in the second group of operations (i.e., they have medium priority). Giving low priority to the first anisotropic dot product operation in these filtering modes helps to synchronize the operation sequences from different sequencers at the operation points in the middle of the anisotropic + trilinear filtering operation sequence. This prevents the sequencers from becoming out of sync with each other, i.e., it prevents one sequencer from starting to process the latter half of its input while other sequencers are blocked in the first half. Desynchronizing in this way would harm the overall performance of the texture filtering unit. This is because if a sequence lags behind other sequences too much, it may be necessary to “catch up” when there are no other operations to hide its waiting loop, resulting in wasted loops of idle DP2 unit 318.

[0186] As described above, in step S606, the arbitrator 314 controls the sequencer 312's access to the hardware logic unit 318 of the data path block 310. The arbitrator 314 achieves this by receiving from the sequencer 312 a request identifying an operation to be performed by the hardware logic unit 318 of the data path block 310, and comparing the priorities of the operations identified by the request to determine which identified operation should be sent to the hardware logic unit 318 of the data path block 310 next. Specifically, the arbitrator 314 determines which identified operation has the highest priority among the identified operations. If it is determined that one of the identified operations has the highest priority, the arbitrator 314 causes that one of the identified operations to be sent to the hardware logic unit 318 of the data path block 310 next (by correspondingly controlling multiplexers 316, 323). If it is determined that multiple identified operations have the highest priority among the identified operations, the arbitrator 314 determines, according to the tie-breaking scheme, which of the multiple identified operations will be sent next to the hardware logic unit 318 of the data path block 310.

[0187] The tie-breaking mechanism sorts the sequencers starting with the base sequencer (e.g., in a round-robin fashion). Arbitrator 314 is configured to determine which of the plurality of identified operations, determined to have the highest priority among the identified operations, was requested by the sequencer, which, according to the sequencer order in the tie-breaking mechanism, is the first of the plurality of identified operations to request the highest priority. In this way, in the event of a tie in the priority of the DP2 operations ready to be sent to DP2 unit 318 for processing, arbitrator 314 starts with the base sequencer, where the "base sequencer" is the first sequencer in the set that still needs to receive input. The base sequencer is the sequencer among the sequencers 312 that has not yet sent all the input operations in its operation sequence for processing by the hardware logic unit 318 before all sequencers 312 have sent all the input operations in their operation sequences for processing by the hardware logic unit 318. In response to the determination that the final input operation of the sequence of operations requested by the sequencer currently acting as the base sequencer has been sent to the hardware logic unit 318, the arbitrator 314 updates the indication of which sequencer is currently the base sequencer, such that, if possible, the base sequencer is the sequencer that has not yet sent all input operations in its sequence of operations for processing by the hardware logic unit 318. As a result, when one sequencer has stopped having its inputs processed by the hardware logic unit, other sequencers can begin having their inputs processed by the hardware logic unit.

[0188] For example, each time a sequencer receives its final input cycle, the indicator indicating which sequencer is the base sequencer can be incremented by 1. For instance, the set could contain four sequences, assigned to sequencers 1, 2, 3, and 4. In this example, before the first sequencer receives its final input cycle, the 'base sequencer' is sequencer 1. When the first sequencer has received its final input cycle, the base sequencer is updated to sequencer 2. When the second sequencer has received its final input cycle, the base sequencer is updated to sequencer 3. When the third sequencer has received its final input cycle, the base sequencer is updated to sequencer 4. In this way, in the event of a tie, the operations of the base sequencer are prioritized, and when the 'base sequencer' changes, the sequencers that still need to receive input cycles (and it should be noted that these input cycles will have high priority) are prioritized, thus ensuring that all sequencers can quickly receive all their input cycles. Compared to using a fixed order of sequencers to resolve ties, i.e., prioritizing the sequence running on sequencer 0, then sequencer 1, then sequencer 2, then sequencer 3, using a basic sequencer as described above to resolve priority ties improves the performance of the texturing filter unit 300.

[0189] As described above, data path block 310 includes a set of temporary registers 320 for each sequencer, wherein the set of temporary registers for the sequencer is arranged to store intermediate result values ​​generated by hardware logic unit 318 when performing operations in the sequence of operations defined by the sequencer. Sequencer 312 determines a destination location in one of the temporary registers 320 for each intermediate result value generated by hardware logic unit 318 to store the intermediate result value. Intermediate result values ​​can be provided from the temporary registers 320 to hardware logic unit 318. Hardware logic unit 318 includes a first input and a second input, such that it can receive two values ​​in a single clock cycle. In each clock cycle, each input of the hardware logic unit can receive a value or an input value from one of the temporary registers. In previous systems (e.g., as described in British Patent GB2567507B), all temporary registers (e.g., 20 registers) could provide input data to both inputs of the hardware logic unit, and the system required multiplexing logic to allow all temporary registers to be connected to both inputs of the hardware logic unit.

[0190] The inventors of this application have recognized that multiplexing logic can occupy a significant area. It is desirable to reduce the area of ​​the texture filtering logic 300. Therefore, in the data path block 310, it is possible that each of the temporary registers can provide data to only one input of the hardware logic component 318. For example, this set of temporary registers 320 for the sequencer may include: (i) a first subgroup of temporary registers configured to provide values ​​to a first input of the hardware logic component but not to a second input; and (ii) a second subgroup of temporary registers configured to provide values ​​to a second input of the hardware logic component but not to a first input. This significantly reduces the area used for multiplexing; that is, it limits the physical connections required between the temporary registers and the hardware logic component. For example, the set of temporary registers 320 for each sequencer may include five temporary registers (R0, R1, R2, R3 and R4), and the first subgroup of the sequencer's temporary registers includes two temporary registers (e.g., registers R0 and R2), and the second subgroup of the sequencer's temporary registers includes three temporary registers (e.g., registers R1, R3 and R4).

[0191] Multiplexing logic 323 is configured to receive values ​​from temporary register 320, receive input values ​​(304, 306), and output values ​​to the first and second inputs of hardware logic unit 318. Arbitrator 314 uses multiplexing logic 323 to control which values ​​are provided to the first and second inputs of hardware logic unit 318 in each of multiple clock cycles.

[0192] The sequencer 312's FSM ensures that intermediate results are stored in appropriate registers so that they can be provided to the correct input to the hardware logic unit 318 when subsequently used. Specifically, the sequencer 312 is configured such that if two intermediate result values ​​will be used together in subsequent operations of the hardware logic unit, the first of the two intermediate result values ​​is stored in the temporary register of a first subgroup (e.g., R0 or R2), and the second of the two intermediate result values ​​is stored in the temporary register of a second subgroup (e.g., R1, R3, or R4).

[0193] Data path block 310 does not include dedicated addition units. For example, hardware logic unit 318 may be a two-dimensional dot product (DP2) unit 318, and data path block 310 may be configured to perform addition operations using DP2 units by setting the coefficients of the DP2 units to have a value of 1. While performing addition operations using DP2 units may be slightly less efficient than using dedicated addition units, the area of ​​texture filtering unit 300 can be significantly reduced by avoiding the inclusion of dedicated addition units in data path block 310. In particular, texture filtering unit 300 will only use dedicated addition units in filtering modes with high anisotropy ratios (4 or higher), which are of low performance concern because they are rarely used in practice. Therefore, the benefit achieved by not including dedicated addition units in terms of area reduction (e.g., a 10% reduction in the area of ​​texture filtering unit 300) outweighs the slight performance degradation in filtering modes with high anisotropy ratios (4 or higher).

[0194] In the above example, hardware logic unit 318 is a DP unit. More generally, hardware logic unit can be any module that performs the operation of combining two values ​​and outputting a value of the same format, such as a 2-input adder or multiplier.

[0195] Furthermore, the techniques described above in the context of texture filtering within the GPU (where the input values ​​are input texture values ​​and the output values ​​are output texture values) can also be used in other applications involving floating-point operations, which include evaluating a sum of multiple products (SOP) followed by an accumulation phase. In such instances, the texture filtering unit described above can alternatively be referred to as a computation unit, and the filter coefficients can alternatively be replaced by SOP coefficients. Examples of other applications include coordinate transformations, depth calculations, and some common shader computations.

[0196] Figures 8A-8G The example timing diagrams for different example filtering modes are shown, along with the registers used to store the results of the operations performed in the different filtering modes.

[0197] In particular, Figure 8AThe timing diagram is shown for a filtering mode that implements one of volumetric filtering (V), anisotropic filtering with an anisotropy ratio of 2 (A2), or trilinear filtering (T). In these filtering modes, the operation sequence contains only one input loop. Figure 8A Only one operation 802 is shown for the operation sequence. Operation 802 receives input values ​​(denoted as a0 and a1), and is therefore an 'input operation' with high priority (i.e., priority '3'). Operation 802 is scheduled in clock cycle 0, begins execution by DP2 unit 318 in clock cycle 1, and completes execution in clock cycle 3. Operation 802 is shown in the column 'Y' of the output register because the result of operation 802 is stored in the output register (this adds a delay cycle, shown as a line extending from the operation). 'X' shown for operation 802 indicates a volumetric filtering operation (in V filtering mode), an anisotropic filtering operation (in A2 filtering mode), or a trilinear filtering operation (in T filtering mode).

[0198] Figure 8B The timing diagram is shown for a filtering mode that implements either volumetric filtering and trilinear filtering (VT) or anisotropic filtering and trilinear filtering (A2T) with an anisotropy ratio of 2. In these filtering modes, the operation sequence contains two input loops. Figure 8B Three operations 804, 806, and 808 are shown for execution in the sequence of operations. Operations 804 and 806 receive input values, and are therefore 'input operations', with high priority (i.e., priority of '3'). Operation 804 is scheduled in clock cycle 0, begins execution by DP2 unit 318 in clock cycle 1, and completes execution in clock cycle 3, with its result stored in register R4. Operation 806 is scheduled in clock cycle 1, begins execution by DP2 unit 318 in clock cycle 2, and completes execution in clock cycle 4, with its result stored in register R0. Operation 808 receives values ​​determined by operations 804 and 806 from registers R4 and R0 (it should be noted that registers R0 and R4 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 808 does not involve any input values ​​and determines the output value, therefore it has low priority (i.e., priority of '1'). Operation 808 is scheduled in clock cycle 4, begins execution by DP2 unit 318 in clock cycle 5, and completes execution in clock cycle 7, with the result stored in output register Y.

[0199] Figure 8C The timing diagram is shown in a filtering mode that implements anisotropic filtering (A6) with an anisotropy ratio of 6. In this filtering mode, the operation sequence contains three input loops. Figure 8CFive operations 810 to 818 are shown for execution of the operation sequence. Operations 810, 812, and 814 receive input values ​​and are therefore 'input operations' with high priority (i.e., priority '3'). Operation 810 is scheduled in clock cycle 0, begins execution by DP2 unit 318 in clock cycle 1, and completes execution in clock cycle 3, with its result stored in register R0. Operation 812 is scheduled in clock cycle 1, begins execution by DP2 unit 318 in clock cycle 2, and completes execution in clock cycle 4, with its result stored in register R1. Operation 814 is scheduled in clock cycle 2, begins execution by DP2 unit 318 in clock cycle 3, and completes execution in clock cycle 5, with its result stored in register R3. Operation 816 receives the values ​​determined by operations 810 and 812 from registers R0 and R1 (it should be noted that registers R0 and R1 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 816 involves no input value and determines the output value, therefore it has medium priority (i.e., priority of '2'). Operation 816 is scheduled in clock cycle 4, begins execution by DP2 unit 318 in clock cycle 5, and completes execution in clock cycle 7, with its result stored in register R0. Operation 818 receives the values ​​determined by operations 816 and 814 from registers R0 and R3 (it should be noted that registers R0 and R3 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 818 involves no input value and determines the output value, therefore it has low priority (i.e., priority of '1'). Operation 818 is scheduled in clock cycle 7, begins execution by DP2 unit 318 in clock cycle 8, and completes execution in clock cycle 10, with its result stored in output register Y.

[0200] Figure 8D The timing diagram is shown for a filtering mode that implements volumetric filtering, anisotropic filtering with an anisotropy ratio of 2, and trilinear filtering (VA2T). In this filtering mode, the operation sequence contains four input loops. Figure 8DSeven operations 820 to 826 are shown for execution of the operation sequence. Operations 820, 821, 822, and 823 receive input values ​​and are therefore 'input operations', and have high priority (i.e., priority '3'). Operation 820 is a volume interpolation operation and is scheduled in clock cycle 0, begins execution by DP2 unit 318 in clock cycle 1, and completes execution in clock cycle 3, with its result stored in register R0. Operation 821 is a volume interpolation operation and is scheduled in clock cycle 1, begins execution by DP2 unit 318 in clock cycle 2, and completes execution in clock cycle 4, with its result stored in register R1. Operation 822 is a volume interpolation operation and is scheduled in clock cycle 2, begins execution by DP2 unit 318 in clock cycle 3, and completes execution in clock cycle 5, with its result stored in register R2. Operation 823 is a volume interpolation operation, scheduled in clock cycle 3, executed by DP2 unit 318 starting in clock cycle 4, and completed in clock cycle 6, with its result stored in register R3. Operation 824 is an anisotropic dot product operation, receiving values ​​determined by operations 820 and 821 from registers R0 and R1 (note that registers R0 and R1 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 824 does not involve any input values ​​and does not determine the output value, therefore it has medium priority (i.e., priority '2'). Operation 824 is scheduled in clock cycle 4, executed by DP2 unit 318 starting in clock cycle 5, and completed in clock cycle 7, with its result stored in register R4. Operation 825 is an anisotropic dot product operation, receiving values ​​determined by operations 822 and 823 from registers R2 and R3 (note that registers R2 and R3 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 825 involves no input value and determines the output value, therefore it has medium priority (i.e., priority of '2'). Operation 825 is scheduled in clock cycle 6, begins execution by DP2 unit 318 in clock cycle 7, and completes execution in clock cycle 9, with its result stored in register R0. Operation 826 is a trilinear interpolation operation and receives the values ​​determined by operations 824 and 825 from registers R4 and R0 (note that registers R4 and R0 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 826 involves no input value and determines the output value, therefore it has low priority (i.e., priority of '1'). Operation 826 is scheduled in clock cycle 9, begins execution by DP2 unit 318 in clock cycle 10, and completes execution in clock cycle 12, with its result stored in output register Y.

[0201] Figure 8EThe timing diagram is shown in a filtering mode that implements anisotropic filtering with an anisotropy ratio of 6 and trilinear filtering (A6T). It should be noted that... Figure 8E The example shown is similar to Figure 8C The example shown also includes a trilinear filter. In this filtering mode, the operation sequence contains six input loops. Figure 8E Eleven operations 830 to 840 are shown for execution of the operation sequence. Operations 830, 831, and 832 receive input values ​​and are therefore 'input operations' with high priority (i.e., priority '3'). Operation 830 is scheduled in clock cycle 0, begins execution by DP2 unit 318 in clock cycle 1, and completes execution in clock cycle 3, with its result stored in register R0. Operation 831 is scheduled in clock cycle 1, begins execution by DP2 unit 318 in clock cycle 2, and completes execution in clock cycle 4, with its result stored in register R1. Operation 832 is scheduled in clock cycle 2, begins execution by DP2 unit 318 in clock cycle 3, and completes execution in clock cycle 5, with its result stored in register R3. Operation 833 receives the values ​​determined by operations 830 and 831 from registers R0 and R1 (it should be noted that registers R0 and R1 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 833 involves no input value and has no predetermined output value, therefore it has medium priority (i.e., priority of '2'). Operation 833 is scheduled in clock cycle 4, begins execution by DP2 unit 318 in clock cycle 5, and completes execution in clock cycle 7, with its result stored in register R0. Operation 834 receives the values ​​determined by operations 832 and 833 from registers R3 and R0 (note that registers R0 and R3 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 834 involves no input value and is the first anisotropic dot product operation in a filtering mode that includes trilinear filtering and anisotropic filtering with an anisotropy ratio above a threshold, therefore it has low priority (i.e., priority of '1'). Operation 834 is scheduled in clock cycle 7, begins execution by DP2 unit 318 in clock cycle 8, and completes execution in clock cycle 10, with its result stored in register R4.

[0202] Operations 835, 836, and 837 receive input values ​​and are therefore 'input operations' with high priority (i.e., priority of '3'). Operation 835 is scheduled in clock cycle 8, begins execution by DP2 unit 318 in clock cycle 9, and completes execution in clock cycle 11, with its result stored in register R0. Operation 836 is scheduled in clock cycle 9, begins execution by DP2 unit 318 in clock cycle 10, and completes execution in clock cycle 12, with its result stored in register R1. Operation 837 is scheduled in clock cycle 10, begins execution by DP2 unit 318 in clock cycle 11, and completes execution in clock cycle 13, with its result stored in register R3. Operation 838 receives the values ​​determined by operations 835 and 836 from registers R0 and R1 (it should be noted that registers R0 and R1 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 838 involves no input value and has no predetermined output value, therefore it has medium priority (i.e., priority of '2'). Operation 838 is scheduled in clock cycle 12, begins execution by DP2 unit 318 in clock cycle 13, and completes execution in clock cycle 15, with its result stored in register R0. Operation 839 receives the values ​​determined by operations 837 and 838 from registers R3 and R0 (note that registers R0 and R3 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 839 involves no input value and has no predetermined output value, therefore it has medium priority (i.e., priority of '2'). Operation 839 is scheduled in clock cycle 15, begins execution by DP2 unit 318 in clock cycle 16, and completes execution in clock cycle 18, with its result stored in register R0. Operation 840 is a trilinear interpolation operation and receives the values ​​determined by operations 834 and 839 from registers R4 and R0 (note that registers R4 and R0 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 840 does not involve any input values ​​and determines the output value, therefore it has low priority (i.e., priority of '1'). Operation 840 is scheduled in clock cycle 18, begins execution by DP2 unit 318 in clock cycle 19, and completes execution in clock cycle 21, with its result stored in output register Y.

[0203] Figure 8F The timing diagram is shown for a filtering mode that implements anisotropic filtering (A8, A12, A16) with anisotropy ratios of 8, 12, or 16. In this filtering mode, the operation sequence contains four, six, or eight input loops. Figure 8FFifteen operations 850 to 864 are shown for execution of the operation sequence. Operations 850, 851, 852, and 853 receive input values ​​and are therefore 'input operations', and have high priority (i.e., priority of '3'). Operation 850 is an anisotropic addition operation, scheduled in clock cycle 0, executed by DP2 unit 318 starting in clock cycle 1, and completed in clock cycle 3, with its result stored in register R0. Operation 851 is an anisotropic addition operation, scheduled in clock cycle 1, executed by DP2 unit 318 starting in clock cycle 2, and completed in clock cycle 4, with its result stored in register R1. Operation 852 is an anisotropic addition operation, scheduled in clock cycle 2, executed by DP2 unit 318 starting in clock cycle 3, and completed in clock cycle 5, with its result stored in register R2. Operation 853 is an anisotropic addition operation, scheduled in clock cycle 3, executed by DP2 unit 318 starting in clock cycle 4, and completed in clock cycle 6, with the result stored in register R3. In the filtering mode implementing anisotropic filtering with an anisotropy ratio of 8, the process starts from... Figure 8F The clock cycle 4 shown jumps to clock cycle 13; while in the filtering mode that implements anisotropic filtering with an anisotropy ratio of 12 or 16, the process continues from clock cycle 4 to clock cycle 5.

[0204] Operation 854 is an anisotropic addition operation and receives values ​​determined by operations 850 and 851 from registers R0 and R1 (note that registers R0 and R1 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 854 does not involve any input value and does not determine the output value, therefore it has medium priority (i.e., priority of '2'). Operation 854 is scheduled in clock cycle 4, begins execution by DP2 unit 318 in clock cycle 5, and completes execution in clock cycle 7, with its result stored in register R0. Operation 855 receives an input value, therefore it is an input operation and has high priority (i.e., priority of '3'). Operation 855 is an anisotropic addition operation and is scheduled in clock cycle 5, begins execution by DP2 unit 318 in clock cycle 6, and completes execution in clock cycle 8, with its result stored in register R1. Operation 856 is an anisotropic addition operation and receives values ​​determined by operations 852 and 853 from registers R2 and R3 (note that registers R2 and R3 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 856 does not involve any input value and does not determine the output value, therefore it has medium priority (i.e., priority of '2'). Operation 856 is scheduled in clock cycle 6, begins execution by DP2 unit 318 in clock cycle 7, and completes execution in clock cycle 9, with its result stored in register R2. Operation 857 receives an input value, therefore it is an input operation and has high priority (i.e., priority of '3'). Operation 857 is an anisotropic addition operation and is scheduled in clock cycle 7, begins execution by DP2 unit 318 in clock cycle 8, and completes execution in clock cycle 10, with its result stored in register R3. In the filtering mode implementing anisotropic filtering with an anisotropy ratio of 12, the process starts from... Figure 8F The clock cycle 8 shown jumps to clock cycle 13; while in the filtering mode that implements anisotropic filtering with an anisotropy ratio of 16, the process continues from clock cycle 8 to clock cycle 9.

[0205] Operation 858 is an anisotropic addition operation and receives values ​​determined by operations 854 and 855 from registers R0 and R1 (note that registers R0 and R1 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 858 does not involve any input value and does not determine the output value, therefore it has medium priority (i.e., priority of '2'). Operation 858 is scheduled in clock cycle 8, begins execution by DP2 unit 318 in clock cycle 9, and completes execution in clock cycle 11, with its result stored in register R0. Operation 859 receives an input value, therefore it is an input operation and has high priority (i.e., priority of '3'). Operation 859 is an anisotropic addition operation and is scheduled in clock cycle 9, begins execution by DP2 unit 318 in clock cycle 10, and completes execution in clock cycle 12, with its result stored in register R1. Operation 860 is an anisotropic addition operation and receives values ​​determined by operations 856 and 857 from registers R2 and R3 (note that registers R2 and R3 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 860 does not involve any input value and does not determine the output value, therefore it has medium priority (i.e., priority of '2'). Operation 860 is scheduled in clock cycle 10, begins execution by DP2 unit 318 in clock cycle 11, and completes execution in clock cycle 13, with its result stored in register R2. Operation 861 receives an input value, therefore it is an input operation and has high priority (i.e., priority of '3'). Operation 861 is an anisotropic addition operation and is scheduled in clock cycle 11, begins execution by DP2 unit 318 in clock cycle 12, and completes execution in clock cycle 14, with its result stored in register R3.

[0206] Operation 862 is an anisotropic addition operation and receives values ​​from registers R0 and R1 (values ​​determined by operations 850 and 851 in anisotropy ratio of 8, values ​​determined by operations 854 and 855 in anisotropy ratio of 12, and values ​​determined by operations 858 and 859 in anisotropy ratio of 16). It should be noted that registers R0 and R1 can provide values ​​to different inputs of DP2 unit 318 as described above. Operation 862 does not involve any input values ​​and does not determine the output value, therefore it has medium priority (i.e., priority of '2'). Operation 862 is scheduled in clock cycle 12, begins execution by DP2 unit 318 in clock cycle 13, and completes execution in clock cycle 15, with its result stored in register R1. Operation 863 is an anisotropic addition operation and receives values ​​from registers R1 and R2 (where it should be noted that registers R1 and R2 can provide values ​​to different inputs of DP2 unit 318 as described above). The value from register R1 is determined by operation 862. The value from register R2 is determined by operation 852 in the anisotropy ratio mode of 8, by operation 856 in the anisotropy ratio mode of 12, and by operation 860 in the anisotropy ratio mode of 16. Operation 863 does not involve any input value and does not determine the output value, therefore it has medium priority (i.e., priority '2'). Operation 863 is scheduled in clock cycle 15, begins execution by DP2 unit 318 in clock cycle 16, and completes execution in clock cycle 18, with its result stored in register R0. Operation 864 is an anisotropic dot product operation and receives values ​​from registers R0 and R3 (where it should be noted that registers R0 and R3 can provide values ​​to different inputs of DP2 unit 318 as described above). The value from register R0 is determined by operation 863. The value from register R3 is determined by operation 853 in the anisotropy ratio mode of 8, by operation 857 in the anisotropy ratio mode of 12, and by operation 861 in the anisotropy ratio mode of 16. Operation 864 does not involve any input value and determines the output value, therefore it has low priority (i.e., priority of '1'). Operation 864 is scheduled in clock cycle 18, begins execution by DP2 unit 318 in clock cycle 19, and completes execution in clock cycle 21, with its result stored in output register Y.

[0207] It is worth noting that the result of operation 862 is stored in register R1 (not register R0). This is because the next operation (i.e., operation 863) needs to obtain this intermediate result along with the intermediate result in register R2. Since R0 and R2 are connected to the same input of DP2 unit 318, operation 863 would not be able to receive the value it needs if the result of operation 862 were already stored in register R0. This is resolved by storing the result of operation 862 in register R1, allowing operation 863 to receive the value it needs, since registers R1 and R2 are connected to different inputs of DP2 unit 318.

[0208] Figure 8G The timing diagram is shown for a filtering mode that implements trilinear filtering and anisotropic filtering with anisotropy ratios of 8, 12, or 16 (A8T, A12T, A16T). It should be noted that... Figure 8G The example shown is similar to Figure 8F The example shown also includes a trilinear filter. In this filtering mode, the operation sequence contains eight, twelve, or sixteen input loops. Figure 8G The diagram shows 31 operations, 868 to 898, performed on the sequence of operations. For the sake of brevity, this document does not provide... Figure 8G A complete description of all operations shown. It should be noted that operations 868 to 882 involve the first half of the trilinear filtering, which is performed for the first mipmap level. Operations 868 to 881 are related to... Figure 8F Operations 850 through 863 shown are identical. Operation 882 is an anisotropic dot product operation and receives values ​​from registers R0 and R3 (note that registers R0 and R3 can provide values ​​to different inputs of DP2 unit 318 as described above). The value from register R0 is determined by operation 881. The value from register R3 is determined by operation 871 in the anisotropy ratio mode of 8, by operation 875 in the anisotropy ratio mode of 12, and by operation 879 in the anisotropy ratio mode of 16. Operation 882 does not involve any input values ​​and is the first anisotropic dot product operation in a filtering mode that includes trilinear filtering and anisotropic filtering with an anisotropy ratio above a threshold, therefore it has a low priority (i.e., priority of '1'). Operation 882 is scheduled in clock cycle 18, begins execution by DP2 unit 318 in clock cycle 19, and completes execution in clock cycle 21, with its result stored in register R4.

[0209] Operations 883 to 897 involve the latter part of the trilinear filtering, which is performed at the second mipmap level. Operations 883 to 896 and... Figure 8F The operations shown in 850 to 863 are the same (and similar to) Figure 8GOperations 868 to 881 shown are the same. Operation 897 is an anisotropic dot product operation and receives values ​​from registers R0 and R3 (note that registers R0 and R3 can provide values ​​to different inputs of DP2 unit 318 as described above). The value from register R0 is determined by operation 896. The value from register R3 is determined by operation 886 in the anisotropy ratio of 8, by operation 890 in the anisotropy ratio of 12, and by operation 894 in the anisotropy ratio of 16. Operation 897 does not involve any input value and does not determine the output value, therefore it has a medium priority (i.e., priority '2'). Operation 897 is scheduled in clock cycle 37, begins execution by DP2 unit 318 in clock cycle 38, and completes execution in clock cycle 40, with its result stored in register R0. Operation 898 is a trilinear interpolation operation and receives the values ​​determined by operations 882 and 897 from registers R4 and R0 (note that registers R4 and R0 can provide values ​​to different inputs of DP2 unit 318 as described above). Operation 898 does not involve any input values ​​and determines the output value, therefore it has low priority (i.e., priority of '1'). Operation 898 is scheduled in clock cycle 40, begins execution by DP2 unit 318 in clock cycle 41, and completes execution in clock cycle 43, with its result stored in output register Y.

[0210] Figures 8A-8G The timing diagrams shown only illustrate some example filtering patterns that can be implemented. Other filtering patterns can be used in other instances.

[0211] The examples described in detail herein relate to a texture filtering unit configured to perform a texture filtering process. However, it should be understood that a texture filtering unit is an instance of a filtering unit, and a texture filtering process is an instance of a filtering process, and in other instances, the same principles described herein in the context of texture filtering can be applied to other types of filtering. For example, the techniques described herein can be applied to image filtering, such as for blurring, sharpening, and noise reduction. Image filtering often involves applying a weighted sum to a set of pixel values ​​(e.g., color values), which is very similar to the anisotropic pattern described above regarding texture filtering, but with different weights. Another instance of image filtering where the techniques described herein can be applied is in convolutional kernels, for example, as part of a convolutional neural network. This again involves applying a weighted average to pixel values ​​(e.g., color values), but here as part of a neural network. The techniques described herein can also be applied to “non-image” filtering, such as any other linear filtering (i.e., filtering that can be represented as a weighted sum). For example, the Fast Fourier Transform (FFT) belongs to this category, which is useful for many applications, such as (typically) time series from audio signals, which can be transformed into frequency representations, and the techniques described in this paper can also be used in these instances.

[0212] Furthermore, although the examples described in detail herein involve implementing texture filtering within a rasterization system, the same texture filtering techniques can be implemented in other rendering systems, such as ray tracing systems. For example, in a ray tracing system, once it has been found that a ray intersects with a primitive at an intersection point, the texture can then be sampled and filtered as described herein to determine the color value at the intersection.

[0213] Figure 9 A computer system in which the graphics processing system described herein can be implemented is shown. The computer system includes a CPU 902, a GPU 904, memory 906, a neural network accelerator (NNA) 908, and other devices 914, such as a display 916, a speaker 918, and a camera 922. GPU 904 may correspond to GPU 102, and memory 906 may correspond to memory 108. A processing block 910 (which may include a texture filtering unit 300) is implemented on GPU 904. In other instances, one or more of the depicted components may be omitted from the system, and / or processing block 910 may be implemented on CPU 902 or within NNA 908. Components of the computer system can communicate with each other via a communication bus 920. Storage 912 (corresponding to the aforementioned cache) is implemented as part of memory 906.

[0214] Figure 1 , Figure 3 and Figure 5The processing unit is shown as comprising multiple functional blocks. This is merely illustrative and not intended to define a strict division between different logical elements of such an entity. Each functional block may be provided in any suitable manner. It should be understood that the intermediate values ​​described herein formed by the processing unit do not need to be physically generated by the processing unit at any point in time, and may only represent logical values ​​that conveniently describe the processing performed by the processing unit between its inputs and outputs.

[0215] The processing units (e.g., GPUs) described herein may be implemented in hardware on integrated circuits. The processing units described herein may be configured to perform any of the methods described herein. Generally, any of the functions, methods, techniques, or components described above may be implemented in software, firmware, hardware (e.g., fixed logic circuitry), or any combination thereof. The terms “module,” “function,” “component,” “element,” “unit,” “block,” and “logic” may be used herein to generally denote software, firmware, hardware, or any combination thereof. In the case of a software implementation, a module, function, component, element, unit, block, or logic represents program code that, when executed on a processor, performs a specified task. The algorithms and methods described herein may be executed by one or more processors that execute code that causes the processor to perform the algorithm / method. Examples of computer-readable storage media include random access memory (RAM), read-only memory (ROM), optical disks, flash memory, hard disks, and other memory devices that may use magnetic, optical, and other techniques to store instructions or other data and that are accessible by a machine.

[0216] As used herein, the terms computer program code and computer-readable instructions refer to any kind of executable code for a processor, including code expressed in machine language, interpreted language, or scripting language. Executable code includes binary code, machine code, bytecode, code defining integrated circuits (e.g., hardware description languages ​​or netlists), and code expressed in programming languages ​​such as C, Java, or OpenCL. Executable code can be, for example, any kind of software, firmware, script, module, or library that, when properly executed, processed, interpreted, compiled, or run in a virtual machine or other software environment, causes the processor of a computer system that supports the executable code to perform tasks specified by said code.

[0217] A processor, computer, or computer system can be any kind of device, machine, or special-purpose circuit, or a collection or part thereof, having processing capabilities that enable it to execute instructions. A processor can be or includes any kind of general-purpose or special-purpose processor, such as a CPU, GPU, NNA, system-on-a-chip, state machine, media processor, application-specific integrated circuit (ASIC), programmable logic array, field-programmable gate array (FPGA), etc. A computer or computer system may contain one or more processors.

[0218] This invention is also intended to cover software that defines the configuration of hardware as described herein, such as HDL (Hardware Description Language) software, for example, for designing integrated circuits or configuring programmable chips to perform desired functions. That is, a computer-readable storage medium on which computer-readable program code in the form of an integrated circuit definition dataset is provided, which, when processed (i.e., executed) in an integrated circuit manufacturing system, configures the system to manufacture processing units configured to perform any of the methods described herein, or to manufacture processing units comprising any of the devices described herein. The integrated circuit definition dataset may, for example, be an integrated circuit description.

[0219] Therefore, a method for manufacturing processing units as described herein can be provided at an integrated circuit manufacturing system. Furthermore, an integrated circuit definition dataset can be provided, which, when processed in the integrated circuit manufacturing system, enables the method for manufacturing the processing units to be executed.

[0220] Integrated circuit definition datasets can be in the form of computer code, such as netlists, code for configuring programmable chips, hardware description languages ​​defining hardware suitable for manufacturing at any level in integrated circuits, including register-transfer level (RTL) code, high-level circuit representations (such as Verilog or VHDL), and low-level circuit representations (such as OASIS(RTM) and GDSII). Higher-level representations (e.g., RTL) that logically define hardware suitable for manufacturing in integrated circuits can be processed at a computer system configured to generate manufacturing definitions of integrated circuits within the context of a software environment containing definitions of circuit elements and rules for combining those elements to generate the manufacturing definition of an integrated circuit so defined by that representation. As is typically the case where software executes at a computer system to define the machine, one or more intermediate user steps (e.g., providing commands, variables, etc.) may be required to configure the computer system to generate the manufacturing definition of the integrated circuit, executing code that defines the integrated circuit to generate the manufacturing definition of the integrated circuit.

[0221] Now refer to Figure 10Describe an instance in an integrated circuit manufacturing system that processes an integrated circuit definition dataset in order to configure the system to manufacture a processing unit (e.g., a GPU).

[0222] Figure 10 An example of an integrated circuit (IC) manufacturing system 1002 is illustrated, which is configured to manufacture processing units as described in any example herein. Specifically, the IC manufacturing system 1002 includes a layout processing system 1004 and an integrated circuit generation system 1006. The IC manufacturing system 1002 is configured to receive an IC definition dataset (e.g., defining processing units as described in any example herein), process the IC definition dataset, and generate an IC (e.g., embodying the processing units as described in any example herein) based on the IC definition dataset. The processing of the IC definition dataset configures the IC manufacturing system 1002 to manufacture integrated circuits embodying the processing units as described in any example herein.

[0223] The layout processing system 1004 is configured to receive and process an IC definition dataset to determine a circuit layout. Methods for determining a circuit layout based on an IC definition dataset are known in the art and may involve, for example, synthesizing RTL code to determine a gate-level representation of the circuit to be generated, for example, in relation to logic components (e.g., NAND, NOR, AND, OR, MUX, and FLIP-FLOP components). By determining the location information of the logic components, the circuit layout can be determined from the gate-level representation of the circuit. This can be done automatically or with user intervention to optimize the circuit layout. Once the layout processing system 1004 has determined the circuit layout, it can output the circuit layout definition to the IC generation system 1006. The circuit layout definition may be, for example, a circuit layout description.

[0224] As is known in the art, IC generation system 1006 generates ICs according to a circuit layout definition. For example, IC generation system 1006 may implement a semiconductor device manufacturing process for generating ICs, which may involve a multi-step sequence of photolithography and chemical processing steps, during which electronic circuits are gradually formed on a wafer made of semiconductor material. The circuit layout definition may be in the form of a mask, which can be used in the photolithography process to generate ICs according to the circuit definition. Alternatively, the circuit layout definition provided to IC generation system 1006 may be in the form of computer-readable code, which IC generation system 1006 can use to form a suitable mask for generating ICs.

[0225] The various processes performed by the IC manufacturing system 1002 can all be implemented in one location, for example, by one party. Alternatively, the IC manufacturing system 1002 can be a distributed system, allowing some processes to be performed in different locations and by different parties. For example, some of the following stages can be performed in different locations and / or by different parties: (i) synthesizing RTL code representing an IC definition dataset to form a gate-level representation of the circuit to be generated; (ii) generating a circuit layout based on the gate-level representation; (iii) forming a mask based on the circuit layout; and (iv) using the mask to manufacture the integrated circuit.

[0226] In other examples, processing of an integrated circuit definition dataset at an integrated circuit manufacturing system can configure the system to manufacture processing units without processing the IC definition dataset to determine circuit layout. For example, an integrated circuit definition dataset can define the configuration of a reconfigurable processor such as an FPGA, and processing of the dataset can configure the IC manufacturing system (e.g., by loading configuration data into the FPGA) to generate a reconfigurable processor with that defined configuration.

[0227] In some embodiments, when processed in an integrated circuit manufacturing system, an integrated circuit manufacturing definition dataset can enable the integrated circuit manufacturing system to generate apparatuses as described herein. For example, using an integrated circuit manufacturing definition dataset, as referenced above... Figure 10 The described method of configuring an integrated circuit manufacturing system can produce devices as described herein.

[0228] In some instances, an integrated circuit definition dataset may include software running on hardware defined at the dataset, or software running in combination with hardware defined at the dataset. Figure 10 In the example shown, the IC production system can be further configured by the integrated circuit definition dataset to load firmware onto the integrated circuit according to the program code defined at the integrated circuit definition dataset during the manufacturing of the integrated circuit, or otherwise provide the integrated circuit with program code for use with the integrated circuit.

[0229] Compared to known implementations, the concepts set forth in this application can improve performance in the implementation of devices, apparatuses, modules, and / or systems (and in the methods implemented herein). Performance improvements may include one or more of increased computational performance, reduced latency, increased throughput, and / or reduced power consumption. During the manufacture of such devices, apparatuses, modules, and systems (e.g., in integrated circuits), trade-offs can be made between performance improvements and physical implementation methods, thereby improving the manufacturing process. For example, a trade-off can be made between performance improvements and layout area to match the performance of known implementations but using less silicon. This can be accomplished, for example, by reusing functional blocks serially or sharing functional blocks among elements of a device, apparatus, module, and / or system. Conversely, the concepts set forth in this application that cause improvements in the physical implementation of devices, apparatuses, modules, and systems (e.g., reduced silicon area) can be traded against performance improvements. This can be accomplished, for example, by manufacturing multiple instances of modules within a predefined area budget.

[0230] The applicant has independently disclosed each individual feature described herein, as well as any combination of two or more such features, to the extent that such features or combinations can be implemented based on the specification as a whole, in view of the common knowledge of those skilled in the art, regardless of whether such features or combinations of features solve any problem disclosed herein. In light of the foregoing description, those skilled in the art will understand that various modifications can be made within the scope of this invention.

Claims

1. A processing unit, the processing unit comprising a filtering unit configured to apply filtering to a plurality of input value sequences to determine an output value, the filtering unit comprising: A control block comprising a plurality of sequencers and an arbitrator, wherein the control block is configured to assign each of the input value sequence to one of the sequencers, and wherein each of the sequencers is configured to define a sequence of operations for a filtering process to be performed on the input value sequence assigned to the sequencer. as well as A data path block, the data path block including hardware logic components configured to perform the operation by processing the value of the operation to generate the result of the operation as part of the filtering process; The arbitrator is configured to control the sequencer's access to the hardware logic components of the data path block according to a prioritization rule, wherein each operation performed by the hardware logic components has a priority according to the prioritization rule, wherein: The operations in the first group of operations have high priority, and the first group of operations includes input operations involving input values; The operations in the second group of operations have medium priority, and the second group of operations includes intermediate operations that do not involve input values ​​and determine intermediate result values ​​rather than output values. and The operations in the third group of operations have low priority, and the third group of operations includes output operations that determine the output value.

2. The processing unit according to claim 1, wherein the hardware logic component is configured to: In each of the multiple clock cycles, multiple values ​​are received for operation; and The received values ​​are processed through a pipeline within multiple clock cycles to generate the result of the operation as part of the filtering process.

3. The processing unit according to any of the preceding claims, wherein the first group of operations includes all input operations, and wherein the third group of operations includes all output operations, wherein the output operations are not input operations.

4. The processing unit according to any of the preceding claims, wherein the third set of operations includes all operations that do not involve input values ​​and accumulate values ​​derived from all previously received input values ​​in the operation sequence.

5. The processing unit according to any of the preceding claims, wherein the second set of operations includes all operations that do not involve input values ​​and do not accumulate values ​​derived from all previously received input values ​​in the operation sequence.

6. The processing unit according to any preceding claim, wherein each of the sequencers is configured to define an operation sequence in a filtering mode comprising anisotropic filtering and trilinear filtering, the operation sequence comprising: (i) a first group of anisotropic filtering operations ending with a dot product operation, (ii) a second group of anisotropic filtering operations, and (iii) a trilinear interpolation operation that combines the results of the first group of anisotropic filtering operations with the results of the second group of anisotropic filtering operations to determine the output value of the operation sequence. The dot product operation at the end of the first group of anisotropic filtering operations is in the third group of operations, and The second group of operations includes all intermediate operations of the dot product operation that are not at the end of the first group of anisotropic filtering operations in the filtering mode that includes anisotropic filtering and trilinear filtering.

7. The processing unit according to any preceding claim, wherein the arbitrator is configured to control the sequencer's access to the hardware logic components of the data path block by: Receive from the sequencer a request to identify the operation to be performed by the hardware logic component of the data path block; and The priorities of the operations identified by the request are compared to determine which of the identified operations should be sent to the hardware logic component of the data path block next.

8. The processing unit of claim 7, wherein the arbitrator is configured to control the sequencer's access to the hardware logic components of the data path block by: Determine which of the identified operations has the highest priority among the identified operations; If it is determined that one of the identified operations has the highest priority among the identified operations, then the one of the identified operations is subsequently sent to the hardware logic component of the data path block; and If it is determined that a plurality of the identified operations have the highest priority among the identified operations, then a tie-breaking mechanism is used to determine which of the plurality of the identified operations will be sent next to the hardware logic component of the data path block.

9. The processing unit of claim 8, wherein the tie-breaking mechanism sorts the sequencers starting from the base sequencer, wherein the arbiter is configured to determine which of the plurality of identified operations, determined to have the highest priority of the identified operation, was requested by the sequencer, the sequencer being the first sequencer among the plurality of identified operations that requested to be determined to have the highest priority, according to the ordering of the sequencers in the tie-breaking mechanism.

10. The processing unit of claim 9, wherein the base sequencer is one of the sequencers that has not yet sent all of the input operations in its operation sequence for processing by the hardware logic unit, prior to all of the sequencers having sent all of the input operations in its operation sequence for processing by the hardware logic unit. The arbitrator is configured to update the indication of which of the sequencers is currently the base sequencer in response to determining that the final input operation of the sequencer requesting the operation sequence has been sent to the hardware logic unit, such that, if possible, the base sequencer is the sequencer that has not yet sent all of the input operations in its operation sequence for processing by the hardware logic unit.

11. The processing unit according to any of the preceding claims, wherein the data path block further includes a set of temporary registers for each of the sequencers, wherein the set of temporary registers for the sequencers is arranged to store intermediate result values ​​generated by the hardware logic components when an operation in the sequence of operations defined by the sequencers is performed.

12. The processing unit according to claim 11, wherein the hardware logic component includes a first input and a second input; The set of temporary registers used for the sequencer includes: The first subgroup of temporary registers is configured to provide values ​​to the first input of the hardware logic component instead of the second input of the hardware logic component; as well as The second subgroup of temporary registers is configured to provide values ​​to the second input of the hardware logic component instead of the first input of the hardware logic component.

13. The processing unit according to claim 12, wherein the data path block includes multiplexing logic, and the multiplexing logic is configured to: Receive a value from the temporary register; Receive input values; and The value is output to the first and second inputs of the hardware logic component. Furthermore, the arbitrator is configured to use the multiplexing logic to control which values ​​are provided to the first and second inputs of the hardware logic component in each of a plurality of clock cycles.

14. The processing unit of claim 12 or 13, wherein the sequencer is configured to determine a destination location in one of the temporary registers for storing each intermediate result value generated by the hardware logic component, and wherein the sequencer is configured such that if two intermediate result values ​​are to be used together in subsequent operations of the hardware logic component, the first of the two intermediate result values ​​is stored in a temporary register in the temporary register of the first subgroup, and the second of the two intermediate result values ​​is stored in a temporary register in the temporary register of the second subgroup.

15. The processing unit according to any of the preceding claims, wherein the hardware logic component is a two-dimensional dot product unit, wherein the data path block does not include a dedicated addition unit, and wherein the data path block is configured to perform an addition operation using the two-dimensional dot product unit by setting the coefficients of the two-dimensional dot product unit to have a value of 1.

16. The processing unit according to any preceding claim, wherein the filtering unit is configured to apply filtering in a filtering mode including one or more of volumetric filtering, anisotropic filtering, and trilinear filtering, wherein the filtering unit is further configured to: Receive an indication of which of the following filters—volume filtering, anisotropic filtering, and trilinear filtering—will be applied; and Receive one or more filter coefficients for applying the indicated filter.

17. The processing unit according to any preceding claim, wherein the filtering unit is arranged to receive the plurality of input value sequences as a set of input values, wherein the input values ​​in the set are interleaved input values ​​from the plurality of input value sequences, and wherein the input values ​​in the set are accessed simultaneously from memory.

18. A method for applying filtering to a sequence of multiple input values ​​within a processing unit to determine an output value, wherein the processing unit includes a filtering unit, the filtering unit including a plurality of sequencers and a data path block, the data path block including hardware logic components, the method comprising: Each of the input value sequences is assigned to one of the sequencers, wherein each sequencer defines a sequence of operations for a filtering process to be performed on the input value sequence assigned to the sequencer; The sequencer's access to the hardware logic components of the data path block is controlled according to the prioritization rules. The operation is performed at the hardware logic component by processing the value of the operation to generate the result of the operation as part of the filtering process; Each operation to be performed by the hardware logic component has a priority according to the prioritization rule, wherein: The operations in the first group of operations have high priority, and the first group of operations includes input operations involving input values; The operations in the second group of operations have medium priority, and the second group of operations includes intermediate operations that do not involve input values ​​and determine intermediate result values ​​rather than output values. and The operations in the third group of operations have low priority, and the third group of operations includes output operations that determine the output value.

19. A computer-readable storage medium storing computer-readable code configured to cause the method of claim 18 to be executed when the code is run.

20. A computer-readable storage medium storing an integrated circuit definition dataset thereon, wherein when the integrated circuit definition dataset is processed in an integrated circuit manufacturing system, the integrated circuit manufacturing system is configured to manufacture a processing unit according to any one of claims 1 to 17.

Images