Circuit and Method

Abstract

The circuit includes an instruction decoder circuit that decodes instructions for execution, a processing circuit that executes the instructions decoded by the instruction decoder circuit, and an interface circuit that defines an interface for data communication with a data compression circuit. The processing circuit, in response to one or more instructions of an instruction set defined for the processing circuit, provides to the interface input data to be processed by the data compression circuit and identification data that identifies a compression method to be used by the data compression circuit for processing the input data. The processing circuit receives from the interface status data indicating whether the data compression circuit connected to the interface can process data using the compression method identified by the identification data, and when the status data indicates that the data compression circuit can process data using the compression method identified by the identification data, receives output data processed from the input data by the data compression circuit using the compression method identified by the identification data.

Description

Technical Field

[0001] The present disclosure relates to a data processing apparatus, method, and virtual machine.

[0002] Some data processing configurations enable processing operations, such as neural processing, to be performed using a set of processing values, such as weights or activation values.

Summary of the Invention

[0003] In an exemplary configuration, a circuit comprising: an instruction decoder circuit that decodes instructions for execution; a processing circuit that executes the instructions decoded by the instruction decoder circuit; an interface circuit that defines an interface for data communication with a data compression circuit, wherein in response to one or more instructions of an instruction set defined for the processing circuit, the processing circuit provides to the interface input data to be processed by the data compression circuit and identification data identifying a compression method to be used by the data compression circuit for processing the input data; the processing circuit receives from the interface status data indicating whether the data compression circuit connected to the interface can process data using the compression method identified by the identification data, and output data processed from the input data by the data compression circuit using the compression method identified by the identification data when the status data indicates that the data compression circuit can process data using the compression method identified by the identification data.

[0004] In another exemplary configuration, a method comprising: decoding instructions for execution; using a processing circuit to execute the instructions decoded in the instruction decoding step; defining an interface for data communication with a data compression circuit; In response to one or more instructions of an instruction set defined for a processing circuit, the processing circuit provides, to an interface, input data processed by a data compression circuit and identification data for identifying a compression method to be used by the data compression circuit to process the input data. The processing circuit receives, from the interface, status data indicating whether a data compression circuit connected to the interface can process data using the compression method identified by the identification data, and when the status data indicates that the data compression circuit can process data using the compression method identified by the identification data, the processing circuit receives output data processed from the input data by the data compression circuit using the compression method identified by the identification data. A method is provided that includes these steps.

[0005] Each further aspect and feature of the present disclosure is defined by the appended claims.

Brief Description of the Drawings

[0006] With reference to those embodiments shown in the accompanying drawings, the present technology will be further described by way of example only.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Modes for Carrying Out the Invention

[0007] Outline of the Processor Referring now to the drawings, FIG. 1 schematically shows a data processing system 10 comprising a processor 20 coupled to a memory 30 that stores data values 32 and program (or processing) instructions 34. The processor 20 includes an instruction fetch unit 40 for fetching program instructions 34 from the memory 30 and supplying the fetched program instructions to a decoder circuit 50. The decoder circuit 50 decodes the fetched program instructions and generates control signals to control a processing circuit 60 to perform processing operations on registers stored in a register circuit 70 as specified by the decoded vector instructions.

[0008] The processor 20 can optionally access a storage array 90. This is drawn as a dashed line, indicating that it may or may not be provided as part of the processor 20. In an exemplary embodiment, the storage array 90 may store a larger or higher-dimensional array of data items in the memory or a square array portion of a matrix. The storage array can, in at least some examples, be regarded as an accumulation array called "ZA" in a so-called Scalable Matrix Extension (SME) system provided or specified by Arm Limited.

[0009] In at least some examples, ZA is implemented as an n×m (square or rectangular) array of storage (or accumulation) elements. In some examples, n and m may be the same and may be equal to the streaming vector length used within a system such as SVL, or in other words, as defined in a so-called Scalable Vector Extension (SVE) or SVE2 system provided or specified by Arm Limited.

[0010] The SME instruction may refer to various types of matrix operands, including "tiles" that represent a subset of ZA. In some examples, a tile is itself a square array, although this is not a requirement, and (for example) rectangular tiles can be used. So-called "tile vectors" represent the rows or columns of ZA. An operand called the "accumulator matrix" refers to the entirety of ZA.

[0011] Processing circuit 60 may provide or include vector and / or matrix processing circuits. The general distinction between scalar processing and vector processing is as follows. Vector processing involves applying a single vector processing instruction to the data items of a data vector that has a plurality of data items at each position within the data vector. Scalar processing effectively performs an operation on a single data item rather than a data vector. Vector processing can be useful when the processing operation is to be executed on many different instances of the data being processed. In a vector processing configuration, a single instruction can be applied simultaneously to a plurality of data items (of the data vector). This can improve the efficiency and throughput of data processing compared to scalar processing.

[0012] The processing circuit may be used to perform operations on matrices. Here, a matrix can be regarded as an array of matrix elements. The array may be two-dimensional or may have a higher dimension.

[0013] This embodiment may be related to vector and / or matrix processing, although it is not a requirement that a vector or matrix processor be used.

[0014] The following description relates to exemplary program instruction 34. Embodiments of the present disclosure include, for example, an apparatus of the type shown in FIG. 1 and are operable or configured to decode and execute such program instructions.

[0015] The register circuit 70 provides a set of physical registers that can be allocated to architectural registers for the execution of processing instructions. The architectural registers are defined by the processor architecture and its instruction set architecture (ISA). Instructions will define one or more architectural registers that hold source or destination (output) operands, but in actual execution, these architectural registers will be implemented by respective physical registers 70. When the registers are vector registers, they may be called "Z" registers. In some examples, these may be scalable vector registers according to the dominant vector length SVL in use, in accordance with the SVE or SVE2 systems described above.

[0016] When performing such execution, the storage array can function as an accumulation array of the type described above as ZA. The processing circuit can execute processing operations such as matrix processing operations using the storage array in response to the decoded instructions and accumulate the results of the operations. In other words, the instruction execution circuit executes the instructions decoded by the instruction decoder circuit 50, and the instruction execution circuit executes the decoded instructions by referring to one or more source operands stored by a set of architectural registers and is configured to hold one or more values generated by the decoded instructions. This can be, for example, an output for storage in an architectural register or one or more values for storage in memory.

[0017] A compression circuit 55 is provided that can provide a compression function as discussed below. In some examples, the compression circuit 55 can access at least a subset of the registers provided by the register circuit 70. Generally speaking, an interface circuit between the compression circuit 55 and the processing circuit 60 may be provided, but this is not shown in FIG. 1 for clarity of the figure and will be discussed in relation to FIG. 2.

[0018] Neural Processing and Weight Compression In certain examples, in some situations, the circuit of FIG. 1 can be used for neural network processing. In this type of configuration, the data values being processed are typically combined with (e.g., multiplied by) a set of weight values. Such processing can include performing many of these combinations and, thus, can advantageously be performed by a vector or matrix processor.

[0019] Weight compression can be a useful technique for neural networks. Compressing the weights has the potential advantage of making the overall model smaller and easier to distribute, and also has the potential for higher performance since less bandwidth is required to load the weight data into the processor for processing.

[0020] Two examples of this are lookup table - based clustering and structured sparsity.

[0021] Examples of these techniques are discussed in Gong, Yunchao et al. “Compressing Deep Convolutional Networks using Vector Quantization” ArXiv abs / 1412.6115 (2014) (https: / / arxiv.org / pdf / 1412.6115.pdf) and https: / / developer.nvidia.com / blog / accelerating - inference - with - sparsity - using - ampere - and - tensorrt / , and the entire contents of both are incorporated herein by reference.

[0022] Clustering is a technique in which the processing values (typically weight values or activation values) are forced to be one of a finite number of specific values. Each processing value can then be stored as an index into a codebook of values. Before such processing values can be used in a computation, they need to be expanded back to their full - size representation.

[0023] Structured sparsity is another technique where some of the processed values are forced to zero. By "structured", it means that the level of sparsity is implemented at a fine-grained level. For example, so-called "2-in-4 sparsity" divides the processed values into 4 blocks, and at least 2 of them must be zero. This makes it possible to represent the data as an array of half the size of the non-zero data values and some sideband "presence" data indicating where they are within each block.

[0024] Both of these techniques are compelling for the advantages of memory and bandwidth. Sparse representations or clustered representations allow processed values to occupy less memory space and consume less bandwidth (or provide higher performance for a given fixed bandwidth) when used in computations. Technical literature has shown that this compression can be achieved with little or no impact on the functionality of neural networks (as measured by accuracy). Also, these are very likely not the only techniques that can be used for this purpose. This presents the challenge of either the processor converting the data into a more appropriate form before computation or implementing specialized operations to work directly on the compressed representation.

[0025] Exemplary configurations relate to providing and interfacing a compression circuit for handling such decompression of processing values, such as weights or activation values (however, the technique is independent of the nature of the actual data being decompressed). However, while decompression of processing values is discussed as an example of the technique, this example is not limited to decompression. The compression circuit can be used for data compression rather than decompression, for example, by applying the 2-in-4 sparsity technique discussed in this application. In fact, at a basic level, the operation of the compression processor can be regarded as providing a process of converting a data set from one representation to another. Generally speaking, "compression" is usually understood to mean that the destination format provides a representation that uses less data (regardless of whether it is lossless) than the input format, while "decompression" is usually understood to mean that the destination format provides a representation that uses more data than the input format. When used herein in connection with the data compression circuit 55, or algorithms, systems, sets of parameters, etc. related to the operation of the compression circuit 55, the term "compression" can include compression or decompression.

[0026] As described above, various compression schemes are used, which means that this is an area of ongoing research and there is a possibility that further compression schemes will be developed in the future. Although compression is valuable, considering the lack of broad standardization of compression methods, exemplary embodiments aim to provide a mechanism that can support various compression schemes.

[0027] One approach to supporting data compression or decompression could be to add extensions to an instruction set (such as that defined by an instruction set architecture, or ISA, applicable to a processing circuit). This approach would require modifying the ISA in response to the development (or increased prevalence) of compression techniques. However, the process for modifying a processor's ISA can be relatively elaborate and cumbersome. Also, this can lead to legacy techniques that occupy the ISA encoding space, i.e., the set of values that can be used to represent instructions. This may be undesirable considering that the ISA encoding space is finite and that, if legacy techniques exist within the ISA, compatible processors may be required to provide hardware for implementing such legacy techniques.

[0028] The techniques of the present disclosure can potentially provide one or more of enabling various compression schemes to be supported, enabling inclusion of new schemes (and / or deprecation of old schemes) without necessarily requiring changes to the operating system or processing circuit, and avoiding the need to consume additional instruction set space for each additional compression scheme.

[0029] Example of compression circuit 55 FIG. 2 provides a schematic example showing a compression circuit 55 (as an example of a data compression processor or circuit), a processing circuit 60, and an interface circuit 200 that defines an interface for data communication between the processing circuit 60 and the compression circuit 55.

[0030] For the purposes of this discussion, the compression circuit 55 can be regarded as a "compression scheme black box". That is, from the perspective of the processing circuit 60, how the compression circuit 55 operates is not important in many respects, and the compression circuit 55 need only be able to receive data and / or parameters in a particular format and output data in another particular format.

[0031] In some examples, the compression circuit may receive clustered processing (e.g., weight or activation) values and / or weight values in a structured sparsity format, and output decompressed processing values for use in neural processing by the processing circuit 60.

[0032] In response to one or more instructions of an instruction set defined for the processing circuit, the processing circuit 60 provides to the interface · input data processed by the data compression circuit, and · identification data identifying the compression scheme used by the data compression circuit to process the input data. In other words, in these examples, the ISA provides control of the general capabilities of the processing circuit for communicating with the compression circuit 55 via the interface circuit 200, but the ISA need not provide specific control of a particular compression scheme or algorithm.

[0033] Specifically, data communication with the compression circuit 55 may include a scheme ID (identifier) 240 that can define or identify the specific compression scheme used, e.g., a lookup table (LUT) for use in connection with the selected compression scheme, a configuration state 250 that defines parameters for use by the selected compression scheme, and optionally (such that the dynamic state of the compression processor 55 can be retrieved and stored by the processing circuit in a context switch and then restored when appropriate in a subsequent context switch) a dynamic state 260 that enables operations such as a context switch to occur. In other words, in connection with the dynamic state 260, the processing circuit may be configured to receive state data defining the current operating state of the data compression circuit from the interface, perform a context switch including at least storing the state data, and perform a further context switch including at least returning the stored state data to the data compression circuit.

[0034] The scheme ID and the configuration state (and optionally, the dynamic state) together enable the definition of a compression scheme that is operated by the compression circuit 55. With respect to the data processed by the compression circuit 55, this can be provided as input data to the handler 10 via a subset, such as 1 to 4 of the Z registers that are accessible by the processing circuit and by the compression circuit 55, and the processed data 220 can be output by the compression circuit 55 using again a subset, such as 1 to 4 of the Z registers.

[0035] The configuration state can be a two-way communication such that the compression circuit 55 can return to the processing circuit 60 via the interface circuit 200 data indicating matters such as the ability of the compression circuit 55 to execute the compression scheme defined by the scheme ID. Thus, in these examples, the processing circuit receives from the interface · status data indicating whether a data compression circuit connected to the interface can process data using the compression method identified by the identification data (e.g., via two-way communication of the configuration state), and · when the status data indicates that the data compression circuit can process data using the compression method identified by the identification data, configured to receive output data (e.g., as the processed data 220) processed from the input data by the data compression circuit using the compression method identified by the identification data.

[0036] The interface circuit 200 can provide data communication with the compression circuit 55, for example, using one or more of the registers implemented by the register circuit 70, and these registers can be accessible by the processing circuit 60 and by the compression circuit 55. In fact, all of the data communication functions of the interface circuit 200 can be implemented by processor registers, or a separate circuit configured to receive data from each of the processing circuit 60 and the compression circuit 55 and provide data to each of the processing circuit 60 and the compression circuit 55 can be used. In other words, in the example, the interface circuit comprises one or more processor registers that are accessible by the processing circuit and by a data compression circuit connected to the interface. For example, the processing circuit can be configured to write at least identification data to one or more processor registers and read at least status data from one or more processor registers.

[0037] The techniques defined herein can be applied to the processing circuit and the interface circuit whether or not the compression circuit 55 is present. In other words, the features of the present disclosure can be defined with respect to the configuration of the processing circuit and the interface circuit.

[0038] As described above, this configuration can include a data memory (e.g., memory 30 and / or storage array 90), and the processing circuit executes instructions that define a neural network to which processing values are applied to data values, reads compressed data that defines the processing values from the data memory, provides the compressed data as input data to the interface, and receives, as output data, decompressed processing values for use by the instructions that define the neural network.

[0039] Similarly, using this configuration, for example, a processing circuit can compress a processing value for use by another processor configured to read data defining the processing value from a data memory, provide the data defining the processing value as input data to an interface, and receive the compressed processing value as output data.

[0040] Direct processing path In this example, a direct processing path 230 with one Z register as input and one to four Z registers as output is also shown. This provides selectable passing paths if required by the application in use.

[0041] In some examples, at least two operating modes are supported.

[0042] In one mode, the loading of input data and the extraction of output data are separate operations. This may be suitable for structured sparsity where, for example, a "sideband" presence bit needs to be provided but no output is generated immediately. It may also be useful for more advanced variable rate schemes where there is no fixed relationship between the input stream and the output stream.

[0043] In another mode, so-called direct processing, operations are combined such that a single operation can provide input data and extract the corresponding output data. This can be useful for lookup table schemes where there is a direct correspondence between the two, reducing the number of instructions required to perform decompression and simplifying context switches (when separate input and output instructions are used, the input data becomes part of the state in between).

[0044] In practice, a mixture can be used. For example, structured sparsity can load the existing data using an "input" mechanism and then expand the input vector to the output vector using a "direct decompression" mode (implicitly consuming part of the existing data).

[0045] Exemplary operation FIG. 3 is a schematic flowchart showing an example of the operation between the processing circuit 60 (the operation shown on the left side of the vertical illustration boundary 300) and the compression circuit 55 (the operation shown on the right side of the boundary 300).

[0046] In step 310, the processing circuit establishes parameters for processing by the compression circuit 55. As described above, this may be in response to the execution of one or more instructions within the ISA of the processing circuit 60, the decoder circuit 50 decodes the instructions for execution, and the processing circuit 60 executes those instructions decoded by the decoder circuit 50.

[0047] In step 320, the processing circuit provides at least identification data to the compression circuit 55 via the interface circuit 200, and in step 330, the compression circuit receives the at least identification data.

[0048] In step 340, the compression circuit 55 compares the identification data that defines at least the scheme ID, for example, with the processing capabilities of the compression circuit 55, and in step 343, returns a response to the processing circuit 60 by transmitting status data, and the processing circuit receives the status data in step 346. This response may represent the status data described above.

[0049] In step 350, the processing circuit detects whether the status data indicates that the compression circuit 55 can handle the compression scheme defined by the scheme ID.

[0050] Regarding the negative result of step 350, the processing circuit is configured to execute an instruction to control the processing circuit to execute the processing of the input data when the status data indicates that the compression circuit 55 cannot process the data using the compression method identified by the identification data.

[0051] If the result is yes, the compression circuit 55 is executed to process and return the data. In particular, step 360 represents the start of processing data using the compression circuit 55. In step 363, depending on the interface technology being used, the processing circuit 60 writes the data to be processed into one or more registers, and the compression circuit 55 reads that data in step 366. The compression circuit 55 processes the data in step 370. In step 373, the compression circuit writes the processed data into one or more registers, and in step 376, the processing circuit reads the processed data. Next, when there is further data to be processed in this way, the control optionally returns to step 360.

[0052] Specific example Here, a specific example will be described.

[0053] (As an example of identification data) The scheme ID identifies the specific compression scheme in use. The scheme selected here controls how the input data is converted to output data and how other framework features are used.

[0054] Within the overall system defined by the scheme ID, scheme-specific states can be considered. This can define a configuration state. For example, as described above, many compression schemes require a static state to be configured before processing can occur. For example, a lookup table (LUT)-based scheme requires the LUT itself to be set up. This may not be required by all schemes. If the configuration data is only a few bits (e.g., an input / output width selector), the configuration data can be incorporated into the scheme ID.

[0055] In connection with the terms used above, in these examples, the identification data can be configured to define a compression algorithm and one or more parameters for use during the execution of the compression algorithm. For example, the one or more parameters can include one or more lookup tables for use with the compression algorithm defined by the identification data. In some examples, the one or more parameters can include data that defines an initial state to be applied to the data compression algorithm defined by the identification data.

[0056] Scheme-specific states can also define dynamic states.

[0057] Most compression schemes will have either a dynamic state, such as compressed data that has been loaded but not yet extracted, or an "internal" state, such as a dictionary that is derived from the incoming data stream but needs to be maintained to implement decompression.

[0058] Schemes that support only direct decompression may not require a dynamic state.

[0059] Scheme-specific states can also define combined states.

[0060] For a general compression method to function, there should be a scheme-independent way to switch all the states associated with the scheme regarding context switches (and possibly also in user code if a mechanism similar to ZA is used to manage user code). This can include all configuration states. Also, software that uses a particular scheme (i.e., instructions for execution by processing circuit 60) can use scheme-specific knowledge about how to configure and use it.

[0061] Regarding the above reference to a mechanism similar to ZA, the following further explanation is provided. In user code, an ABI (Application Binary Interface) mechanism may be provided, whereby a calling function that uses ZA can call a second function (the second function may or may not use any SME functions), leave the live state in ZA, and be able to store a pointer to the storage area for the ZA state. If at some point during the processing of the second function (which may involve further function calls) it is necessary to use ZA, the called function checks the relevant status bits and, if necessary, stores the contents of ZA in the pointer before using ZA, regardless of what requirements are defined by the second function or the function called by the second function. When the second function returns, the calling function checks the status bits to see if the contents of ZA were saved, and if so, restores the contents of ZA from the storage area before resuming processing.

[0062] This potentially allows for a lower overhead when the second function does not use ZA (since the calling function saves ZA and does not need to restore ZA later when it has not actually been changed), while not incurring an excessive overhead in that even when the second function uses ZA, status bit checking is required upon entry to a function that uses ZA and upon return from such a function call.

[0063] A similar scheme may be used for the state within the compression circuit, such that the save code should be able to save data without knowing exactly what it is saving (this is similar to the context switch requirements).

[0064] In other words, in some examples, user functions may need to be saved / restored across function calls.

[0065] As described above, with respect to data input / output, all of the schemes use a mechanism that passes (for example) compressed (or input) data to the scheme and retrieves the decompressed (or processed) data again. Here, at least two variations are considered. · Separate input / output instructions. Two separate operations are provided: the transfer from the register to the compressor (1 to 4 Z registers), and the transfer from the compressor to the register (1 to 4 Z registers). This is well-suited for schemes where there is no simple fixed ratio between input and output, or where multiple data are required for processing (for example, block or structured sparsity schemes). · Combined input / output instructions. This is a single instruction that consumes one Z register and generates 1 to 4 Z registers of output.

[0066] In some examples, a register is dedicated or added to select the compression scheme. Bits 31:0 of this register identify the compression scheme used, and a reserved value of 0 indicates that no scheme is selected. If the value written to this register identifies a scheme not supported by the current implementation, the register is set to 0 by the compression circuit 55. Software can read back the register to verify (as status data) that the scheme is supported.

[0067] A further example may allow multiple compression schemes to be used simultaneously.

[0068] The compression scheme can have an associated state that is either preconfigured before the scheme is used or populated during operation.

[0069] In some examples, a system register is added or dedicated (or may be set to constitute the amount of state required, depending on the scheme) to indicate the amount of state currently stored. Whether this register is read-only or read-write is defined by the scheme. Some schemes (e.g., look-up tables) can function without referring to this register.

[0070] For any supported compression scheme, an additional register may be dedicated or added to indicate the maximum possible live state size. This can be used by the operating system to appropriately size the fixed context switch buffer.

[0071] To read and write the state, two new instructions are added. MOVCSSR0 - Move from register to compression scheme state The syntax is MOVCSSR0 Zt,Xn{,# <imm>,MUL VL} can be.

[0072] This instruction will load the state into the compression scheme. The state of the compression scheme is modeled as a single 1D buffer of a certain length (as described in the previous paragraph). Zt identifies the vector register that holds the state to be loaded, and Xn (in some cases, "imm * VL" is added) indicates the offset within the buffer to be written. MOVRCSS0 - Move from Compression Scheme State to Register The syntax is MOVRCSS0 Zt,Xn{,# <imm>,MUL VL} may be.

[0073] This is the same as above, where Xn identifies which bit of the buffer to copy, and Zt indicates where to copy it.

[0074] Three new instructions may be provided to actually perform compression, decompression, or both.

[0075] Load the compressed data. Move from the MOVCR0 - register to the compression scheme. The syntax may be MOVCR0 CRx,Zt{or multiple}.

[0076] CRx is the compressed register ID, which is probably 3 - 4 bits in a scheme - specific meaning.

[0077] The above uses of MOVRCSS0 and MOVCR0 provide examples of the ISA of separate individual instructions to provide identification data to the interface (MOVRCSS0 in this example) and to provide input data to the interface (MOVCR0 in this example).

[0078] Extract the decompressed data. MOVRC0 - Move from the compression scheme to the register The syntax may be MOVRC0 Zt{or multiple},CRx.

[0079] Zt defines the Z - register. CRx is the register ID, and the relationship between MOVCR and MOVRC depends on the compression scheme.

[0080] The use of the MOVRC0 instruction provides an example of the ISA's further instruction provision to retrieve output data from the interface.

[0081] Direct decompression Decompress with DECOMP0 - scheme 0. The syntax can be DECOMP0 Zd{or multiple},Zs,{sub ID}.

[0082] Here, Zs is the source register and Zd is the destination register. The sub - ID can be provided to select a portion of the register when one Zs is decompressed to two or more Zds.

[0083] This direct decompression instruction can provide an example of an instruction (defined by the ISA) for providing input data and identification data to an interface and for starting the processing of input data by a data compression circuit using the compression scheme defined by the identification data.

[0084] Here, an example regarding the look - up table will be described. In this example, a scheme for expanding 2 - bit or 4 - bit compressed values into 8 -, 16 -, or 32 - bit values can be implemented.

[0085] The input / output width is encoded in the scheme ID. Thus, there is a 1 - bit field for the encoded value of either 2 bits or 4 bits. In other examples, this can leave space for a 1 - or 8 - bit field and a 2 - bit field for encoding the output width. The remaining 28 - 29 bits can have a constant value that identifies this as a look - up table scheme. This provides an example where one or more parameters are configured to define the respective input and output data widths for the compression scheme.

[0086] [Table 1]

[0087] MOVCSSR0 is used to load in the look - up table state. Depending on VL and the state size (see the above table), this may require one operation or several operations.

[0088] The lookup is executed in DECOMP0. The lookup table, input, and output widths are all configured as described above, and one input register and one to four output registers are provided. If this ratio is greater than 4:1, a "sub ID" can be used to indicate which slice of the input is consumed.

[0089] An example regarding structured sparsity is described here.

[0090] This can support "2-in-4" structured sparsity for data types such as 8, 16, or 32 bits, for example. The nature of the sparsity scheme, such as defining the number of sparse blocks (such as 2) in a group of blocks (such as the four blocks in an exemplary "2-in-4" scheme), and / or the data type in use can be provided by identification data or defined separately or collectively by parameters associated with the identification data in a manner that can be.

[0091] Since the scheme ID affects the ratio of register access, it can include a 2-bit wide indicator.

[0092] A register for indicating the amount of state can indicate up to the VL byte how much unused sparsity data remains in the sparsity data register.

[0093] Regarding the compression scheme state, it is not always necessary to explicitly set anything before using the scheme. The context switch code will read the register to indicate the amount of state and use state instructions to save / restore the sparsity data.

[0094] It is assumed that sparse data is stored separately during thawing. Use MOVCR0 to load the sparse data. Then use DECOMP0 to convert one register of the packed data into two registers of the unpacked data and consume the appropriate part of the sparse data. The kernel is required to maintain the correct ratio between these two operations (i.e., 4:1 for 8-bit data, 8:1 for 16-bit data, or 16:1 for 32-bit data).

[0095] In summary, an exemplary configuration can provide the following features. · The details of the complete architecture can be defined in a separate specification for each scheme. Scheme development does not need to be aligned with the rest of the architecture process. · In an implementation where the actual operation is handled by a remote unit, the host CPU can decode the architecturally specified operations and pass them to the remote unit without the need to understand the details of the scheme in use. Therefore, a scheme can be added to the remote unit without touching the host CPU. This refers to an exemplary configuration where there are several possibilities for implementing SME with a host processor and a separate SME unit that handles SME instructions. In the case of non-SME code, the processor operates independently like any other code. When SME code is encountered, the SME instructions are sent to the SME unit via the bus. The SME unit typically has its own memory input / output path and register bank. In the context of this example where the "compression circuit" is part of that SME unit, the host CPU does not necessarily need to know which compression scheme is supported. All accesses can be sent to the SME unit for processing. · Unused schemes can be deprecated independently of the actual architecture, but a particular architecture release may require support for a particular scheme if necessary. · Each additional scheme does not need to consume additional instruction set space.

[0096] Method Example Figure 4 shows a method that (at step 400) decodes an instruction for execution, uses a processing circuit to execute the instruction decoded by the instruction decoding step (at step 410), (at step 420) defines an interface for data communication with a data compression circuit, in response to one or more instructions of an instruction set defined for the processing circuit, the processing circuit (at step 430) provides to the interface input data to be processed by the data compression circuit and identification data identifying a compression method to be used by the data compression circuit for processing the input data, the processing circuit (at step 440) receives from the interface status data indicating whether the data compression circuit connected to the interface can process data using the compression method identified by the identification data, and when the status data indicates that the data compression circuit can process data using the compression method identified by the identification data, the output data processed from the input data by the data compression circuit using the compression method identified by the identification data, and is a schematic flowchart showing a method.

[0097] Simulator Example FIG. 5 shows an implementation form of a simulator that can be used. The foregoing embodiments implement the present disclosure in terms of an apparatus and method for operating specific processing hardware that supports the technique, but it is also possible to provide an instruction execution environment according to the embodiments described herein implemented through the use of a computer program. Such a computer program is often referred to as a simulator as long as the computer program provides a software-based implementation of a hardware architecture. Various simulator computer programs include emulators, virtual machines, models, and binary translators including dynamic binary translators. Typically, the simulator implementation form can optionally be executed on a host processor 530 that executes a host operating system 520 that supports a simulator program 510. In some arrangements, there may be multiple layers of simulation between the hardware and the provided instruction execution environment, and / or there may be multiple different instruction execution environments provided on the same host processor. Historically, powerful processors have been required to provide simulator implementations that execute at a reasonable speed, but such an approach may be justified in certain situations, such as when it is desired to execute native code for another processor for reasons of compatibility or reuse. For example, the simulator implementation may provide an instruction execution environment having additional functionality not supported by the host processor hardware, or may provide an instruction execution environment typically associated with a different hardware architecture. An overview of simulation is described in "Some Efficient Architecture Simulation Techniques", Robert Bedichek, 1990 Winter USENIX Conference, pages 53-63.

[0098] Previously, embodiments have been described with reference to specific hardware components or features, but in simulated embodiments, equivalent functionality can be provided by suitable software components or features. For example, a specific circuit may be implemented as computer program logic in a simulated embodiment. Similarly, memory hardware such as registers or caches may be implemented as software data structures in a simulated embodiment. In configurations where one or more of the hardware elements mentioned in the foregoing embodiments are present on host hardware (e.g., host processor 530), some simulated embodiments can use the host hardware where appropriate.

[0099] The simulator program 510 may be stored in a computer-readable storage medium (which may be a non-transitory medium), provide a program interface (instruction execution environment) to the target code 500 (which may include an application, an operating system, and / or a hypervisor), and the program interface is the same as the application program interface of the hardware architecture modeled by the simulator program 510. Thus, the program instructions of the target code 500, including code for implementing the functions described above, such as those described with reference to FIG. 4, may be executed within the instruction execution environment using the simulator program 510, and as a result, a host computer 530 that does not actually have the hardware functions of the device 10 discussed above can emulate these functions.

[0100] General Matters In the present application, the term "configured to..." is used to mean that an element of a device has a configuration capable of performing a defined operation. In this context, "configuration" means a way of arranging or interconnecting hardware or software. For example, a device may have dedicated hardware that provides a defined operation, or a processor or other processing device may be programmed to perform a function. "Configured to" does not mean that some change needs to be made to the device element to provide a defined operation.

[0101] Exemplary embodiments of the present technology have been described in detail herein with reference to the accompanying drawings, but the present technology is not limited to those exact embodiments, and it should be understood that various changes, additions, and modifications can be made by those skilled in the art without departing from the scope and spirit of the technology defined by the appended claims. For example, various combinations of the features of the dependent claims can be made by the features of the independent claims without departing from the scope of the present technology.< / imm> < / imm>

Claims

1. It is a circuit, An instruction decoder circuit that decodes instructions for execution. A processing circuit that executes the instruction decoded by the instruction decoder circuit. It comprises a data compression circuit and an interface circuit that defines an interface for data communication, The processing circuit, in response to one or more instructions of an instruction set defined for the processing circuit, provides the interface with input data to be processed by the data compression circuit and identification data that identifies a compression scheme to be used by the data compression circuit to process the input data. The processing circuit is configured to receive from the interface status data indicating whether a data compression circuit connected to the interface can process data using the compression method identified by the identification data, and, when the status data indicates that the data compression circuit can process data using the compression method identified by the identification data, output data processed by the data compression circuit from the input data using the compression method identified by the identification data.

2. The circuit according to claim 1, wherein the identification data is configured to define a compression algorithm and one or more parameters for use during the execution of the compression algorithm.

3. The circuit according to claim 2, wherein the one or more parameters include at least a portion of one or more lookup tables for use with a compression algorithm defined by the identification data.

4. The circuit according to claim 3, wherein one or more of the parameters are configured to define the respective input and output data widths for the compression scheme.

5. The circuit according to claim 2, wherein the one or more parameters are configured to define one or more embodiments of a structured sparsity compression algorithm, the one or more embodiments comprising one or both of the number of sparse blocks in a group of blocks and the data types used by the structured sparsity compression algorithm.

6. The circuit according to claim 2, wherein one or more of the parameters include data that defines an initial state applied to the compression algorithm defined by the identification data.

7. The instruction set defined for the processing circuit is, The circuit according to any one of claims 1 to 6, comprising an instruction to provide the interface with the input data and the identification data, and to start processing the input data by the data compression circuit using a compression scheme defined by the identification data.

8. The circuit according to any one of claims 1 to 6, wherein the instruction set defined for the processing circuit includes separate instructions for providing identification data to the interface and for providing input data to the interface.

9. The circuit according to claim 8, wherein the instruction set defined for the processing circuit includes further instructions for retrieving output data from the interface.

10. The circuit according to any one of claims 1 to 6, wherein the processing circuit is configured to receive state data from the interface that defines the current operating state of the data compression circuit, perform a context switch which includes at least storing the state data, and perform a further context switch which includes at least returning the stored state data to the data compression circuit.

11. The circuit according to any one of claims 1 to 6, wherein the interface circuit comprises one or more processor registers accessible by the processing circuit and by a data compression circuit connected to the interface.

12. The circuit according to claim 11, wherein the processing circuit is configured to write at least the identification data to one or more processor registers and to read at least the status data from one or more processor registers.

13. The circuit according to any one of claims 1 to 6, comprising a data memory, wherein the processing circuit is configured to execute instructions defining a neural network to which processing values ​​are applied to data values, read compressed data defining the processing values ​​from the data memory, provide the compressed data as input data to the interface, and receive decompressed processing values ​​as output data for use by the instructions defining the neural network.

14. The circuit according to any one of claims 1 to 6, comprising a data memory, wherein the processing circuit is configured to execute an instruction that defines a neural network to which a processing value is applied to a data value, read data that defines the processing value from the data memory, provide the data that defines the processing value to the interface as input data, and receive the compressed processing value as output data.

15. The circuit according to any one of claims 1 to 6, comprising a data compression circuit connected to the interface.

16. The circuit according to any one of claims 1 to 6, wherein the processing circuit is configured to execute an instruction to control the processing circuit to perform processing of the input data when the status data indicates that the data compression circuit cannot process the data using the compression method identified by the identification data.

17. It is a method, Decoding instructions for execution, Using a processing circuit, execute the instruction decoded by the instruction decoding step, Define the data compression circuit and the interface for data communication, In response to one or more instructions of an instruction set defined for the processing circuit, the processing circuit provides the interface with input data to be processed by the data compression circuit and identification data that identifies a compression scheme to be used by the data compression circuit to process the input data. A method comprising: the processing circuit receiving from the interface status data indicating whether a data compression circuit connected to the interface can process data using the compression scheme identified by the identification data; and, when the status data indicates that the data compression circuit can process data using the compression scheme identified by the identification data, output data processed by the data compression circuit from the input data using the compression scheme identified by the identification data.

18. A virtual machine computer program that includes instructions for controlling a host data processing device to provide an instruction execution environment, An instruction decoder circuit that decodes instructions for execution. A processing circuit that executes the instruction decoded by the instruction decoder circuit. It comprises a data compression circuit and an interface circuit that defines an interface for data communication, The processing circuit, in response to one or more instructions of an instruction set defined for the processing circuit, provides the interface with input data to be processed by the data compression circuit and identification data that identifies a compression scheme to be used by the data compression circuit to process the input data. A virtual machine computer program, wherein the processing circuit is configured to receive from the interface status data indicating whether a data compression circuit connected to the interface can process data using the compression method identified by the identification data, and output data processed by the data compression circuit from the input data using the compression method identified by the identification data when the status data indicates that the data compression circuit can process data using the compression method identified by the identification data.

Images