Data Structure Compression Technology Applicable to Artificial Neural Networks

Abstract

The present invention discloses a data structure compression technique suitable for artificial neural networks. In artificial neural networks and other similar applications, a large amount of data that is considered to be sparse data is usually involved. Since the size of the data involved in these applications is large, compressing the data helps save bandwidth resources when sending the data and save memory resources when storing the data. A compression technique is introduced here that selects elements with important values ​​from the data and reconstructs them into a structured sparse format. By generating metadata that enforces the structured sparse format and organizing the data according to the metadata, the introduced technique not only reduces the size of the data, but also always puts the data in a specific format. Therefore, the hardware can be simplified and optimized to process the data faster and more efficiently than conventional compression techniques that rely on unstructured sparse formats.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application Serial No. 62 / 850,727, filed on May 21, 2019, entitled “COMPRESSION TECHNIQUES FOR DATA STRUCTURES SUITABLE FOR ARTIFICIAL NEURAL NETWORKS,” which is commonly assigned with the present application and is incorporated herein by reference in its entirety. Technical Field

[0003] The present invention relates to data compression, and more particularly, to compression of sparse data structures. Background Art

[0004] Artificial neural networks (ANNs) are computing systems inspired by biological neural networks. ANNs provide a framework for many different machine learning algorithms to work together and process complex data inputs. ANNs have been used for a variety of tasks including computer vision, speech and image recognition, machine translation, social network filtering, playing video games, and medical diagnosis. Summary of the invention

[0005] The first aspect provides a method for compressing data, comprising: generating metadata from an N-element data structure, the generation comprising: selecting M elements from the N-element data structure, where N is greater than M; determining positions of the M elements in the N-element data structure; and storing the positions of the M elements as metadata; compressing the N-element data structure into an M-element data structure according to the metadata, the compression comprising: collecting values ​​of the M elements from the N-element data structure according to the positions; and storing the values ​​of the M elements in the M-element data structure.

[0006] A second aspect provides a system for compressing data, comprising: a memory and at least one processor communicatively coupled to the memory. The at least one processor is configured to: generate metadata from an N-element data structure by: selecting M elements from the N-element data structure, where N is greater than M; determining positions of the M elements in the N-element data structure; storing the positions of the M elements as metadata; and compressing the N-element data structure into an M-element data structure according to the metadata by: collecting values ​​of the M elements from the N-element data structure according to the positions; storing the values ​​of the M elements in the M-element data structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Reference is now made to the following description in conjunction with the accompanying drawings, in which:

[0008] Figure 1 is an illustration of an exemplary M:N compression format according to one or more aspects of the present disclosure;

[0009] Figure 2 is an illustration of an exemplary 2:4 compression format according to one or more aspects of the present disclosure;

[0010] Figure 3 is a block diagram of an embodiment of a computing system that performs data compression according to one or more aspects of the present disclosure;

[0011] Figure 4 is a flowchart of an embodiment of a method for compressing data according to one or more aspects of the present disclosure;

[0012] Figure 5-Figure 9 is an example of a GENMETADATA instruction for generating metadata according to one or more aspects of the present disclosure;

[0013] Figure 10-Figure 14 is an example of a GATHER instruction that compresses an N-element data structure into an M-element data structure according to one or more aspects of the present disclosure;

[0014] Fig.15 is an example of a GATHERPLUS instruction that generates metadata and compresses an N-element data structure into an M-element data structure according to one or more aspects of the present disclosure;

[0015] Figure 16-Figure 21 is an example of a GENMETADATA instruction and a GATHER instruction cascaded to achieve a 2:8 compression ratio according to one or more aspects of the present disclosure. DETAILED DESCRIPTION

[0016] In artificial neural network (ANN) and other similar applications, a large amount of data is often involved that is considered as sparse data, or in other words, data that includes a large number of zero values ​​or near-zero values. Due to the large size of the data involved in these applications, compressing the data (e.g., reducing the size of the data) helps to save bandwidth resources when sending the data and save memory resources when storing the data.

[0017] Disclosed herein is a compression technique that compresses data by reconstructing the data to follow structured sparsity. The disclosed technique generates metadata that identifies data elements that contain valid values of the data and conform to structured sparsity, and reconstructs the data based on the metadata. Thus, the disclosed technique not only reduces the size of the data, but also consistently places the data in a specific sparse format. As a result, the hardware can be simplified and optimized to process data, such as computing, transmitting, and storing, faster and more efficiently than conventional compression techniques that rely on unstructured sparse formats such as CSR (compressed sparse row) or COO (coordinate list). For example, using data compressed from the disclosed technique allows for training a deep learning model with a larger amount of data, resulting in better accuracy at the same computational cost as using different compression techniques.

[0018] Figure 1 An exemplary M:N compression format is shown. The input data (data to be compressed) in the uncompressed data format 100 includes N elements in the N-element data structure 102, and the output data (compressed data) in the compressed data format 110 includes M elements in the M-element data structure 112 and metadata 114, where M < N. The N elements in the N-element data structure 102 are compressed into M elements in the M-element data structure 112. The metadata 114 indicates the position (e.g., N-1, N-2, 3, 1) in the N-element data structure 102 from which each element DN-1, ..., D1, D0 in the M-element data structure 112 will be copied. Figure 1 The compression format shown refers to a compression ratio of M to N (“M:N”). Each element in the M-element data structure 112 and the N-element data structure 102, e.g., DN-1, ..., D1, D0 represents a numerical value, which can be any number of bits (e.g., 4 bits (“nibble”), 8 bits (“byte”), 16 bits (“halfword”), 32 bits (“word”), 64 bits (“double”), etc.) encoded as an integer, floating point, fixed, logarithmic, or any other digital format used to encode numerical values on a computer.

[0019] Figure 2 An exemplary 2:4 compression format 200 is shown. In the example shown, 8 elements in the uncompressed format 200 (which are located in two 4-element data structures 204 and 205) are compressed into 4 elements in the compressed data format 202, which includes two 2-element data structures 206 and 207 that hold the numerical values of 4 elements, and two metadata 208 and 209 that hold the positions of 4 elements. Thus, in the illustrated example, N is 4 and M is 2.

[0020] In the example shown, elements D and C in the first 4-element data structure 204 are selected and copied to the first 2-element data structure 206, and elements B and A in the second 4-element data structure 205 are selected and copied to the second 2-element data structure 207. The first metadata 208 indicates the position (e.g., index) of the selected elements in the first 4-element data structure 204, and the second metadata 209 indicates the position of the selected elements in the second 4-element data structure 205.

[0021] The number of elements to be selected and the basis for selecting these elements for compression are specified in the executed compression instruction. In order to facilitate the communication between the 4-element data structure and the 2-element data structure and their metadata, the size, location (such as register) and offset of the 4-element data structure and the 2-element data structure and their metadata are also specified in the compression instruction.

[0022] The compression instruction may be divided into two separate instructions, one for generating metadata and another for collecting data, or combined into a single super instruction. The compression instruction may also be executed to operate on a single data set (e.g., the first 4-element data structure 204 or the second 4-element data structure 205), or to operate on multiple data sets (e.g., both the first 4-element data structure 204 and the second 4-element data structure 205).

[0023] Since the compression shown is a lossy type of compression that logically operates on one or more sets of relatively few elements (e.g., 4 elements), it can be applied to Figure 2 The data shown is much larger as well as much smaller data (eg, in number of elements). Furthermore, as shown, the compression follows an M:N compression ratio, so that for every N elements, M elements are retained, which maintains structured sparsity throughout the compressed data format.

[0024] Figure 3 A block diagram of an embodiment of a computing system 300 is shown, which is configured to perform compression operations on data according to the principles of the present disclosure. In the illustrated embodiment, the system 300 includes parallel processing units (PPUs) 310 to 310-N (hereinafter referred to as PPUs 310) and their respective memories 315 to 315-N (hereinafter referred to as memories 315). The PPUs 310 and memories 315 are communicatively coupled to each other using conventional means (such as high-speed interconnects).

[0025] Each PPU 310 includes one or more processors 312, which are configured to perform various operations, such as compression, decompression, and transposition of data. The data may be, for example, a vector, a tensor, or a matrix. Each processor 312 is multithreaded and is configured to concurrently execute multiple threads (e.g., 32 threads) from a particular thread group. In one embodiment, the processor 312 implements a SIMD (single instruction, multiple data) architecture, in which each thread in a thread group (e.g., a warp) is configured to process different data sets based on the same instruction set. In another embodiment, the processor 312 implements a SIMT (single instruction, multiple thread) architecture, in which each thread in a thread group is configured to process different data sets based on the same instruction set, in which each thread can execute a different subset of the same instruction set.

[0026] Each PPU 310 also includes temporary storage (TS) 314, such as a register file or memory, configured to stage or temporarily store data between the processor 312 and the memory 315. The PPU 310 may be a general or special processing unit of different types, such as a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), and an application specific integrated circuit (ASIC).

[0027] Each memory 315 is configured to store data required to perform various operations. Each memory 315 may store the actual value of the data involved in the operation and a series of instructions that may cause the PPU 310 to perform the operation. In one embodiment, the memory 315 may include a plurality of dynamic random access memory (DRAM) devices and / or random access memory (RAM) devices.

[0028] Figure 4 400 is a flowchart of an embodiment of a method 400 for performing compression of data according to the principles of the present disclosure. The method 400 or at least a portion thereof may be performed by a computing system, such as Figure 3 The computing system 300 in FIG. 400 starts from step 405 .

[0029] In step 410, metadata is generated from an N-element data structure to be compressed by executing a first instruction. N is a positive integer indicating the number of elements in the data structure being compressed. In the illustrated embodiment, the N-element data structure represents an N-element vector. The N-element data structure may also represent an N-element tensor or an N-element matrix. The N-element data structure may be one of many N-element data structures in the source storage. In the illustrated embodiment, the first instruction is a metadata generation instruction, referred to as the GENMETADATA instruction. In one embodiment, the first instruction is an operation of a "super instruction" that performs metadata generation and data compression operations. An example format of the GENMETADATA instruction is provided below.

[0030] As a first sub-step of step 410, an N-element data structure is received as an input to a first instruction. The N-element data structure is pulled from a source register or memory location, which is indicated in the first instruction. The source register / memory location may correspond to a temporary storage (e.g., Figure 3 A specific register file or memory location in TS 314 in .

[0031] As a second sub-step, step 410 uses the function in the first instruction to select M elements from the N-element data structure. M is a positive integer less than N. The function can be, for example, a MAX function that compares the values ​​of all elements in the N-element data structure and selects M elements with larger values, a MAXABS function that compares the sizes of all elements in the N-element data structure and selects M elements with larger sizes, a MIN function that compares the values ​​of all elements in the N-element data structure and selects M elements with smaller values, or a MINABS function that compares the sizes of all elements in the N-element data structure and selects M elements with smaller sizes. It is understood that other functions may also be used, such as a function that selects a certain number of elements with values ​​within a certain range.

[0032] As a third sub-step of step 410, the position of the selected M elements in the N-element data structure is determined. When the N-element data structure is one of multiple N-element data structures in a larger data structure, the starting position of the N-element data structure in the larger data structure is determined using the group index indicated in the first instruction and used as an offset to the position of the M elements in the N-element data structure.

[0033] As a fourth sub-step of step 410, the locations of the M elements are stored as metadata in a destination register / memory location. The metadata index indicated in the first instruction is used to determine the location where the metadata is stored in the destination register, for example, the starting location (offset) of the metadata in the destination register. The destination register may correspond to a temporary storage (e.g., Figure 3It can be understood that the positions of the elements not selected in the second sub-step of step 410 are not determined and therefore are not stored in the third and fourth sub-steps.

[0034] In step 420, the N-element data structure is compressed into an M-element data structure based on the metadata by executing a second instruction. In the illustrated embodiment, unlike the first instruction, the second instruction is a data compression instruction. The data compression instruction is referred to as a GATHER instruction. In one embodiment, the second instruction is an operation of a "super instruction" that performs metadata generation and data compression operations. An example format of the GATHER instruction is provided below. As a first substep of step 420, the metadata generated in step 410 and the N-element data structure are received as inputs to the second instruction. The N-element data structure and the metadata are accessed from their respective source registers / memory locations. The source registers may correspond to temporary storage (such as Figure 3 A specific register file or memory location in TS 314 in .

[0035] As a second substep of step 420, values ​​of selected elements (i.e., M elements) are collected from the N-element data structure according to the positions indicated in the metadata. When the N-element data structure is one of a plurality of N-element data structures in a larger data structure, the starting position of the N-element data structure in the larger data structure is determined using the group index and used as an offset to the positions of the M elements. The starting position of the metadata in the source register is determined using the metadata index. The group index and the metadata index are included in / indicated in the second instruction.

[0036] As a third substep of step 420, the values ​​of the collected M elements are stored in an M-element structure. In the illustrated embodiment, the M-element data structure represents an M-element vector. In other embodiments, the M-element data structure may represent an M-element tensor or an M-element matrix. The M-element structure is located at a destination register / memory location, which is indicated in the second instruction. The destination register may correspond to a temporary storage (e.g., Figure 3 At the end of step 420, the method 400 implements the M:N structured sparse format. The method 400 ends at step 425.

[0037] Method 400 may be performed by a single execution thread. Alternatively, method 400 may be performed by multiple threads. In one embodiment, a single thread compresses all N-element data structures of the input. In another embodiment, each of each pair of threads compresses half of the N-element data structures of the input. In another embodiment, each of the four threads compresses a quarter of the N-element data structures of the input. Although the present description above and below describes execution threads (e.g., on a GPU), it should be noted that execution channels (e.g., on each execution path of a single instruction, multiple data (SIMD) CPU) may also be used in place or outside of threads.

[0038] Method 400 may be repeated / cascaded to achieve higher compression ratios or sparsity. For example, when N is 4 and M is 2, method 400 may be repeated three times to achieve a compression ratio of 2:8. Method 400 may be performed twice to generate two intermediate M-element data structures from two N-element structures, and then performed a third time to generate a final M-element data structure from the combination of the two intermediate M-element structures.

[0039] An example format of the GENMETADATA directive is as follows:

[0040] GENMETADATA.func.selectN.fmt.idxsize.num{.SEQ}Rd,Ra,Rb,Rc,

[0041] #VecIdxU06,#MDidxU04

[0042] in:

[0043] .func: a function that controls how the values ​​of elements in an N-element data structure are compared to each other for selection (as described above, .func can be, for example, MAX, MAXABS, MIN, MINABS, etc.);

[0044] .selectN: The number of elements selected using .func, such as 1 or 2;

[0045] .fmt: a format for the elements in Ra and Rb, including integer formats such as signed 2-, 4-, 8-, 16-, and 32-bit integers (.S2, .S4, .S8, .S16, .S32) and unsigned 2-, 4-, 8-, 16-, and 32-bit integers (.U2, .U4, .U8, .U16, .U32), and non-integer formats such as IEEE half-precision (16-bit) or single-precision (32-bit) floating point (.FP16, .FP32), truncated versions of FP16 and FP32 (E8M7, E8M10), LNS (logarithmic system), POSITs (a proposed new dynamically scaled number format), other 8-bit floating point-like formats, and block exponent formats;

[0046] .idxsize: The size of the metadata index, such as 2 bits, 4 bits, or 8 bits (.U2, .U4, .U8)

[0047] .num: the number of N-element data structures entered in the source storage, such as 1 data structure, 2 data structures, or 4 data structures (.1G, .2G, .4G);

[0048] .SEQ: Optional sequential indexing: When operating on more than one N-element data structure, all N-element data structures are indexed sequentially, as if they were part of a single larger data structure, rather than repeating the same N index for each N-element data structure;

[0049] Rd: destination register or memory location;

[0050] Ra: The first source register or memory location containing all or part of an N-element data structure. Rb: The second source register or memory location containing all or part of an N-element data structure. Rc: The optional source register or memory location. When GENMETADATA specifies .idxsize and .num so that not all Rd bits are written, the contents of Rc flow into the other unwritten bits of Rd. When the optional source storage is not used, Rz (e.g., a register or memory location with a zero value) is used as a placeholder.

[0051] #VecIdxU06: group index; a 6-bit immediate value indicating the (starting) position of the N-element data structure in a larger data structure, for example, when the N-element data structure is one of multiple N-element data structures in a larger data structure;

[0052] #MDidxU04: Metadata index; a 4-bit immediate value that determines the offset (in bits) where the metadata is written to Rd.

[0053] Figure 5-Figure 9 An example of a GENMETADATA directive using the above format is provided.

[0054] Figure 5 An example of a GENMETADATA instruction 500 is illustrated that generates metadata 510 from an N-element data structure 520. In the example shown, N is 4, M is 2, and the example generates metadata 510 that indicates the location of 2 elements in the 4-element data structure 520.

[0055] In the example shown, the 4-element data structure 520 is stored in source registers R1 and R2, and the generated metadata 510 is stored in destination register R10. In the GENMETADATA instruction 500, ".1G" indicates that only one 4-element data structure is stored in the source register, and ".F16" indicates that each element in the 4-element data structure 520 is 16 bits. Since the size of each source position is limited to 32 bits in the example shown, each source register contains two elements. The #VecIdx U06 value of 0 indicates that the starting position of the 4-element data structure 520 is "0".

[0056] As indicated by MAX.2, instruction 500 selects two elements with larger values ​​of '3' and '7' from R1 and R2, which are at positions '2' and '1', respectively. Since the index size in metadata 510 is 2 bits (as indicated by .U2), position '2' is written as 1 0 and position '1' is written as 0 1 in metadata 510. Since #MDidxU04 is zero, the position bits of the selected elements are written to the first 4 bits (2x2 bits) of destination register R10.

[0057] Figure 6 Another example of a GENMETADATA instruction 600 is illustrated that generates metadata 610 from an N-element data structure 620. In the example shown, N is 4, M is 2, and the example generates metadata 610 that indicates the location of 2 elements in a 4-element data structure 620.

[0058] In the example shown, two 4-element input data structures (64 bits total) are stored in two source registers R1 and R2, as indicated by .2G and R1 and R2. Since each element of the 4-element data structure 620 is 8 bits (as indicated by .S8), each source register contains 4 8-bit elements. The value of #VecIdx U06 is 2, indicating that the starting position of the 4-element data structure 620 in each source register is "8". The starting position can be calculated by multiplying the value of #VecIdx U06 by 4. As indicated by MAXABS.2, instruction 600 selects two larger elements from each of the input 4-element data structures 620. Elements with values ​​of '3' and '-7' are selected from R1, and elements with values ​​of '6' and '5' are selected from R2. Since the starting position of the elements in each source position is '8', the positions of the elements with values ​​'6' and '5' in R2 are '11' and '9', respectively, and the positions of the elements with values ​​'3' and '-7' in R1 are '10' and '9', respectively. Since the index size in the metadata is 4 bits (as indicated by .U4), positions '11' and '9' are written in hexadecimal format as 'B' and '9', and positions '10' and '9' are written in metadata 610 as 'A' and '9'. Since no offset is specified, the positions are written to the first 16 bits (4x4 bits) of the destination register R10. It should be understood that when the index size in the metadata is 8 bits (as indicated by .U8), the positions are written in hexadecimal format with the most significant 4 bits set to zero.

[0059] Figure 7 Another example of a GENMETADATA instruction 700 is illustrated that generates metadata 710 from an N-element data structure 720. In the example shown, N is 4, M is 2, and the example generates metadata 710 that indicates the location of 2 elements in the input 4-element data structure 720.

[0060] In the example shown, two input 4-element data structures (64 bits total) are stored in two source registers R1 and R2, as indicated by .2G with R1 and R2. Since each element of the 4-element data structure 720 is 8 bits (as indicated by .S8), each source register contains four 8-bit elements. The #VecIdx U06 value of 1 indicates that each 4-element data structure 720 should start at position 4 (1x4). However, the presence of .SEQ indicates that the 4-element data structures 720 in the source registers R1 and R2 are arranged in sequence, and the 4-element data structures 720 do not start at position 4, but start at position 8 and advance to position 15. When .SEQ is set, the starting position of the elements in the source registers can be calculated by multiplying the value of #VecIdx U06 by 4 and the value of .num. As indicated by MAXBS.2, the instruction 700 selects two larger-sized elements from each 4-element data structure 720. The elements with values ​​3 and -7 are selected from R1, and the elements with values ​​6 and 5 are selected from R2. Since the starting position of the element in the source position is 8, the positions of the selected elements with values ​​'6' and '5' are '15' and '13', respectively, and the positions of the selected elements with values ​​'3' and '-7' are '10' and '9', respectively. Since the index size in the metadata is 4 bits (as indicated by .U4), the positions '11' and '9' are written as 'F' and 'D' in hexadecimal format in metadata 710, while '10' and '9' are written as 'A' and '9'. Since #MDidxU04 is zero, the position is written to the first 16 bits (4x4 bits) of the destination register R10.

[0061] Figure 8 Another example of a GENMETADATA instruction 800 is illustrated that generates metadata 810 from an input N-element data structure 820. In the example shown, N is 4, M is 2, and the example generates metadata 810 that indicates the location of 2 elements in the input 4-element data structure 820.

[0062] In the example shown, two 4-element data structures (64 bits total) are stored in two source registers R1 and R2, as indicated by .2G and R1 and R2. Since each element of the input data structure 820 is 8 bits (as indicated by .S8), each source register R1, R2 contains four 8-bit elements. Since the value of #VecIdxU06 is 2 and there is no .SEQ, each input data structure 820 starts at position 8 (2x4) of each source register. Since the value of #MDidxU04 is 4, the starting bit of the metadata 810 in the destination register is 16, which is calculated by multiplying the value of #MDidxU04 by the value of .idxsize.

[0063] As indicated by MAXABS.2, the example instruction 800 selects two elements of larger size from each input data structure 820. Therefore, elements with values ​​'3' and '-7' are selected from R1, and elements with values ​​'6' and '5' are selected from R2. Since the starting position of each input data structure 820 is 8, the positions of the selected elements with values ​​'6' and '5' in R2 are '11' and '9', respectively, and the positions of the selected elements with values ​​'3' and '-7' in R1 are '10' and '9', respectively. Since the index size in the metadata is 4 bits (as indicated by .U4), in the metadata 810, positions '11' and '9' are written as 'B' and '9' in hexadecimal format, while '10' and '9' are written as 'A' and '9'. When the offset is 16, the positions are written in bits 16-31 of the destination register R10.

[0064] Fig. 9 An example of a GENMETADATA instruction that generates metadata 910 from an input N-element data structure 912 is illustrated. In the example shown, N is 4, M is 1, and the example generates metadata 910 that indicates the location of 1 element in the input 4-element data structure 912. As indicated by .1G, R10, and R11, an input 4-element data structure (64 bits total) is stored in two source registers, R100 and R11. Since each element of the 4-element data structure 912 is 16 bits (as indicated by .F16), each source register contains 2 16-bit elements.

[0065] In the example shown, the instructions select and position an element for each pass (as indicated by the .1 after MAX), and use #PassU01 and #NibbleU03 instead of #VecIdxU06 and #MDidxU04, respectively. #PassU01 is a 1-bit immediate value indicating the number of passes and the presence of filtering / masking (0 indicates the first pass and no filtering), while #NibbleU03 is a 3-bit immediate value used to position the element used for filtering.

[0066] As indicated by #PassU01 being 0, the first instruction 914 selects an element with a maximum value ("5") and writes its position ("2") to metadata 916 in the first pass. The "0" of #nibbleU03 in the second instruction indicates that the value 2 at position 0 in source register R3 is the position of the element to be filtered from the source input data structure. Therefore, in the second pass, instruction 918 first filters / masks the element with a value of '5', which is located at position 2 in the 4-element data structure 912 of the remaining elements, and then the second instruction 918 selects the element with a maximum value of '4' and writes its position ('3') to metadata 910. The '3' will overwrite the '2' in the metadata because the value of #MDidxU04 is not indicated. At the end of the second pass, metadata 910 indicates the position of the element with the second largest value. It will be appreciated that the filtering process described can be modified, such as using a different function, and / or can be extended, such as using one or more additional passes, to filter out certain elements from the input data structure. It will also be appreciated that by setting #MDidxU04 to hold the position value from the previous pass, the filtering process described emulates the 2:4 GENMETADATA instruction, which is equivalent to executing 1:4

[0067] This is more expensive in terms of processing resources as GENMETADATA is called twice.

[0068] An example format of an example of a GATHER instruction is shown below.

[0069] GATHER.datasize.idxsize.num Rd,Ra,Rb,Rc,#VecIdxU06,#MDidxU04,

[0070] #DstByteU02, #SrcHalfU01

[0071] in:

[0072] .datasize: the bit width of the data elements in Ra, Rb, Rd, such as 16 bits, 8 bits, or 4 bits (.16, .18, .4); .idxsize: the size of the metadata index, such as 2 bits, 4 bits, or 8 bits (.U2, .U4, .U8);

[0073] .num: the number of input N-element data structures represented by Ra and Rb, such as 1 data structure, 2 data structures, or 4 data structures (.1G, .2G, .4G);

[0074] Rd: destination register or memory location;

[0075] Ra: the first source register or memory location containing all or part of the N-element data structure;

[0076] Rb: a second source register or memory location containing all or part of an N-element data structure;

[0077] Rc: a third source register or memory location containing metadata;

[0078] #VecIdxU06: Group index; a 6-bit immediate value that indicates the (starting) position of the N-element data structure in a larger data structure, for example, when the N-element data structure is one of multiple N-element data structures in a larger data structure.

[0079] #MDidxU04: metadata index; 4-bit immediate value that determines the offset (in bits) where the metadata is located in Rc;

[0080] #DstByteU02: 2-bit immediate value used to determine the location of the 16-bit or 8-bit word written in Rd when the total input data is 32 bits or 16 bits, respectively (1 group of int8, 2 groups of int4, or 1 group of int4). When the total input data is 32 bits, the value of this immediate value must be a multiple of 2. When the value of this immediate value is a multiple of 2, Ra itself contains all the input data, the collected data elements are 16 bits or 8 bits, and only part of Rd is written with this data; the rest of Rd is written with the contents of Rb; #SrcHalfU01: 1-bit immediate value used to determine whether to read from the lower half or the upper half of Ra when operating on a single group of int4 data elements.

[0081] Figure 10-Figure 14 An example of a GENMETADATA directive using the above format is provided.

[0082] Fig.10 An example of a GATHER instruction 1000 is illustrated for compressing an N-element data structure into an M-element data structure based on metadata generated using a metadata generation instruction (such as GENMETADATA described above). In the example shown, N is 4 and M is 2.

[0083] In the example shown, an input 4-element data structure 1010 (indicated by .1G) in source registers R1 and R2 is compressed into a 2-element data structure in destination register R15 according to metadata 1030 in source register R100. .16 indicates that each element in the input 4-element data structure 1010 is 16 bits.

[0084] Since each metadata index is 2 bits (as indicated by .U2), the first 2 values ​​1 0 (bit positions 3 and 2 in metadata 1030) represent position 2, and the second 2 values ​​0 1 (bit positions 1 and 0 in metadata 1030) represent position 1. Therefore, the "3" in position 2 and the "7" in position 1 of the input 4-element data structure 1010 are collected and written to the 2-element data structure 1020 in the destination register R15. In the example shown, #VecIdxU06, #MDidxU04, #DstByteU02, and #SrcHalfU01 are not specified.

[0085] Fig.11 Another example of a GATHER instruction that compresses an N-element data structure into an M-element data structure based on metadata generated using a metadata generation instruction (such as GENMETADATA described above) is illustrated. In the example shown, N is 4 and M is 2.

[0086] In the example shown, two input 4-element data structures 1110 in source registers R1 and R2 (as indicated by .2G) are compressed into two 2-element data structures 1120 in destination register R20 according to metadata 1130 in source register R100. .8 indicates that each element in the input 4-element data structure 1110 is 8 bits.

[0087] Since each metadata index is 2 bits (as indicated by .U2), the first 2 values ​​1 1 represent position 3 in R2, the next 2 values ​​0 1 represent position 1 in R2, the next 2 values ​​1 0 represent position 2 in R1, and the last 2 values ​​0 1 represent position 1 in R1. Therefore, the values ​​'6' and '5' in positions 3 and 1 of source register R2, and the values ​​'3' and '-7' in positions 2 and 1 of source register R1 are collected and written into two 2-element data structures 1120 in destination register R20. In the example shown, #VecIdxU06, ##MDidxU04,

[0088] #DstByteU02 and #SrcHalfU01.

[0089] Fig.12Another example 1200 of a GATHER instruction that compresses an N-element data structure into an M-element data structure based on metadata generated using a metadata generation instruction (such as GENMETADATA described above) is illustrated. In the example shown, N is 4 and M is 2. In the example shown, two input 4-element data structures 1210 (as indicated by .2G) in source registers R1 and R2 are compressed into two 2-element data structures 1220 in destination register R20 based on metadata 1230 in source register R100. .8 indicates that each element in the input 4-element data structures 1210 is 8 bits. Since #VecIdxU06 is 1, each input 4-element data structure 1210 starts at position 4 (1x4). Since #MDidxU04 is zero, there is no offset for the metadata 1230 in source register R100.

[0090] Since each metadata index is 4 bits (as indicated by .U4), the first value '7' of metadata 1230 represents position 7 in R2, the second value '5' represents position 5 in R2, the third value '6' represents position 6 in R1, and the fourth value '5' represents position 5 in R1. Therefore, the values ​​'6' and '5' in positions 7 and 5 of source register R2, and the values ​​'3' and '-7' in positions 6 and 5 of source register R1 are collected and written into two 2-element data structures 1220 in destination register R20. In the example shown, #DstByteU02 and #SrcHalfU01 are not specified.

[0091] Fig.13 Another example of a GATHER instruction that compresses an N-element data structure into an M-element data structure based on metadata generated using a metadata generation instruction (such as GENMETADATA described above) is illustrated. In the example shown, N is 4 and M is 2.

[0092] In the example shown, two input 4-element data structures 1310 (as indicated by .2G) stored in source registers R1 and R2 are compressed into two 2-element data structures 1320 in destination register R20 according to metadata 1330 stored in source register R100. .8 indicates that each element in the input 4-element data structures 1310 is 8 bits. Since #VecIdxU06 is 1, the elements in each input data structure start at position 4 (1x4), and since #MDidxU04 and .idxsize are both 4, the starting bit of metadata 1330 in the second source register is 16 (4x4).

[0093] Since each metadata index is 4 bits (as indicated by .U4), the first two values ​​'7' and '5' of metadata 1330 represent positions 7 and 5 in R2, and the second two values ​​'6' and '5' represent positions 6 and 5 in R1. Therefore, the values ​​'6' and '5' in positions 7 and 5 of source register R2, and the values ​​'3' and '-7' in positions 6 and 5 of source register R1 are collected and written into two 2-element data structures in destination register R20. In the example, #DstByteU02 and #SrcHalfU01 are not specified.

[0094] Fig.14 Another example of a GATHER instruction that compresses an N-element data structure into an M-element data structure based on metadata generated using a metadata generation instruction (such as GENMETADATA described above) is illustrated. In the example shown, N is 4 and M is 2.

[0095] In the example shown, the presence of #DstByteU02 indicates that source register R1 contains all of the input data, and the collected data is 16 bits. Additionally, the value of 2 in #DstByteU02 indicates that the collected data will be offset by 2 bytes or 16 bits in R20, and the remainder of R20 will be written with the contents of another source register R2. Thus, based on the metadata 1230 in source register R100, the two input 4-element data structures 1410 in source register R1 (as indicated by .2G) are compressed into two 2-element data structures 1320 in locations 16-31 of destination register R20. .4 indicates that each element in the input 4-element data structure 1410 is 4 bits.

[0096] Since each metadata index is 4 bits, the first two values ​​'3' and '1' represent positions 3 and 1 in group 1, and the second two values ​​'2' and '1' represent positions 2 and 1 in group 0. Therefore, the values ​​'6' and '5' in group 1 and the values ​​'3' and '-7' in group 0 are collected and written into two 2-element data structures in positions 16-31 of the destination register R20. The remainder of R20 is written with the contents of another source register R2. Since instruction 1400 processes two groups of 4-bit elements, #SrcHalfU01 is not used.

[0097] An example format of an example of a "super instruction" called a GATHERPLUS instruction is shown below.

[0098] GATHERPLUS{.mode}.func.selectN.fmt.idxsize.num{.SEQ}Rd1,Rd2,Ra,

[0099] Rb,Rc,#VecIdxU06,#MDidxU04,#DstByteU02,#SrcHalfU01

[0100] in:

[0101] .mode: optional mode indicating whether to generate metadata ("METADATA") or compressed data ("DATA"); if missing, both metadata and compressed data are generated;

[0102] .func: a function that controls how the values ​​of elements in an N-element data structure are compared to each other for selection (as described above, .func can be, for example, MAX, MAXABS, MIN, MINABS, etc.);

[0103] .selectN: The number of elements selected using .func, such as 1 or 2;

[0104] .fmt: a format for the elements in Ra and Rb, including integer formats such as signed 2-, 4-, 8-, 16-, and 32-bit integers (.S2, .S4, .S8, .S16, .S32) and unsigned 2-, 4-, 8-, 16-, and 32-bit integers (.U2, .U4, .U8, .U16, .U32), and non-integer formats such as IEEE half-precision (16-bit) or single-precision (32-bit) floating point (.FP16, .FP32), truncated versions of FP16 and FP32 (E8M7, E8M10), LNS (logarithmic system), POSITs (a proposed new dynamically scaled number format), other 8-bit floating point-like formats, and block exponent formats;

[0105] .idxsize: The size of the metadata index, such as 2 bits, 4 bits, or 8 bits (.U2, .U4, .U8)

[0106] .num: the number of N-element data structures entered in the source storage, such as 1 data structure, 2 data structures, or 4 data structures (.1G, .2G, .4G);

[0107] .SEQ: Optional sequential indexing: When operating on more than one N-element data structure, all N-element data structures are indexed sequentially, as if they were part of a single larger data structure, rather than repeating the same N index for each N-element data structure;

[0108] Rd1: first destination register or memory location;

[0109] Rd2: second destination register or memory location;

[0110] Ra: the first source register or memory location containing all or part of the N-element data structure;

[0111] Rb: A second source register or memory location containing all or part of an N-element data structure; Rc: An optional source register or memory location; When GENMETADATA specifies .idxsize and .num so that not all bits of Rd1 or Rd2 are written, the contents of Rc flow into the remaining unwritten bits of Rd1 or Rd2;

[0112] #VecIdxU06: group index; a 6-bit immediate value indicating the (starting) position of the N-element data structure in a larger data structure, for example, when the N-element data structure is one of multiple N-element data structures in a larger data structure;

[0113] #MDidxU04: metadata index; 4-bit immediate value that determines the offset (in bits) where the metadata is located in Rc;

[0114] #DstByteU02: 2-bit immediate value used to determine the location of the 16-bit or 8-bit word written in Rd when the total input data is 32 bits or 16 bits, respectively (1 group of int8, 2 groups of int4, or 1 group of int4). When the total input data is 32 bits, the value of this immediate value must be a multiple of 2. When the value of this immediate value is a multiple of 2, Ra itself contains all the input data, the collected data elements are 16 bits or 8 bits, and only part of Rd is written with this data; the rest of Rd is written with the contents of Rb; #SrcHalfU01: 1-bit immediate value used to determine whether to read from the lower half or the upper half of Ra when operating on a single group of int4 data elements.

[0115] As described above, a superinstruction is a combination of a metadata generation instruction (eg, GENMETADATA) and a data compression instruction (eg, GATHER). A superinstruction includes all common fields / parameters of the combined instructions, as well as two registers / memory locations for the metadata and compressed data.

[0116] One major difference between a superinstruction and the two separate instructions described above is that a superinstruction generates (e.g., writes) two sets of outputs, metadata and compressed data. These two sets of outputs can, for example, be 1) generated in parallel and / or 2) generated sequentially. When generated in parallel, the superinstruction can write to two different register / memory locations at the same time. When generated sequentially, the superinstruction can write to one location at a time, so the operations (metadata generation and data compression) can be performed one after the other. This means that instructions have internal state.

[0117] Fig.15An example of a GATHERPLUS instruction 1300 is illustrated that generates metadata from an N-element data structure and compresses the N-element data structure into an M-element data structure. In the example shown, N is 4 and M is 2.

[0118] In the first operation of instruction 1300, the positions of two elements of each of the 4-element data structure 1520 (as indicated by .2G) from the two inputs in source registers R1 and R2 are written to the metadata 1510 in the destination register R10. More specifically, during the first operation of instruction 1500, the positions of the two elements with the larger size (as indicated by MAXABS.2) of each of the 4-element data structure 1520 from the two inputs in source registers R1 and R2, i.e., positions '3' and '1' of the elements with values ​​'6' and '5' in R2 and positions '2' and '1' of the elements with values ​​'3' and '-7' in R1 are written to the metadata 1510 in the destination register R10.

[0119] In another operation of instruction 1300, the values ​​of the two elements with larger size from each of the two input 4-element data structures 1520, namely the values ​​'6' and '5' of the elements in R2, and the values ​​'3' and '-7' of the elements in R1, are collected and written into two 2-element data structures 1530 in the destination register R20. These two operations can be performed in parallel or sequentially. Although in the example shown, #VecIdxU06 and #MDidxU04 are both zero, they can be set to indicate offsets in various registers / memory locations, similar to the GENMETADATA and GATHER instructions.

[0120] Therefore, positions '3' and '1' of elements with values ​​'6' and '5' in R2 and positions '2' and '1' of elements with values ​​'3' and '-7' in R1 are written to the first 16 bits (4x4 bits) of metadata 1510 in destination register R10. At the same time, elements with values ​​'6' and '5' in R2 and elements with values ​​'3' and '-7' in R1 are collected and written to two 2-element data structures 1530 in destination register R20.

[0121] Figure 16-Figure 21 An example of cascading GENMETADATA and GATHER instructions to achieve a 2:8 sparse / compressed ratio is shown. Fig.16 As shown, the example shown compresses the original data (composed of two 4-element data structures 1602 and 1604 ) into compressed data, a 2-element data structure 1620 , which has two elements with values ​​of '12' and '7.' The 2-element data structure 1620 will have metadata 1630 .

[0122] exist Fig.17 , metadata 1730 is generated from two input 4-element data structures 1602, 1604. First, using the first GENMETADATA instruction 1710, the position of the element with the larger value in the input 4-element data structure 1602 is determined and written to the metadata 1730. Since the values ​​of #VecIdxU06 and #MDidxU04 are both zero in the first GENMETADATA instruction 1710, the position of the element with the larger value in the input 4-element data structure 1602, '2' and '1' (which starts at bit 0 (of the larger data structure), is stored in bits 0-7 of the metadata 1730. Second, using the second GENMETADATA instruction 1720, the position of the element with the larger value in the input 4-element data structure 1604 is determined and written to the metadata 1730. In the second GENMETADATA instruction, the values ​​of #VecIdxU06 and #MDidxU04 are set to '1' and '2', respectively. Therefore, positions '7' and '6' (starting from the 4th position) of the elements having larger values ​​in the input 4-element data structure 1604 are stored in positions 8 to 15 of the metadata 1730 .

[0123] exist Fig.18 , the values ​​of the elements with larger values ​​in the input 4-element data structures 1602, 1604 are gathered into two 2-element data structures 1830, 1840 using metadata 1730. First, the values ​​of the elements with larger values ​​in the input 4-element data structure 1602 are gathered into the first 2-element data structure 1830 using the first GATHER instruction 1810. Since the values ​​of #VecIdxU06 and #MDidxU04 are both zero in the first GATHER instruction 1810, the values ​​'2' and '1' are gathered from the first input 4-element data structure 1602 starting at position 0 (of the larger data structure) using positions '2' and '1' of bits 0-7 of metadata 1730. Second, the values ​​of the elements with larger values ​​in the input 4-element data structure 1604 are gathered into the second 2-element data structure 1840 using the second GATHER instruction 1820. In the second GATHER instruction 1820, the values ​​of #VecIdxU06 and #MDidxU04 are set to '1' and '2' respectively. Therefore, the values ​​'7' and '6' at positions '7' and '6' of bits 8-15 of the metadata 1730 are collected from the input 4-element data structure 1604, which starts at position 4 (of the larger data structure).

[0124] exist Fig.19In the example, by executing another GENMETADATA instruction 1910, the position of the element with the larger value in the two 2-element data structures 1830, 1840 is determined and written to the metadata 1730. Since the value of #MDidxU04 is '4', the positions '2' and '1' of the elements with values ​​'12' and '7' are offset by 16 bits and written to bits 16-23 of the metadata 1730.

[0125] exist Fig. 20 In the example, by executing another GATHER instruction 2010, the values ​​of the elements with larger values ​​in the two 2-element data structures 1830 and 1840 are collected and written into the 2-element data structure 2020. The value of #MDidxU04 is set to '4', so that the positions '2' and '0' at bits 16-23 of the metadata 1730 are used to collect the values ​​'12' and '7' of the corresponding elements. The 2-element data structure 2020 is the same as the final compressed data, that is, Fig.16 The 2-element data structure 1620 in .

[0126] exist Fig.21 In the example, metadata 1730 is treated as an input data structure and metadata for another GATHER instruction 2110. Since the value of #MDidxU04 is '4', the positions at bits 16-23 of metadata 1730 are used to collect the values ​​of the corresponding elements. Therefore, the data structure 2120 generated by GATHER instruction 2110 is the same as Fig.16 It should be understood that Figure 15-Figure 21 The instructions shown can be repeated as needed to achieve higher degrees of sparsity. For example, the metadata generation and then compression of four 4-element data structures are repeated until only 2 elements remain ( Figure 17-20 ), and then generating the final metadata for the remaining 2 elements, can create a 2:16 sparsity / compression ratio.

[0127] It will be appreciated that each of the GENMETADATA and GATHER instructions in the illustrated example may be replaced with a superinstruction, such as GATHERPLUS, by setting the optional mode parameter to match the data being generated. For example, to replace the GENMETADATA instruction, the mode parameter of the GATHERPLUS instruction is set to "METADATA", and to replace the GATHER instruction, the mode parameter of the GATHERPLUS instruction is set to "DATA".

[0128] Parts of the above-mentioned apparatus, system or method may be embodied in or performed by various digital data processors or computers, wherein the computer is programmed or stores an executable program of a sequence of software instructions to perform one or more steps of the method. The software instructions of these programs may represent algorithms and are encoded in a machine-executable form on a non-transitory digital data storage medium, such as a magnetic disk or optical disk, a random access memory (RAM), a magnetic disk, a flash memory and / or a read-only memory (ROM), so that various types of digital data processors or computers can perform one, more or all steps of one or more of the above-mentioned methods or functions, systems or apparatuses described herein.

[0129] Portions of the disclosed embodiments may involve a computer storage product with a non-transitory computer-readable medium having program code for performing various computer-implemented operations, which comprises a portion of a device, apparatus, or steps for performing the methods described herein. As used herein, non-transitory refers to all computer-readable media except transient propagating signals. Examples of non-transitory computer-readable media include, but are not limited to: magnetic media such as magnetic disks, floppy disks, and tapes; optical media such as CD-ROM disks; magneto-optical media such as soft magnetic optical disks; and hardware devices specially configured for storing and executing program code, such as ROM and RAM devices. Examples of program code include machine code (such as code generated by a compiler) and files containing high-level code (which can be executed by a computer using an interpreter).

[0130] In interpreting this disclosure, all terms should be interpreted in the broadest possible manner consistent with the context. In particular, the terms "comprise" and "include" should be interpreted as referring to elements, components or steps in a non-exclusive manner, indicating that the referenced elements, components or steps may be present, used, or used in combination with other elements, components or steps not explicitly mentioned.

[0131] Those skilled in the art to which this application relates will appreciate that other and further additions, deletions, substitutions and modifications may be made to the embodiments. It should also be understood that the terms used herein are only used to describe specific embodiments and are not intended to be limiting, as the scope of the present disclosure is limited only by the claims. Unless otherwise defined, the meanings of all technical and scientific terms used in the present invention are the same as those generally understood by those of ordinary skill in the art to which the present invention belongs. Although any methods and materials similar or equivalent to the methods and materials described herein may also be used in the practice or testing of the present invention, a limited number of exemplary methods and materials are described herein.

Claims

1. A method for compressing data, comprising: Generating metadata from an N-element data structure, the generating comprising: Selecting M elements from the N-element data structure, where N is greater than M; Determining the positions of the M elements in the N-element data structure; and storing the positions of the M elements as the metadata; and Compressing the N-element data structure into an M-element data structure according to the metadata, the compressing comprising: collecting the values ​​of the M elements from the N-element data structure according to the positions; and The values ​​of the M elements are stored in the M-element data structure, wherein: The generating is performed using a first instruction; and The generating includes receiving the N-element data structure as input to the first instruction.

2. The method of claim 1, wherein said generating comprises: A group index in the first instruction is used to determine a starting position of the N-element data structure within a larger data structure.

3. The method of claim 1, wherein said generating further comprises: A metadata index in the first instruction is used to determine a starting location of the metadata in a destination register. The method of claim 1 , wherein the first instruction is a GENMETADATA instruction.

5. The method of claim 1, wherein the selecting is based on a function in the first instruction that compares values ​​of N elements in the N-element data structure to each other. The method according to claim 5 , wherein the function is a MAX function or a MAXABS function. The method of claim 1 , wherein the compressing is performed using a second instruction.

8. The method of claim 7, wherein the compressing comprises: The group index in the second instruction is used to determine the starting position of the N-element data structure within a larger data structure.

9. The method of claim 7, wherein the compressing comprises: The metadata index in the second instruction is used to determine the starting position of the metadata in the source register.

10. The method of claim 7, wherein the compressing comprises: The metadata and the N-element data structure are received as input to the second instruction. The method of claim 7 , wherein the second instruction is a GATHER instruction.

12. The method of claim 1, wherein: The N-element data structure is an N-element vector, and the M-element data structure is an M-element vector; or The N-element data structure is an N-element tensor, and the M-element data structure is an M-element tensor. The method according to claim 1 , wherein M is 2 and N is 4.

14. The method according to claim 1, further comprising: Repeat the generating and the compressing to compress the N-element data structure into a P-element data structure, where P is less than M.

15. The method of claim 1, wherein said generating and said compressing are performed using a single instruction.

16. A system for compressing data, comprising: Memory; and at least one processor communicatively coupled to the memory, the at least one processor configured to: Generate metadata from an N-element data structure by following these steps: Selecting M elements from the N-element data structure, where N is greater than M; Determining positions of the M elements in the N-element data structure; as well as storing the positions of the M elements as the metadata; as well as According to the metadata, the N-element data structure is compressed into an M-element data structure by the following steps: collecting values ​​of the M elements from the N-element data structure according to the positions; storing the values ​​of the M elements in the M-element data structure; wherein: The at least one processor is configured to generate the metadata by executing a first instruction; and The at least one processor is further configured to receive the N-element data structure as input to the first instruction.

17. The system of claim 16, wherein the first instruction is a GENMETADATA instruction.

18. The system of claim 16, wherein said storing said location comprises: A metadata index in the first instruction is used to determine a starting location of the metadata in a destination register.

19. The system of claim 16, wherein the determining comprises: A group index in the first instruction is used to determine a starting position of the N-element data structure within a larger data structure.

20. The system of claim 16, wherein the selection is based on a function in the first instruction that compares values ​​of N elements in the N-element data structure to each other.

21. The system of claim 20, wherein the function is a MIN function or a MINABS function.

22. The system of claim 16, wherein the at least one processor is configured to compress the N-element data structure by executing a second instruction.

23. The system of claim 22, wherein said collecting comprises: The group index in the second instruction is used to determine the starting position of the N-element data structure within a larger data structure.

24. The system of claim 22, wherein said collecting comprises: The metadata index in the second instruction is used to determine the starting position of the metadata in the source register.

25. The system of claim 22, wherein the at least one processor is further configured to: receive the metadata and the N-element data structure as input to the second instruction.

26. The system of claim 22, wherein the second instruction is a GATHER instruction.

27. The system of claim 16, wherein M is 2 and N is 4.

28. The system of claim 16, wherein: The N-element data structure is an N-element vector, and the M-element data structure is an M-element vector; or The N-element data structure is an N-element tensor, and the M-element data structure is an M-element tensor.

Images