A sparse identification scheduling method, device, apparatus and medium

Abstract

The embodiment of the application provides a sparse identification scheduling method, device, equipment and medium, which has the following beneficial effects: a sparse matrix is obtained; the sparse matrix is divided into multiple identification blocks in units of preset data length, when all elements in the identification block are zero, the identification block is determined as sparse; when there is at least one non-zero element in the identification block, the identification block is determined as valid; the determination result is stored in a memory space independent of the sparse matrix in units of bit to form sparse mask data; in units of a main block, the sparse matrix and the sparse mask data are loaded into a register stack, the register stack includes a head register and a data register, and is used for performing sparse matrix and weight matrix multiplication calculation. The application can efficiently judge the sparse block in a unified granularity under multiple data formats, jointly encode the sparse bitmap and the reconstruction parameter, thereby improving the scheduling efficiency of sparse activation and reducing the storage and bandwidth overhead.

Description

Technical Field

[0001] This invention relates to the field of computer architecture, and more specifically to a sparse identification and scheduling method, apparatus, device, and medium. Background Technology

[0002] As the scale of deep neural networks continues to expand, the computation and storage overhead of models increases significantly, making sparsification acceleration technology an important direction for improving operating efficiency.

[0003] Current mainstream research and applications largely focus on weight sparsity, especially structured sparse pruning, which has been accelerated on multiple hardware platforms, such as NVIDIA GPUs' 2:4 sparsity mechanism. NVIDIA GPUs have introduced native support for 2:4 structured weight sparsity in their Ampere architecture. This mechanism requires that only two non-zero values ​​be retained in every four consecutive weight elements (i.e., two effective weights and two set to zero), forming a fixed pattern of structured sparse blocks. This sparsity requires the weight matrix to be generated under forced constraints through a pruning algorithm, enabling efficient skipping of zero-value operations during hardware computation. Although 2:4 structured sparsity can achieve a certain degree of inference acceleration with hardware support, this method still has several limitations. First, its sparsity is fixed at 50%, meaning that exactly two out of every four weight elements are non-zero, lacking the ability to adaptively adjust to the sparsity distribution of the model itself. Second, this technique only applies to static weight tensors and cannot utilize the activation sparsity that is widespread during inference. Activation sparsity is usually dynamically generated by input, and its location is not fixed, making it difficult for the current 2:4 mechanism to recognize and accelerate.

[0004] In contrast, while activation sparsity is widespread in the outputs of nonlinear units such as ReLU, its dynamic and irregular nature currently limits effective hardware support and general acceleration solutions. The potential computational savings inherent in activation sparsity have not been fully explored, becoming a weak link in existing sparse computing systems. ProSparse proposes a method to transform language models that widely employ non-ReLU activation functions (such as Swish or GELU) into ReLU functions with high activation sparsity, further enhancing sparsity through the design of the loss function. This method generates element-level sparse activation matrices, ensuring accuracy while achieving significant inference acceleration. Although ProSparse has achieved significant results in improving activation sparsity, its overall strategy still has certain limitations. First, ProSparse is mainly based on unstructured sparse patterns at the single-element level, making sparse distribution difficult to predict or compress, resulting in low utilization of hardware matrix operation units, especially in SIMD architectures such as GPUs, where it is difficult to fully leverage the sparsity potential. Second, ProSparse does not fully utilize spatial locality or structural sparsity features, leading to limited space for actual storage compression and memory access optimization. Especially in small batch or low parallel scenarios, there is a lack of higher-level block sparse scheduling and coding mechanisms, resulting in redundant data access to zero elements.

[0005] As can be seen from the above description, current sparse computing, especially for sparse scheduling, suffers from low efficiency and high storage and bandwidth overhead. How to improve sparse scheduling efficiency and reduce overhead is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] To overcome the shortcomings of existing sparse scheduling methods, such as low efficiency and high overhead, this invention proposes a sparse identification and scheduling method, apparatus, device, and medium.

[0007] To achieve the above objectives, according to a first aspect of the present invention, an embodiment of the present invention provides a sparse identification and scheduling method, the method comprising the following steps:

[0008] Obtain a sparse matrix, which is arranged continuously in memory in units of combinatorial blocks, wherein each combinatorial block includes multiple main blocks and each main block includes multiple sub-blocks;

[0009] The sparse matrix is ​​divided into multiple recognition blocks with a preset data length. When all elements in a recognition block are zero, the recognition block is judged as sparse. When there is at least one non-zero element in the recognition block, the recognition block is judged as valid. The judgment results are stored continuously in memory space independent of the sparse matrix in bits to form sparse mask data.

[0010] The sparse matrix and sparse mask data are loaded into a register file, which includes a header register and a data register, on a main block basis. This register file is used to perform sparse matrix and weight matrix multiplication calculations, including:

[0011] When the sparse matrix storage data only includes the data part, the data part is loaded into the data register in units of sub-blocks. The header register is divided into a corresponding header storage area for each sub-block, and the sparse mask data of each sub-block is loaded into the corresponding header storage area in sequence.

[0012] When the sparse matrix stores data including a header and a data portion, the data portion is loaded into the data register sequentially in units of sub-blocks. The header register is divided into a corresponding header storage area for each sub-block. After the header storage area is divided, the header portion and sparse mask data corresponding to the sub-block are loaded respectively.

[0013] Optionally, dividing the sparse matrix into multiple recognition blocks based on a preset data length includes:

[0014] When the element bit width of the sparse matrix storage data is a power of 2, the preset first data length is 1×32 bytes, and the sparse matrix is ​​divided into multiple recognition blocks according to the first data length.

[0015] When the element width of the sparse matrix storage data is not a power of 2, the second data length is preset to 1×48 bytes, and the sparse matrix is ​​divided into multiple recognition blocks according to the second preset data length.

[0016] Optionally, when the sparse matrix storage data only includes the data portion, the data portion is loaded into the data register sequentially in sub-blocks. A corresponding header storage area is allocated for each sub-block in the header register, and the sparse mask data of each sub-block is loaded into the corresponding header storage area sequentially, including:

[0017] The sparse mask data of the sub-block is loaded first into the high bits of the corresponding header storage area, and the remaining bits of the header storage area are left empty.

[0018] Optionally, when the sparse matrix storage data includes a header and a data portion, the data portion is loaded sequentially into the data register in units of sub-blocks. The header register is divided into corresponding header storage areas for each sub-block. After partitioning the header storage areas, the header portion and sparse mask data corresponding to each sub-block are loaded separately, including:

[0019] The sparse mask data of the sub-block is loaded first into the high-bit portion of the corresponding header storage area, and the header portion of the sub-block is loaded first into the low-bit portion of the corresponding header storage area. The remaining bits in the header storage area are left empty.

[0020] According to a second aspect of the present invention, embodiments of the present invention also provide a sparse identification scheduling apparatus, comprising:

[0021] The acquisition module is used to acquire a sparse matrix, which is arranged continuously in memory in units of combinatorial blocks, wherein each combinatorial block includes multiple main blocks and each main block includes multiple sub-blocks;

[0022] The judgment module is used to divide the sparse matrix into multiple recognition blocks with a preset data length. When all elements in the recognition block are zero, the recognition block is judged as sparse; when there is at least one non-zero element in the recognition block, the recognition block is judged as valid. The judgment result is stored continuously in memory space independent of the sparse matrix in bits to form sparse mask data.

[0023] The loading module is used to load sparse matrix and sparse mask data into a register file, which includes a header register and a data register, on a main block basis. This register file is used to perform sparse matrix and weight matrix multiplication calculations, including:

[0024] When the sparse matrix storage data only includes the data part, the data part is loaded into the data register in units of sub-blocks. The header register is divided into a corresponding header storage area for each sub-block, and the sparse mask data of each sub-block is loaded into the corresponding header storage area in sequence.

[0025] When the sparse matrix stores data including a header and a data portion, the data portion is loaded into the data register sequentially in units of sub-blocks. The header register is divided into a corresponding header storage area for each sub-block. After the header storage area is divided, the header portion and sparse mask data corresponding to the sub-block are loaded respectively.

[0026] Optionally, the judgment module divides the sparse matrix into multiple recognition blocks based on a preset data length, including:

[0027] When the element bit width of the sparse matrix storage data is a power of 2, the preset first data length is 1×32 bytes, and the sparse matrix is ​​divided into multiple recognition blocks according to the first data length.

[0028] When the element width of the sparse matrix storage data is not a power of 2, the second data length is preset to 1×48 bytes, and the sparse matrix is ​​divided into multiple recognition blocks according to the second preset data length.

[0029] Optionally, when the sparse matrix storage data only includes the data portion, the loading module loads the data portion sequentially into the data register in units of sub-blocks. In the header register, a corresponding header storage area is allocated for each sub-block, and the sparse mask data of each sub-block is sequentially loaded into the corresponding header storage area, including:

[0030] The sparse mask data of the sub-block is loaded first into the high bits of the corresponding header storage area, and the remaining bits of the header storage area are left empty.

[0031] Optionally, when the sparse matrix storage data includes a header and a data portion, the loading module loads the data portion sequentially into the data register in units of sub-blocks. The header register is divided into corresponding header storage areas for each sub-block. After partitioning the header storage areas, the header portion and sparse mask data corresponding to each sub-block are loaded separately, including:

[0032] The sparse mask data of the sub-block is loaded first into the high-bit portion of the corresponding header storage area, and the header portion of the sub-block is loaded first into the low-bit portion of the corresponding header storage area. The remaining bits in the header storage area are left empty.

[0033] According to a third aspect of the present invention, embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the sparse identification scheduling method as described in any of the above embodiments.

[0034] According to a fourth aspect of the present invention, embodiments of the present invention also provide a storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the steps of the sparse identification scheduling method as described in any of the above embodiments.

[0035] As described above, the sparse identification scheduling method, apparatus, device, and medium provided by the embodiments of the present invention have the following beneficial effects: A sparse matrix is ​​obtained, which is continuously arranged in memory in units of combined blocks, wherein each combined block includes multiple main blocks, and each main block includes multiple sub-blocks; the sparse matrix is ​​divided into multiple identification blocks in units of a preset data length; when all elements in an identification block are zero, the identification block is determined to be sparse; when at least one non-zero element exists in an identification block, the identification block is determined to be valid; the determination result is continuously stored in a memory space independent of the sparse matrix in units of bits to form sparse mask data; the sparse matrix and... Sparse mask data is loaded into a register file, which includes a header register and a data register, for performing sparse matrix and weight matrix multiplication calculations. This includes: when the sparse matrix storage data only includes the data portion, the data portion is loaded sequentially into the data register in sub-blocks; a corresponding header storage area is partitioned in the header register for each sub-block; and the sparse mask data for each sub-block is loaded sequentially into the corresponding header storage area. When the sparse matrix storage data includes both a header and a data portion, the data portion is loaded sequentially into the data register in sub-blocks; a corresponding header storage area is partitioned in the header register for each sub-block; and after partitioning the header storage area, the header portion and sparse mask data corresponding to each sub-block are loaded separately. This invention establishes a unified granularity activation sparse format: it proposes an activation sparse judgment granularity based on 1×32 bytes. This granularity can achieve element boundary alignment under mainstream data formats such as FP8, FP16, and INT4, while also being compatible with non-power-two data formats such as MXFP6. It adapts to multi-precision scenarios and precisely matches the width of the computing array, contributing to efficient hardware scheduling and computation. Furthermore, by employing a bitmask compression encoding method suitable for activation tensors, and a block-level compression approach based on bitmasks, this invention can adapt to dynamically generated sparse activation structures. It also stores the activation data, along with the scaling factor present in the MX data format, in the on-chip register file, achieving joint storage, unified loading, and skip scheduling of activation data, thus improving the utilization rate of sparse activation during the execution phase. In addition, this invention features an activation-sparse-friendly storage and cache layout structure. Addressing the skip execution requirement of sparse activation, it constructs a memory layout that physically separates the mask and data, and uniformly stores the bitmask and scaling factor in the on-chip cache, further improving access efficiency. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating a sparse identification and scheduling method provided in an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of sparsity determination provided by an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the memory arrangement of sparse matrix and sparse mask data provided in an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of register file data arrangement provided in an embodiment of the present invention;

[0040] Figure 5 This is a diagram of a sparse matrix multiplication computation architecture provided by an embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram of the structure of a sparse identification and scheduling device provided in an embodiment of the present invention;

[0042] Figure 7 This is a schematic diagram of the hardware structure of an electronic device for executing a sparse identification and scheduling method provided in an embodiment of the present invention. Detailed Implementation

[0043] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0044] Please see Figures 1 to 7 It should be noted that the illustrations provided in this embodiment are only schematic representations of the basic concept of the present invention. Therefore, the illustrations only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0045] See Figure 1 This is a flowchart illustrating a sparse identification and scheduling method provided in an embodiment of the present invention, as shown below. Figure 1 As shown in the figure, the embodiment of the present invention illustrates the flow of the sparse identification and scheduling method.

[0046] Step S101: Obtain a sparse matrix, which is arranged continuously in memory in units of combined blocks, wherein each combined block includes multiple main blocks and each main block includes multiple sub-blocks.

[0047] First, this invention supports matrix calculations in multiple data formats. In specific implementations, the data format for sparse matrices can include only the data portion, identified as Non-MX, such as FP8 (corresponding to an element width of 8 bits, i.e., 1 data element occupies 8 bits of storage space), FP16 (corresponding to an element width of 16 bits, i.e., 1 data element occupies 16 bits of storage space), and INT4 (corresponding to an element width of 4 bits, i.e., 1 data element occupies 4 bits of storage space). Alternatively, the data format of a sparse matrix can also include a header and a data portion. The MX data format belongs to this type of data; in addition to the data portion, every 32 elements also correspond to a 1-bit MXScale (i.e., the header). Specifically, MX8 / MX6 / MX4 correspond to element widths of 8 bits, 6 bits, and 4 bits, respectively.

[0048] Furthermore, in this embodiment of the invention, the sparse matrix is ​​arranged in memory in units of combined blocks (i.e., SuperTile). One combined block further includes multiple main blocks (i.e., Tiles), and one main block further includes multiple sub-blocks (i.e., SubTile). The number of main blocks included in a combined block and the number of sub-blocks further included in a main block can be configured as needed according to specific data format requirements and element bit widths, etc., and this embodiment of the invention does not impose any limitations. In an exemplary implementation, for Non-MX data and MX4 / 8 data, one SuperTile may include 4 Tiles, and one Tile further includes 4 SubTiles; for MX6 data, one SuperTile may include 4 Tiles, and one Tile further includes 8 SubTiles.

[0049] Step S102: Divide the sparse matrix into multiple recognition blocks in units of preset data length. When all elements in a recognition block are zero, the recognition block is determined to be sparse. When there is at least one non-zero element in a recognition block, the recognition block is determined to be valid. The determination results are stored continuously in memory space independent of the sparse matrix in units of bits to form sparse mask data.

[0050] This invention first preprocesses the input sparse matrix, mainly including two core parts: a unified granularity partitioning mechanism and a bitmask compression encoding mechanism. See also... Figure 2 This is a schematic diagram of sparsity determination provided by an embodiment of the present invention, and is combined with Figure 3 This is a schematic diagram of the memory arrangement of sparse matrix and sparse mask data provided in an embodiment of the present invention.

[0051] In the first implementation, when the element bit width of the sparse matrix storage data is a power of 2, the preset first data length is 1×32 bytes, and the sparse matrix is ​​divided into multiple recognition blocks according to the first data length. For example... Figure 2 The diagram illustrates the granular partitioning and encoding process for two data formats (FP16 and FP8), with element bit widths of 16 and 8 respectively (both powers of 2). In an exemplary embodiment, the first data length is configured as 1×32 bytes, and the sparse matrix is ​​partitioned into rows of 1×32 bytes each. Specifically, a 1×32-byte data length contains 32 FP8 elements at FP8 precision, 16 elements at FP16 precision, and 64 elements (not shown) at INT4 precision. Figure 2 The diagram shows the result of partitioning one main block (1024B). Thus, for the FP16 data format, the data of one main block is divided into 32 identification blocks (32 rows in the diagram), each row contains 16 elements, and each element occupies 2B of storage space, satisfying a data length of 1×32B. For the FP8 data format, the data of one main block is also divided into 32 identification blocks (32 rows in the diagram), each row contains 32 elements, and each element occupies 1B of storage space, satisfying a data length of 1×32B.

[0052] Continuing, this invention performs sparsity determination on each 1×32B data length identification block obtained from the above division. If all elements in the identification block are zero, it is marked as "sparse," i.e., a sparse block; if at least one non-zero element exists, it is marked as "valid," i.e., a valid block. In specific implementation, the sparsity determination results of all identification blocks are encoded into a bitmap (bitmask shown in the figure), where each bit corresponds to an identification block of 1×32B data length. For example, a bit of 1 indicates that the block is a valid block and needs to participate in subsequent calculations; a bit of 0 indicates that the block is all zeros and is a sparse block, which can be skipped in the calculation stage. Finally, all the bitmaps together constitute the sparse mask data corresponding to the sparse matrix, and the storage space of this sparse mask data is independent of the storage space of the sparse matrix.

[0053] It should be noted that the above bit map encoding method can also be to use 0 to represent valid blocks and 1 to represent sparse blocks, or the bits can be numbers or symbols in other bases, etc. Any method that uses bits to identify a data block of a certain length should fall within the protection scope of this invention.

[0054] In the second implementation, when the element bit width of the sparse matrix stored data is not a power of 2, a preset second data length of 1×48 bytes is used to divide the sparse matrix into multiple identification blocks based on this preset data length. For specific division methods and bitmap encoding methods similar to those in the above embodiments, please refer to the description of the above embodiments.

[0055] Furthermore, in the MX format, every 32 elements share a Shared Scale (8 bits). The sparsity determination process in this embodiment of the invention also applies, taking into account both the Data and MXScale in the MX format. For the Data portion, each block can be composed of 1×32B elements (1×32 in MX8bit, 1×64 in MX4bit) or 1×48B (1x64) elements in MX6bit. If it is all zeros, the corresponding Data is all zeros, and the corresponding MXScale (1 MXscale = 1B in MX8bit, 2 MXscale = 2B in MX6bit and MX4bit) is also zero.

[0056] By unifying the data length as the block size, the embodiments of the present invention can achieve consistent sparse judgment logic across data precision and ensure that the width of the block is fully aligned with the accumulation unit of the computing array, thus avoiding overhead such as cross-block fusion and rewriting the accumulation path.

[0057] Figure 3 The arrangement of the sparse matrix and the encoded sparse mask data in memory is further illustrated, such as... Figure 3 As shown, the memory layout includes three cases: MX8bit / MX4bit, MX6bit, and Non-MX data formats.

[0058] For data formats MX8bit / MX4bit, the sparse tiles are arranged in memory as follows: One main block (SuperTile0 in the diagram) comprises four main blocks. Since the data is in MX format (including header and data portions), each combined block includes a data portion (Tile 0 Data, Tile 1 Data, Tile 2 Data, and Tile 3 Data in the diagram) and a header portion (Tile 0 MXScale, Tile 1 MXScale, Tile 2 MXScale, and Tile 3 MXScale). Both are arranged sequentially from their respective base addresses in ascending order. Each main block further includes four sub-blocks; that is, the main header of each main block includes the sub-headers of four sub-blocks (not shown in the diagram), and the main data portion of each main block includes the sub-data portions of four sub-blocks (SubTile 0 Data, SubTile 1 Data, SubTile 2 Data, and SubTile 3 Data in the diagram). Each sub-data portion occupies 256 bytes of storage space. This allows us to calculate that the storage space occupied by the data portion of one SuperTile is 256B × 4 (one main block includes 4 sub-blocks) × 4 (one composite block includes 4 main blocks) = 4KB, and the storage space occupied by the header portion (MXScale) of one SuperTile is 16B (one sub-header portion occupies 16B) × 4 (one main block includes 4 sub-blocks) × 4 (one composite block includes 4 main blocks) = 256B. According to the above embodiment, the process of dividing the sparse matrix and then encoding it to form sparse mask data involves storing the sparse mask data (Sparse Mask in the figure) centrally in an independent, continuous region, without mixing it with the corresponding sparse matrix data. This simplifies the mask access logic and facilitates hardware pre-loading or compressed transmission. Specifically, the data portion of one Tile is 256B×4=1KB, so its corresponding sparse mask data occupies 1KB / (32B*8)=4B of storage space; the sparse mask data of one SuperTile occupies 4×4B=16B of storage space. In this way, the sparse mask data of each SuperTile is arranged in an independent memory space according to the corresponding order of the Tile, and finally arranged in the order of SuperTile to form the sparse mask data arrangement corresponding to the entire sparse matrix.

[0059] Similarly, for the Non-MX data format, since it only includes the data portion, one composite block (SuperTile 0 in the diagram) comprises four main blocks (Tile 0 Data, Tile 1 Data, Tile 2 Data, and Tile 3 Data in the diagram), and each main block comprises four sub-blocks (SubTile 0 Data, SubTile 1 Data, SubTile 2 Data, and SubTile 3 Data in the diagram). Each sub-block occupies 256 bytes of storage space. Thus, the storage space occupied by one SuperTile is 256 bytes × 4 (one main block includes four sub-blocks) × 4 (one composite block includes four main blocks) = 4 KB, and the storage space occupied by one Tile is 256 bytes × 4 = 1 KB. Therefore, the corresponding sparse mask data occupies 1 KB / (32 bytes * 8) = 4 bytes, and the sparse mask data of one SuperTile occupies 4 × 4 bytes = 16 bytes.

[0060] For the MX6-bit data format, a composite block (SuperTile0 in the diagram) also comprises four main blocks. Each composite block includes a data portion (Tile 0 Data, Tile 1 Data, Tile 2 Data, and Tile 3 Data in the diagram) and a header portion (Tile 0 MXScale, Tile 1 MXScale, Tile 2 MXScale, and Tile 3 MXScale), arranged sequentially. Each main block further includes eight sub-blocks; that is, the main header of each main block includes the sub-headers of eight sub-blocks (not labeled in the diagram), and the main data portion of each main block includes the sub-data portions of eight sub-blocks (SubTile 0 Data, SubTile 1 Data, SubTile 2 Data, SubTile 3 Data, SubTile 4 Data, SubTile 5 Data, SubTile 6 Data, and SubTile 7 Data in the diagram). The data portion of each sub-block occupies 384 bytes of storage, and the header portion of each sub-block occupies 16 bytes of storage. Therefore, the data portion of one SuperTile occupies 384 bytes × 8 bytes × 4 = 12 KB of storage, and the header portion of one SuperTile occupies 16 bytes × 8 bytes × 4 = 512 bytes of storage. One Tile occupies 384 bytes × 8 bytes = 3 KB of storage, the sparse mask data of one Tile occupies 3 KB / (48 bytes * 8) = 8 bytes of storage, and the sparse mask data of one SuperTile occupies 8 bytes × 4 = 32 bytes of storage. Similarly, the sparse mask data corresponding to the sparse matrix is ​​also arranged sequentially in independent memory spaces.

[0061] Step S103: Load the sparse matrix and sparse mask data into the register file, which includes a header register and a data register, on a main block basis. The register file is used to perform sparse matrix and weight matrix multiplication calculations.

[0062] As described in the above embodiments, after sparsity discrimination and bitmask encoding are completed, the embodiments of the present invention continue to perform storage scheduling control on the bitmask to ensure that it has good access efficiency and layout regularity in the main memory. In the embodiments of the present invention, all Sparse Mask data is centrally stored in an independent contiguous area, without being mixed with the data of the corresponding block, thereby simplifying the mask access logic and facilitating hardware pre-loading or compressed transmission. Each tile unit corresponds to a sparse mask, and the mask length depends on the sparse discrimination block (i.e., the number of rows) contained in the tile. For the Non-MX format, each tile corresponds to a 4-byte Sparse Mask. For the MX format, the mask size varies depending on the precision and structure: at 8-bit and 4-bit precision, the Sparse Mask of each tile is 4 bytes; while at 6-bit precision, since a single tile contains more sub-blocks, the Sparse Mask length is 8 bytes, corresponding to the number of rows of data within the tile.

[0063] In computer architecture, the register files within the on-chip Processing Unit (PU) are responsible for providing one or more operands to the computing units (Tensor Cores, etc.). Data can be loaded from memory into these register files via memory access instructions. To support efficient access and decoding of sparse activation in on-chip caches, this invention provides a unified design for the loaded register file structure. Specifically, the register file includes a header register and a data register. The header register stores necessary information such as sparse mask data and MXScale; the data register stores MX format and non-MX format data portions.

[0064] See Figure 4This is a schematic diagram of register file data arrangement provided by an embodiment of the present invention. As shown in the figure, it includes four register file data arrangement methods with four data formats: non-MX format, MX8bit (MXFP8 / MXINT8), MX4bit (MXFP4 / MXINT4), and MX6bit (MXFP6). It should be noted that in this embodiment of the present invention, the register file represents the entire register space. In specific implementations, the register file may further contain multiple register entries, each register entry corresponding to a main block. Each register entry includes a corresponding header register and a data register. This embodiment of the present invention provides a detailed description of the data arrangement based on the main block and its corresponding register entries.

[0065] In the first implementation, when the sparse matrix storage data only includes the data portion, the data portion is loaded sequentially into the data register in units of sub-blocks. A corresponding header storage area is allocated for each sub-block in the header register, and the sparse mask data of each sub-block is loaded sequentially into the corresponding header storage area. In specific implementations, the sparse mask data of the sub-blocks is preferentially loaded into the high-order bits of the corresponding header storage area, leaving the remaining bits in the header storage area empty.

[0066] like Figure 4 For non-MX formats, each tile has a logical dimension of 32×32B (32×8 elements with a 32-bit element width, 32×16 elements with a 16-bit element width, 32×32 elements with an 8-bit element width, and so on). The data portion of a tile is 1KB, completely filling the data register portion of the register unit (Tile Register 0 in the diagram). The sparse mask for each tile is 4B. Specifically, each sub-tile (8×32B, a total of 8 1×32B identification blocks) corresponds to a 1B sparse mask. In this embodiment of the invention, the header register is configured to be 128B. Since it corresponds to 4 SubTile, it is divided into 4 header storage areas, each header storage area is 32B. The 4 1B sparse masks are loaded into the corresponding header storage areas in sequence, and are loaded first into the high bit part of the corresponding header storage area. That is, each header storage area in the figure is further divided into a high bit part and a low bit part. The high bit part is 16B and the low bit part is 16B. Then the sparse mask of each SubTile (the figure shows the sparse mask of SubTile 0: 1B, which occupies 1B space) is loaded into the beginning of the high bit part 16B. The remaining 15B of the high bit part is left empty, and the low bit part 16B is left empty.

[0067] In the second implementation, when the sparse matrix storage data includes a header and a data portion, the data portion is loaded sequentially into the data register in units of sub-blocks. The header register is then divided into corresponding header storage areas for each sub-block. After partitioning the header storage areas, the header portion and sparse mask data corresponding to each sub-block are loaded separately. In specific implementations, the sparse mask data of the sub-block is preferentially loaded into the high-order bits of the corresponding header storage area, and the header portion of the sub-block is preferentially loaded into the low-order bits of the corresponding header storage area. The remaining bits in the header storage area are left empty.

[0068] like Figure 4 It corresponds to three data formats: MXFP8 / MXINT8, MXFP4 / MXINT4, and MXFP6, as detailed below:

[0069] For the MX8bit format (MXFP8 / MXINT8, etc.), each tile is 32×32 in logical dimension (including the number of data elements), divided into 4 sub-tiles, each 8×32. The data portion of each tile is 32×32B=1KB, including 4 sub-tiles, each sub-tile being 256B, continuously stored in the 1KB data register of Tile Register 0. The MXScale portion (i.e., the header portion) of each tile is 32B, with each sub-tile corresponding to 8B, stored in the header register portion of Tile Register 0. The header register portion is further divided into a high-bit portion of 16B and a low-bit portion of 16B. The MXScale of each sub-tile is added at the beginning of the low-bit portion (the figure shows the header portion of SubTile 0: 8B, occupying 8B space), leaving the remaining 8B of the low-bit portion empty. The SparseMask of a Tile is 4 bytes, and each SubTile corresponds to 1 byte, which is stored at the beginning of the high-bit portion of the header register of TileRegister 0 (the figure shows the sparse mask of SubTile0: 1 byte), with the remaining 15 bytes of the high-bit portion left blank.

[0070] For the MX4bit format (MXFP4 / MXINT4, etc.), each tile is 32×64 logical dimensions in size, divided into 4 sub-tiles, each 8×64. The data portion of the tile is 1KB, with each sub-tile being 256B, and the data register portion of TileRegister 0 is continuously stored for 1KB. The MXScale portion of the tile is 64B in total, with each sub-tile corresponding to 16B. The header register is divided into 4 header storage areas, each 32B, and the header storage area is further divided into high-bit and low-bit portions, each 16B. The MXScale of each sub-tile is stored in the low-bit portion (the figure shows the header portion of SubTile 0: 16B, occupying 16B space), and is completely filled. The SparseMask of the tile is 4B, with each sub-tile corresponding to 1B, stored at the beginning of the high-bit portion (the figure shows the sparse mask of SubTile 0: 1B, occupying 1B space), leaving the remaining 15B of the high-bit portion empty.

[0071] For the MX6bit format (MXFP6, etc.), the element width is 6 bits, not a power of 2. Each tile has a logical dimension of 32x128. One tile is completely stored using three consecutive register units (Tile Register 0, Tile Register 1, and Tile Register 2 in the diagram), fully utilizing the data registers of all three Tile Registers. At this point, the tile is divided into 8 sub-tiles, each representing an 8×64 logical dimension (384 bytes of data). The data portion of the tile is 3KB (32×128×6bit / 8bit), with each sub-tile being 384 bytes. As shown in the diagram, the data register portion of these three Tile Registers is continuously stored, filling a total of 3×1KB=3KB. The MXScale portion of the tile is 128 bytes, with each sub-tile corresponding to 16 bytes, stored in the header registers of Tile Register 0 and Tile Register 1. Similarly, the header registers are divided into four header storage areas, each 32 bytes. Each header storage area is further divided into a high-bit portion and a low-bit portion, each 16 bytes. The MXScale of each SubTile is stored in the low-bit portion (the diagram shows the SubTile 0 header: 16B, occupying 16B space), and is completely filled. The Tile Sparse Mask is 8B in total, with 1B corresponding to each SubTile, stored at the beginning of the high-bit portion of the corresponding storage area of ​​Tile Register 0 and Tile Register 1 (the diagram shows the SubTile 0 sparse mask: 1B, occupying 1B space), leaving the remaining 15B of the high-bit portion blank. The header register of Tile Register 2 is left blank as it is not used.

[0072] In summary, this embodiment of the invention divides the header register of the register unit into four 32-byte regions, each corresponding to one SubTile. Each 32-byte region is further divided into a high 16-byte and a low 16-byte region. The low 16-byte region is used for storing data such as MXScale, and the high 16-byte region is used for storing data such as SparseMask. This embodiment of the invention is compatible with multiple data formats, even when additional data such as MXScale and Sparse Mask are present, ensuring compatibility while reducing hardware overhead such as selectors. Moreover, through the unified design of the header register structure, it supports the storage of sparse information of multiple data formats, while ensuring that MXScale and Sparse Mask are stored and accessed in the same space, thus reducing storage overhead and decoding complexity. During scheduling, the array can directly read the bitmask information of the header register, skipping the loading and accumulation of all-zero blocks at the block level, significantly reducing unnecessary computation and memory access overhead.

[0073] Additionally, the register file in the above embodiment is used to perform multiplication calculations of the sparse matrix and the weight matrix, see [link to documentation]. Figure 5This is a sparse matrix multiplication computation architecture diagram provided by an embodiment of the present invention. As shown in the figure, the sparse matrix multiplication computation supports activation matrices as sparse matrices (Sparse Activation Tensors), weight matrices as normal dense matrices (DenseWeight Tensors), and output matrices as normal dense matrices (Dense Output Tensors). The sparse activation matrix consists of sparse mask data (Activation Sparse Mask) and sparse matrices (Activation Tensors), corresponding one-to-one in units of tiles. These matrices are copied into memory before computation. During execution, the Activation Sparse Mask (the sparse mask data corresponding to the sparse matrix main block) of the Activation Tile (sparse matrix main block) in memory is loaded into the header register of the Tile Register within the Processing Unit (PU). The header portion of the main block tile (including MXScale in MX format) in memory is loaded into the header register of the same Tile Register, and the data portion (MX or Non-MX) of the main block is loaded into the data register of the same Tile Register. Similarly, the main tiles of the weight matrix (Weight Tensor) in memory are loaded into another tile register. The matrix multiplication instruction controls the sparse tensor core to perform sparse matrix multiplication calculations using the two tile registers mentioned above as input operands and another tile register as output operands. This produces dense main tiles of the output matrix (typically with data precision such as FP32 / BF16) and saves them into the tile register. These can be written back to memory via the tile store, ultimately yielding the output matrix (Output Tensor), which contains the calculation results of multiple tiles.

[0074] As can be seen from the description of the above embodiments, the sparse identification and scheduling method provided by the embodiments of the present invention obtains a sparse matrix, which is continuously arranged in memory in units of combined blocks, wherein each combined block includes multiple main blocks and each main block includes multiple sub-blocks; the sparse matrix is ​​divided into multiple identification blocks in units of a preset data length; when all elements in an identification block are zero, the identification block is determined to be sparse; when at least one non-zero element exists in an identification block, the identification block is determined to be valid; the determination result is continuously stored in memory space independent of the sparse matrix in units of bits to form sparse mask data; the sparse matrix and sparse mask data are stored in units of main blocks. The data is loaded into a register file, which includes a header register and a data register, for performing sparse matrix and weight matrix multiplication calculations. This includes: when the sparse matrix storage data only includes the data portion, the data portion is loaded sequentially into the data register in sub-blocks; a corresponding header storage area is partitioned in the header register for each sub-block; and the sparse mask data of each sub-block is loaded sequentially into the corresponding header storage area. When the sparse matrix storage data includes both a header and a data portion, the data portion is loaded sequentially into the data register in sub-blocks; a corresponding header storage area is partitioned in the header register for each sub-block; and after partitioning the header storage area, the corresponding header portion and sparse mask data of each sub-block are loaded separately. This invention establishes a unified granularity activation sparse format: it proposes an activation sparse judgment granularity based on 1×32 bytes. This granularity can achieve element boundary alignment under mainstream data formats such as FP8, FP16, and INT4, while also being compatible with non-power-two data formats such as MXFP6. It adapts to multi-precision scenarios and precisely matches the width of the computing array, contributing to efficient hardware scheduling and computation. Furthermore, by employing a bitmask compression encoding method suitable for activation tensors, and a block-level compression approach based on bitmasks, this invention can adapt to dynamically generated sparse activation structures. It also stores the activation data along with scaling factors present in the MX data format in the on-chip register file, achieving joint storage, unified loading, and skip scheduling of activation data, thus improving the utilization rate of sparse activation during the execution phase. In addition, this invention features an activation-sparse-friendly storage and cache layout structure. Addressing the skip execution requirement of sparse activation, it constructs a memory layout that physically separates the mask and data, and uniformly stores the bitmask and scale in the on-chip cache, further improving access efficiency.

[0075] Through the description of the above method embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0076] This invention provides a non-volatile computer storage medium storing computer-executable instructions that can execute the sparse identification and scheduling method in any of the above method embodiments.

[0077] Corresponding to the sparse identification and scheduling method embodiment provided by the present invention, the present invention also provides a sparse identification and scheduling device.

[0078] See Figure 6 Figure 1 is a schematic diagram of a sparse identification and scheduling device provided in an embodiment of the present invention. As shown in the figure, the device includes:

[0079] The acquisition module 11 is used to acquire a sparse matrix, which is arranged continuously in memory in units of combinatorial blocks, wherein each combinatorial block includes multiple main blocks and each main block includes multiple sub-blocks;

[0080] The judgment module 12 is used to divide the sparse matrix into multiple recognition blocks in units of preset data length. When all elements in the recognition block are zero, the recognition block is judged as sparse; when there is at least one non-zero element in the recognition block, the recognition block is judged as valid; the judgment result is stored continuously in memory space independent of the sparse matrix in units of bits to form sparse mask data.

[0081] Loading module 13 is used to load sparse matrix and sparse mask data into a register file in units of main blocks. The register file includes a header register and a data register, and is used to perform sparse matrix and weight matrix multiplication calculations, including:

[0082] When the sparse matrix storage data only includes the data part, the data part is loaded into the data register in units of sub-blocks. The header register is divided into a corresponding header storage area for each sub-block, and the sparse mask data of each sub-block is loaded into the corresponding header storage area in sequence.

[0083] When the sparse matrix stores data including a header and a data portion, the data portion is loaded into the data register sequentially in units of sub-blocks. The header register is divided into a corresponding header storage area for each sub-block. After the header storage area is divided, the header portion and sparse mask data corresponding to the sub-block are loaded respectively.

[0084] Optionally, the judgment module 12 divides the sparse matrix into multiple recognition blocks based on a preset data length, including:

[0085] When the element bit width of the sparse matrix storage data is a power of 2, the preset first data length is 1×32 bytes, and the sparse matrix is ​​divided into multiple recognition blocks according to the first data length.

[0086] When the element width of the sparse matrix storage data is not a power of 2, the second data length is preset to 1×48 bytes, and the sparse matrix is ​​divided into multiple recognition blocks according to the second preset data length.

[0087] Optionally, when the sparse matrix storage data only includes the data portion, the loading module 13 loads the data portion sequentially into the data register in units of sub-blocks. In the header register, a corresponding header storage area is allocated for each sub-block, and the sparse mask data of each sub-block is sequentially loaded into the corresponding header storage area, including:

[0088] The sparse mask data of the sub-block is loaded first into the high bits of the corresponding header storage area, and the remaining bits of the header storage area are left empty.

[0089] Optionally, when the sparse matrix storage data includes a header and a data portion, the loading module 13 loads the data portion sequentially into the data register in units of sub-blocks. The header register is divided into corresponding header storage areas for each sub-block. After partitioning the header storage areas, the header portion and sparse mask data corresponding to each sub-block are loaded separately, including:

[0090] The sparse mask data of the sub-block is loaded first into the high-bit portion of the corresponding header storage area, and the header portion of the sub-block is loaded first into the low-bit portion of the corresponding header storage area. The remaining bits in the header storage area are left empty.

[0091] Figure 7 This is a schematic diagram of the hardware structure of an electronic device that performs a sparse identification and scheduling method according to an embodiment of the present invention, as shown below. Figure 7 As shown, the device includes:

[0092] One or more processors 710 and memory 720, Figure 7 Take the 710 processor as an example.

[0093] The device for performing the sparse identification scheduling method may further include an input device 730 and an output device 740.

[0094] The processor 710, memory 720, input device 730, and output device 740 can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0095] The memory 720, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the sparse identification and scheduling method in this embodiment of the invention (e.g., attached...). Figure 6 The acquisition module 11, judgment module 12, and loading module 13 are shown. The processor 710 executes various functional applications and data processing of the server by running non-volatile software programs, instructions, and modules stored in the memory 720, thereby implementing the sparse identification and scheduling method of the above method embodiment.

[0096] The memory 720 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the use of the sparsely identified and scheduled processing device. Furthermore, the memory 720 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 720 may optionally include memory remotely located relative to the processor 710, and these remote memories may be connected to the sparsely identified and scheduled processing device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0097] Input device 730 can receive input digital or character information, as well as generate key signal inputs related to user settings and function control of the sparse recognition scheduling processing device. Output device 740 may include display devices such as a display screen.

[0098] The one or more modules are stored in the memory 720, and when executed by the one or more processors 710, they execute the sparse identification scheduling method in any of the above method embodiments.

[0099] The above-described product can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in the embodiments of the present invention.

[0100] The electronic devices of this invention exist in various forms, including but not limited to:

[0101] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0102] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0103] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes: audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0104] (4) Server: A device that provides computing services. The components of a server include a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0105] (5) Other electronic devices with data interaction functions.

[0106] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0107] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0108] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0109] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A sparse identification and scheduling method, characterized in that, include: Obtain a sparse matrix, which is arranged continuously in memory in units of combinatorial blocks, wherein each combinatorial block includes multiple main blocks and each main block includes multiple sub-blocks; The sparse matrix is ​​divided into multiple recognition blocks with a preset data length. When all elements in a recognition block are zero, the recognition block is judged as sparse. When there is at least one non-zero element in the recognition block, the recognition block is judged as valid. The judgment results are stored continuously in memory space independent of the sparse matrix in bits to form sparse mask data. The sparse matrix and sparse mask data are loaded into a register file, which includes a header register and a data register, on a main block basis. This register file is used to perform sparse matrix and weight matrix multiplication calculations, including: When the sparse matrix storage data only includes the data part, the data part is loaded into the data register in units of sub-blocks. The header register is divided into a corresponding header storage area for each sub-block, and the sparse mask data of each sub-block is loaded into the corresponding header storage area in sequence. When the sparse matrix stores data including a header and a data portion, the data portion is loaded into the data register sequentially in units of sub-blocks. The header register is divided into a corresponding header storage area for each sub-block. After the header storage area is divided, the header portion and sparse mask data corresponding to the sub-block are loaded respectively. The header is divided into scaling factors in MX format.

2. The sparse identification and scheduling method according to claim 1, characterized in that, The process of dividing the sparse matrix into multiple recognition blocks based on a preset data length includes: When the element bit width of the sparse matrix storage data is a power of 2, the preset first data length is 1×32 bytes, and the sparse matrix is ​​divided into multiple recognition blocks according to the first data length. When the element width of the sparse matrix storage data is not a power of 2, the second data length is preset to 1×48 bytes, and the sparse matrix is ​​divided into multiple recognition blocks according to the second data length.

3. The sparse identification and scheduling method according to claim 1, characterized in that, When the sparse matrix storage data only includes the data portion, the data portion is loaded into the data register sequentially in sub-blocks. A corresponding header storage area is allocated for each sub-block in the header register, and the sparse mask data of each sub-block is loaded into the corresponding header storage area sequentially, including: The sparse mask data of the sub-block is loaded first into the high bits of the corresponding header storage area, and the remaining bits of the header storage area are left empty.

4. The sparse identification and scheduling method according to claim 1, characterized in that, When the sparse matrix storage data includes a header and a data portion, the data portion is loaded sequentially into the data register in units of sub-blocks. The header register is divided into corresponding header storage areas for each sub-block. After partitioning the header storage areas, the header portion and sparse mask data corresponding to each sub-block are loaded separately, including: The sparse mask data of the sub-block is loaded first into the high-bit portion of the corresponding header storage area, and the header portion of the sub-block is loaded first into the low-bit portion of the corresponding header storage area. The remaining bits in the header storage area are left empty.

5. A sparse identification and scheduling device, characterized in that, include: The acquisition module is used to acquire a sparse matrix, which is arranged continuously in memory in units of combinatorial blocks, wherein each combinatorial block includes multiple main blocks and each main block includes multiple sub-blocks; The judgment module is used to divide the sparse matrix into multiple recognition blocks with a preset data length. When all elements in the recognition block are zero, the recognition block is judged as sparse; when there is at least one non-zero element in the recognition block, the recognition block is judged as valid. The judgment result is stored continuously in memory space independent of the sparse matrix in bits to form sparse mask data. The loading module is used to load sparse matrix and sparse mask data into a register file, which includes a header register and a data register, on a main block basis. This register file is used to perform sparse matrix and weight matrix multiplication calculations, including: When the sparse matrix storage data only includes the data part, the data part is loaded into the data register in units of sub-blocks. The header register is divided into a corresponding header storage area for each sub-block, and the sparse mask data of each sub-block is loaded into the corresponding header storage area in sequence. When the sparse matrix stores data including a header and a data portion, the data portion is loaded into the data register sequentially in units of sub-blocks. The header register is divided into a corresponding header storage area for each sub-block. After the header storage area is divided, the header portion and sparse mask data corresponding to the sub-block are loaded respectively. The header is divided into scaling factors in MX format.

6. The sparse identification and scheduling device according to claim 5, characterized in that, The judgment module divides the sparse matrix into multiple recognition blocks based on a preset data length, including: When the element bit width of the sparse matrix storage data is a power of 2, the preset first data length is 1×32 bytes, and the sparse matrix is ​​divided into multiple recognition blocks according to the first data length. When the element width of the sparse matrix storage data is not a power of 2, the second data length is preset to 1×48 bytes, and the sparse matrix is ​​divided into multiple recognition blocks according to the second data length.

7. The sparse identification and scheduling device according to claim 5, characterized in that, When the sparse matrix storage data only includes the data portion, the loading module loads the data portion sequentially into the data register in units of sub-blocks. In the header register, a corresponding header storage area is allocated for each sub-block, and the sparse mask data of each sub-block is sequentially loaded into the corresponding header storage area, including: The sparse mask data of the sub-block is loaded first into the high bits of the corresponding header storage area, and the remaining bits of the header storage area are left empty.

8. The sparse identification and scheduling device according to claim 5, characterized in that, When the sparse matrix storage data includes a header and a data portion, the loading module loads the data portion sequentially into the data register in units of sub-blocks. The header register is divided into corresponding header storage areas for each sub-block. After partitioning the header storage areas, the corresponding header portion and sparse mask data for each sub-block are loaded, including: The sparse mask data of the sub-block is loaded first into the high-bit portion of the corresponding header storage area, and the header portion of the sub-block is loaded first into the low-bit portion of the corresponding header storage area. The remaining bits in the header storage area are left empty.

9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the sparse identification scheduling method as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the steps of the sparse identification scheduling method as described in any one of claims 1 to 4.

Images